Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Psieye]

I'm demoting this to -Science- until you get your numbers straight and your procedure pegged down and documented.

That's generous. What I see here is a scrap book of someone jotting down numbers. That's not science at all until it's been properly written up. The data gathered doesn't even touch the stated aim of the endeavour.

In the interest of trying to save this thread as it was intended to be an interesting topic, here's a suggested procedure:

--------------

Aim: find out how attributes affects the distribution of quality levels in an artisan's output.

Tools used: Runesmith, Dwarf Therapist

Method: Use Runesmith to make identical clones of some artisan and have them crank out some trade goods to sample quality levels from.

- Cloning includes personality, attributes and skills. This can be achieved by editing the raws before worldgen so the distribution of personality parameters is extremely narrow.
- Have 10 clones produce 100 items (e.g. stonecraft toys) each.
- Avoid using dwarves with a preference for the item being produced.
- Perform statistics on the 10 groups of 100 results. This is now the control sample.

> Repeat experiment but with one attribute changed drastically and everything else kept the same.
> The wiki has a page on which jobs use what attributes, e.g. which jobs exercise creativity.

* Collect data from many experiments and present the final conclusion on how each attribute affects quality output.

Optional extra: apply tags so dwarves don't have to eat, drink or sleep. Time how long the cloned dwarves get through the 100 item queue.

[Girlinhat]

Actually, Runesmith allows you to read AND write attributes, so you can very directly level out the dwarves to an exact number, and equalize their skills if needed as well.

[Crustypeanut]

Pressing and Waxworking are 60 experience per-job.  Beekeeping is around the same.  I know this from my Morul II challenge, by the time he collected about 300 honeycombs he hit legendary.  Using those same 300 honeycombs, he hit legendary in both pressing and waxworking.

Plus I used DT to check out the exp after the jobs.

[Kogut]

Optional extra: apply tags so dwarves don't have to eat, drink or sleep. Time how long the cloned dwarves get through the 100 item queue.

Done, yesterday: http://www.bay12forums.com/smf/index.php?topic=94787.0

I think that [SPEED:0] will change nothing in quality of items and will allow for more tests.

[Kogut]

First really useful test:

[NO_EAT] [NO_SLEEP] [NO_DRINK] [SPEED:O] [MENT_ATT_RANGE:CREATIVITY:1000:1000:1000:1000:1000:1000:1000] [MENT_ATT_RANGE:KINESTHETIC_SENSE:1000:1000:1000:1000:1000:1000:1000] [MENT_ATT_RANGE:SPATIAL_SENSE:1000:1000:1000:1000:1000:1000:1000] [PHYS_ATT_RANGE:STRENGTH:1000:1000:1000:1000:1000:1000:1000] [PHYS_ATT_RANGE:AGILITY:1000:1000:1000:1000:1000:1000:1000] dwarves embarked with 3000 tower-cap logs

I prepared stockpiles to achieve automatic sorting.

Single dwarf, without "likes tower-cap/bin", assigned to burrow with 100 logs, without carpentry skill produces 100 wooden bins

First batch of 9 22  tests

/-/+/*/≡/☼
20/28/17/32/3/0
35/27/15/22/1/0
25/32/21/18/4/0
31/36/13/18/2/0
34/29/14/20/2/1
33/33/20/12/2/0
31/30/22/17/0/0
32/27/20/21/0/0
29/26/18/22/5/0
29/38/15/13/5/0
35/28/15/19/3/0
29/36/16/17/2/0
32/25/15/22/2/0
33/28/14/19/6/0
30/38/14/16/2/0
23/40/20/16/1/0
30/28/17/22/3/0
29/25/23/22/1/0
30/32/22/15/1/0
30/31/15/22/2/0
30/25/20/20/5/0

save, ready to run: http://dffd.wimbli.com/file.php?id=5071

Spoiler: Prepared map (click to show/hide)

[AWdeV]

A no-skill carpenter can't ever make masterful whatevers? What a surprise. How much experience did these 100 bins net, btw?

[Miuramir]

First really useful test:

It's 21 tests, and there's an error somewhere in the 13th test, it only adds up to 96.  A very simple averaging gives:

None ( ):             30.00%
Well-crafted (-):     30.57%
Finely-crafted (+):   17.43%
Superior Quality (*): 19.29%
Exceptional (≡):       2.48%
Masterwork (☼):        0.05%

The interesting thing that leaps out is that Superior (*) is actually more likely than Finely-crafted (+) in this test sample.  I'm curious as to whether this is a result of an error in the test (unlikely; it isn't the result of a single error skewing things, as just under 62% of the sub-trials had more Superior (*) than Finely-crafted (+)), the result of too few trials (possible, but things seemed to be otherwise converging reasonably well), or is telling us something we didn't know about the way quality is determined (possibly including a bug in DF). 

It would make sense for the odds to have been originally set to something like 30%, 30%, 20%, 20%-small, small, miniscule; except it looks like if something along those lines was done, the chance for the high end results was pulled out of Finely-crafted (+) rather than Superior (*) as would have been the typical sort of RPG progression. 

The fact that there was a single Masterwork (☼) crafted is also interesting, although the odds are probably no better than 1/1000. 

It's not clear to me from the test description whether the dwarf could gain skill during the 100 bin runs; if skill gain happened, then the discontinuity is probably the result of changes in the output likelihood with increased skill.  What we probably need, if this isn't already that, is to collect statistics on runs just short of the amount needed to level, manually produce one or a few more un-counted ones to kick to the next skill level, and then start counting as a separate group until just short of the next level, etc.  (There are some other unanswered questions buried in this which the above is intended to avoid; for instance, when producing an item that will result in a skill up, is the quality rolled at the lower or higher skill chances?  We'd need the results uncontaminated by this first, and then a very large number of only skill-up-items recorded, at a level where the chances on either side are distinguishable; not really practical.)

[Kogut]

96 bins was due to "OHMYGODBLIZZARDMANEVERYBODYISSCARED"
Yes, dwarf started with 0XP and finished test with 6000XP (Adept), it may be better to make 10000 games and make single bin in every single one (or maybe run DFHACK and constantly set XP to 0?). But it requires automation and my main goal is to compare quality in various situations, not measure it.

New test, stats as above (SPEED:0, no nees, starts with 0 XP) and likes bins (BIN LOVER from published save). Results are really interesting.

4   19   21   36   19   1
1   14   14   44   25   2
5   13   24   40   17   1
6   15   19   41   17   2
3   19   26   33   17   2
1   15   23   32   26   3
2   18   19   39   19   3
2   13   23   39   22   1
3   15   18   41   21   2
2   17   15   40   25   1
3   19   19   30   28   1
3   14   24   32   25   2
2,92%   15,92%   20,42%   37,25%   21,75%   1,75%

[Girlinhat]

An overall more direct means of testing would be to set up a series of wortkshops, each set to accept a different level of skill only.  Set them all on repeat.  This way you can accurately judge the quality rate of different skill levels.

[Kogut]

An overall more direct means of testing would be to set up a series of wortkshops, each set to accept a different level of skill only.  Set them all on repeat.  This way you can accurately judge the quality rate of different skill levels.

Hmm, good idea.

New test, now on legendary+5 skill level, single run

Standard dwarf, legendary+5, 1000 creativity
8   10   11   43   1030   498
0,50%   0,63%   0,69%   2,69%   64,38%   31,13%   100,00%
                  
Standard dwarf, legendary+5, 5000 creativity
1   3   10   25   1051   511
0,06%   0,19%   0,62%   1,56%   65,65%   31,92%   

So creativity is not important.

[Shinotsa]

I wouldn't necessarily say that it isn't important since  the 5000 creativity dwarf crafted far fewer low quality items. If we were to cut out the last two groupings and analyze the data just based on how many low quality items were crafted there would be a fairly large statistical difference. Another line of study might be how does a 0 or other low creativity dwarf fare? I can't say I'm terribly familiar with runesmith or the numerical ratings of stats, so forgive me if 1000 is the base creativity rating.

Good job coming to us with the first batch before you continued though. No writer publishes before being edited and reviewed, so we might turn this -science- into ☼Science☼ yet. Just watch out for necro, he'll turn it into ‼SCIENCE‼ 

[Kogut]

I wouldn't necessarily say that it isn't important since  the 5000 creativity dwarf crafted far fewer low quality items.

Well, I want masterwork bed for everybody. Next batch:

max carpenter level, 500 creativity and likes bins

0   0   2   11   1054   534   
0,00%   0,00%   0,12%   0,69%   65,83%   33,35%   

[Girlinhat]

I wouldn't necessarily say that it isn't important since  the 5000 creativity dwarf crafted far fewer low quality items.

Lrn2Anylize.  That "far fewer" is 25 compared to 43.  That's 18 less on the widest variant.  A dwarf with 5 times more creativity gathered a .44% drop in lower quality production.  I cannot stress how extremely marginal this is, and how likely it is to be a fluke.  The crafter's mood can have an impact as well.  The max carpenter producing +bins+ could ONLY happen because he was hungry/thirsty/sleepy, as otherwise they never naturally produce items so low in quality.

So accounting for some mood variance, I feel it's very safe to say that there's no difference.  Only two runs were done, proper scientific method would demand a dozen tries at least to get good averages.  And even if there were a difference, it's so slight as to be non-existent.

[Kogut]

hungry/thirsty/sleepy

I modded in [NO_SLEEP][NO_EAT][NO_DRINK] and removed alcoholism. And 5000 creativity is extremely rare, typical values are up to to 2500.

[Girlinhat]

Ah, well then yeah, I'd count it for just regular number variances.  It's much much too small to be relevant.  It's less than one half of one percent.