SimulationCraft Model Development - Page 5

dedmonwakeen · 12/02/08, 2:35 PM

Althor has added support for more flasks and food...... but I haven't been pushing it too much, because I really want to pull the flask/elixir/food/weapon_buff directives off of the action list. But given just how LONG I've been wanting to see that happen.... I really need to get my butt in gear.

The "enchant_" (and flask and food and trinket proc and etc) aren't really necessary..... you can code them all up into "gear_".

But..... someday I would really like to have a tool to rip armory profiles and generate SimulationCraft configs. In order to make that as easy as possible, I've tried to be as "granular" as possible with the character definition.

BTW... Althor has also coded up many of the interesting caster trinkets. In general the format is: Take the trinket name, remove punctuation/apostrophes/etc and replace spaces with underscores.

So.....
extract_of_necromantic_power=1
sundial_of_the_exiled=1
etc

I'll ask him nicely to document them in the "HowToRun" wiki.

Starfox · 12/02/08, 2:44 PM

Originally Posted by Roywyn

You enter the stats purely from gear, and from outside sources.
I.e. you add food buffs and average trinkets/item proc, but you don't add flask (the sim drinks a defines flask) or totems or auras (the sim casts totems and applies auras).

For the hit rating:
The default gear set has 11% hit rating, so that you cap with 3% misery and a 3% talent that most classes have.
Druids have a 4% hit talent, so they only need 10% hit from gear.
The sim then pretends that the unneeded 25 hit rating on gems are replaced by 25 haste rating.

Ah thanks for the clarifications, just tried a bit around with the sim, really nice i have to say.

SageoftheTimes · 12/05/08, 12:41 AM

Based on this post from the Affliction Rotations Thread, wouldn't it be better to have the Affliction Model(s) use Drain Life for it's higher DPS, as well as fishing for Nightfall Procs?

Morrigan · 12/05/08, 2:54 AM

Originally Posted by SageoftheTimes

Based on this post from the Affliction Rotations Thread, wouldn't it be better to have the Affliction Model(s) use Drain Life for it's higher DPS, as well as fishing for Nightfall Procs?

No, because the assumptions made in this posting are most probably not correct. As far as I remember, Drain Soul/Life do not benefit from the 20% Haunt bonus (they do, however, benefit from it through Soul Siphon, but that's not nearly as much), so Shadow Bolts as filler should be better before 25%.

Koshir · 12/05/08, 6:24 AM

I started simulation several times with the same data and got:
1st:
Warlock_2_13_56 intellect=0.39 spirit=0.45 spell_power=1.16 hit_rating=1.64 crit_rating=0.53 haste_rating=0.90
2nd:
Warlock_2_13_56 intellect=0.23 spirit=0.38 spell_power=0.93 hit_rating=1.76 crit_rating=0.45 haste_rating=0.77
3rd:
Warlock_2_13_56 intellect=0.31 spirit=0.45 spell_power=1.01 hit_rating=1.81 crit_rating=0.59 haste_rating=0.85

It looks like scaling factor is random number. I think it would be useful get true (average) scaling every try

dedmonwakeen · 12/05/08, 8:23 AM

Originally Posted by Koshir

I started simulation several times with the same data and got:
1st:
Warlock_2_13_56 intellect=0.39 spirit=0.45 spell_power=1.16 hit_rating=1.64 crit_rating=0.53 haste_rating=0.90
2nd:
Warlock_2_13_56 intellect=0.23 spirit=0.38 spell_power=0.93 hit_rating=1.76 crit_rating=0.45 haste_rating=0.77
3rd:
Warlock_2_13_56 intellect=0.31 spirit=0.45 spell_power=1.01 hit_rating=1.81 crit_rating=0.59 haste_rating=0.85

It looks like scaling factor is random number. I think it would be useful get true (average) scaling every try

It is simulation...... so there are LOTS of random numbers.......

In general, I would not run with less than 10000 iterations when calculating scale factors. The default number of iterations is small (only 1000) because I don't want first-time users to think the tool is hung. Remember that most modern processors are dual-core, so feel free to add "threads=2" to the command line to speed things up.

Starfox · 12/05/08, 11:20 AM

Originally Posted by dedmonwakeen

It is simulation...... so there are LOTS of random numbers.......

In general, I would not run with less than 10000 iterations when calculating scale factors. The default number of iterations is small (only 1000) because I don't want first-time users to think the tool is hung. Remember that most modern processors are dual-core, so feel free to add "threads=2" to the command line to speed things up.

hehe, yea. like Yo!-simulator was for enhancement shamans in TBC.
DPS values where pretty solid for a few hundred hours of simulation, but for ratings you had to set it to serveral thousands because variance can be a bitch

edit:
Found a bug in the calcuation of the manapool, at least for moonkins
I've got 805 int in casterform unbuffed, with 15291 mana.
Buffs: Int +60, MotW +51, Furor +10%, BoK +10%
Which should put me at ~1108, or 303 intellect more when buffed, which evaluates to a increase in mana by ~4545.
But the charts resource timeline shows me a max value of 23349 mana, which is quite a difference to the expected 19841.
Are there buffs i forgot to take into account? I'm missing ~193 int to reach the simulators value of mana

dedmonwakeen · 12/05/08, 2:22 PM

Originally Posted by Starfox

Found a bug in the calcuation of the manapool, at least for moonkins
I've got 805 int in casterform unbuffed, with 15291 mana.
Buffs: Int +60, MotW +51, Furor +10%, BoK +10%
Which should put me at ~1108, or 303 intellect more when buffed, which evaluates to a increase in mana by ~4545.
But the charts resource timeline shows me a max value of 23349 mana, which is quite a difference to the expected 19841.
Are there buffs i forgot to take into account? I'm missing ~193 int to reach the simulators value of mana

Perhaps the difference is "base" mana?

The "gear_intellect" setting should be ONLY that intellect that comes from gear.

Is it possible to tell that from the in-game character profile?

Erdluf · 12/05/08, 2:48 PM

Originally Posted by dedmonwakeen

Perhaps the difference is "base" mana?

The "gear_intellect" setting should be ONLY that intellect that comes from gear.

Is it possible to tell that from the in-game character profile?

I'm pretty sure the character profiler will show (Base + Other) for Intellect. Unbuffed, in caster form, all of the "other" intellect should be from gear.

I think that the only Moonkin stats that you might want a calculator for are spellpower-from-gear (Lunar Guidance and IMkF) MP5-from-gear (Dreamstate and Intensity), and deciding how to value most on-use or on-proc effects.

Starfox · 12/05/08, 10:23 PM

Originally Posted by Erdluf

I'm pretty sure the character profiler will show (Base + Other) for Intellect. Unbuffed, in caster form, all of the "other" intellect should be from gear.

I think that the only Moonkin stats that you might want a calculator for are spellpower-from-gear (Lunar Guidance and IMkF) MP5-from-gear (Dreamstate and Intensity), and deciding how to value most on-use or on-proc effects.

Ah ok, simulating again using only the stats from gear for default_<stat> variables.
edit: Ok, now it's only off by ~9-10int, thx

Another question, do you let Earth&Moon and Master Shapeshifter increase the druids spellpower or increase the damage done by all spells like moonfury?

Kalle · 12/11/08, 8:13 AM

Originally Posted by dedmonwakeen

When it "worked right" were you ever running with multiple threads? (The sfmt package was not previously thread-safe.)

Can you try something: Re-run the experiment with "sfmt=0" in the config file. This will use the libc rand() function. Run with "threads=0" or simply not set because the mutex locking in rand() destroys any benefit you'll get from multiple threads.

I stole the guts of sfmt (with proper acknowledgment of credit, ofc) and made it thread-safe.

I'm not sure if our error-analysis is/was off because the rng package was behaving oddly..... but more unfortunately, in cases like this "more variance" is probably the "more correct" scenario.

Let me know how the rand() experiment goes. I'll do some research on how our "error calculation" is performed. BTW: I welcome your input on this. I make no claims to be being a statistics expert.

I finally had time to do some additional testing. sfmt=0 doesn't change anything. But the problem seems to exist for some time, I tested with r750 and it also showed the increased variance.

The first test I then did, was comparing the distribution of dps between one run with x iterations and x runs with 1 iteration. They were identical (apart from some rather small statistical fluctuations).

So I tested the actual scaling of the error simcraft reports (on the example of Warlock_7_0_54, r1073) and got this diagramm (I hope the link is ok... I can't check it while not at home). The red "points" is my empirical data (with 1-sigma errorbars). The green graph is the ~ 1/sqrt(iterations) scaling that we assumed for statistically independent iterations. The blue graph is a fit for a model that includes the possibility of correlated iterations (I used the formula from Standard error (statistics) - Wikipedia, the free encyclopedia). I got a sample bias coefficient of approx. 0.0011 (1.0 would be perfect correlation, 0.0 statistical independence).

So the conclusion seems to be that iterations are not fully independent. Currently I have no idea of what could be the cause of this.

The effect is rather catastrophic. At 10,000 iterations the error is ~3.5 times as large as we currently expect; at 100,000 it's 10.5 times (if extrapolation of the fittet function yields correct results).

dedmonwakeen · 12/12/08, 11:57 AM

I'm just getting back from vacation.... I'll jump on this ASAP once I get up and running.

(Currently stuck at the in-laws: Came back from vacation to an ice storm.... No power at home......)

One thing to note is that I have integrated the sfmt code, so I can't turn it off. I'll have to remove the sfmt option.

A couple questions jump to mind:

(1) me: Perhaps I should reseed the sfmt package with a call to rand() before each iteration?

(2) me: Perhaps the depth of my "pseudo-random" sequence is not large enough? I can re-implement the sfmt package with a much larger depth.

(3) you: What version of code? Did you build it yourself? If so, what compiler?

(4) you: What platform are you running on?

(5) you: Have you tried this analysis with any other implemented classes?

Kalle · 12/13/08, 8:48 AM

(1) If the period of the random number generator is not exceeded that should have no positive effect. Everything what you want is to have a "random" starting point in the generator's sequence.

(2) If you mean the length of the generator's period, then there should be no problem. According to the source the period of sfmt is 2^1279 - 1 > 10^385.

(3) I used the r1073 archive from the downloads page, compiled with gcc 4.2.2.

(4) 32bit linux on c2d

(5) Not such a detailed analysis, but I noticed a larger than expexted variance for all players of the default raid_70.txt.

Another plot indicating the problem (I think): Link
It shows the absolute value of the difference between the mean and the average of dps over 'iterations' iterations (of course what I called the 'mean' is also an average, but over a much larger number of iterations). In red: x runs of simcraft with iterations=1. Green: 1 run with x iterations. The blue graph is essentially the same as the green one, but for the green graph I used the same order as simcrafts iterations, for the blue one I put the values in a random order.

So you gain precision by randomizing the order of simcrafts iterations (that is because the number of iterations is larger than the number used to compute the average). If the sequence was already random (in the sense of random sampling from the underlying probability distribution), there would be no difference.

dedmonwakeen · 12/15/08, 2:50 PM

Originally Posted by Kalle

Another plot indicating the problem (I think): Link
It shows the absolute value of the difference between the mean and the average of dps over 'iterations' iterations (of course what I called the 'mean' is also an average, but over a much larger number of iterations). In red: x runs of simcraft with iterations=1. Green: 1 run with x iterations. The blue graph is essentially the same as the green one, but for the green graph I used the same order as simcrafts iterations, for the blue one I put the values in a random order.

So you gain precision by randomizing the order of simcrafts iterations (that is because the number of iterations is larger than the number used to compute the average). If the sequence was already random (in the sense of random sampling from the underlying probability distribution), there would be no difference.

Thanks for the detailed analysis...... this is going to take me a while to digest and dig into......

dedmonwakeen · 12/15/08, 2:52 PM

SampleOutput - simulationcraft - Google Code has been updated for patch 3.0.8

A new download is not yet available...... I'm still waiting on modeling details for JoW and Glyph of Arcane Blast. I'd also prefer to wait until the changes go live and the wowhead talent trees are updated.

dedmonwakeen · 12/18/08, 1:11 PM

SimulationCraft version r1212 for Patch 3.0.8 is available for download.

Sample output can be found here: SampleOutput - simulationcraft - Google Code

As part of the scale factor generation, I now generate lootrank/wowhead weighted links for each profile.

Grengorr · 12/18/08, 11:56 PM

I was wondering if options for the set bonuses of [Valorous Plagueheart Circlet] were in the pipeline? I had a look through the warlock source file, but it doesn't appear to be implemented yet.

Also, I used to get crazy numbers for the scaling factors (particularly for mages) when running using multithreading a while ago (last time I tried was about two weeks ago), but I tried again with r1215 today and it's running great with 8 threads (the scale factors for 10000 iterations are quite close to running the same sim singlethreaded - without any actual analysis it at least appears to be within a believable amount of error). I didn't see a fix flicking over the svn logs or I would have tried earlier, but thanks for that fix

Is there a way to only calculate scaling factors for specific actors? I'm generally only interested in seeing that information for a couple of actors in the raid.

dedmonwakeen · 12/19/08, 12:25 AM

Originally Posted by Grengorr

I was wondering if options for the set bonuses of [Valorous Plagueheart Circlet] were in the pipeline? I had a look through the warlock source file, but it doesn't appear to be implemented yet.

Also, I used to get crazy numbers for the scaling factors (particularly for mages) when running using multithreading a while ago (last time I tried was about two weeks ago), but I tried again with r1215 today and it's running great with 8 threads (the scale factors for 10000 iterations are quite close to running the same sim singlethreaded - without any actual analysis it at least appears to be within a believable amount of error). I didn't see a fix flicking over the svn logs or I would have tried earlier, but thanks for that fix

Is there a way to only calculate scaling factors for specific actors? I'm generally only interested in seeing that information for a couple of actors in the raid.

We recently picked up a developer who is focusing on the Warlock class, so hopefully soon.

The multithreading-scaling bug was stealth-fixed by Althor very recently. So recently, in fact, that the tarball had the bug while the executable download (built 10min later) was clean. He snuck it in there......

(Of course, the tarball has since been updated with the fix after he chatted with me.)

You can set "quiet=1" in any player profile...... but it won't save you very much time because it only affects reporting.

I've been considering providing "default_buff" and "default_debuff" global directives. It kind of stands against the who multi-player theme of SimulationCraft, but the convenience may be worth the apostasy.......

EDIT: By the way..... If you have a google ID, feel free to register Issues at the simulationcraft googlecode site. Issues can be for both bugs as well as feature requests. Seeing that list grow is like a thorn in my side, so you have a better chance getting the code updated sooner rather than later.

dedmonwakeen · 12/19/08, 12:35 AM

Now that the scale-factor generation has been tied into lootrank/wowhead weight item searches, I'm hoping to get some feedback on whether the scale factors from the simulation match up with experienced intuition. Feedback would be most appreciated.

Mandalay · 12/20/08, 9:56 PM

Being new to simcraft I have some questions that are not answered on either the HowToUse page or in any EJ thread that I can find.

1. Why does "blessing_of_might=550", instead of "=1" like other buffs?
2. How can I find a comprehensive list of supported buffs and procs? E.g. potions/flasks/trinkets
3. Does warlock_2_13_56 conflagrate default to casting at time "immolate duration<3sec"?
4. Can I and if so how do I model specific cast sequences? E.g. warlock_2_13_56 backlash ---> chaos bolt/immolate/incinerate.
5. Are there any plans to support doomguard and infernal?

Thanks for your effort creating and improving this great tool.

Merry Christmas

dedmonwakeen · 12/21/08, 12:44 AM

Originally Posted by Mandalay

Being new to simcraft I have some questions that are not answered on either the HowToUse page or in any EJ thread that I can find.

1. Why does "blessing_of_might=550", instead of "=1" like other buffs?
2. How can I find a comprehensive list of supported buffs and procs? E.g. potions/flasks/trinkets
3. Does warlock_2_13_56 conflagrate default to casting at time "immolate duration<3sec"?
4. Can I and if so how do I model specific cast sequences? E.g. warlock_2_13_56 backlash ---> chaos bolt/immolate/incinerate.
5. Are there any plans to support doomguard and infernal?

1. No good reason....... It was one of the earlier "static" buffs I implemented before I switched to simple "boolean" options. I did not change it because I didn't feel like dealing with backward-compatibility issues AND I've been under the (foolish) assumption that Paladin support was "just around the corner".

2. simcraft.h (yes, I know how lousy of an answer that is.... )

3. yes: conglagrate_t has a conditional "ticks_lost" that defaults to 1...... but I should really add that to the config file just so that people can see it

4. there is no way at all to model a specific cast SEQUENCE...... only priority lists at the moment...... however, i have seen people ask for spell conditionals that essentially enabled them to force sub-sequences from time to time

5. it felt like an archeology dig just trying to get all the raw detail i needed for the "ordinary" pets....... adding support is very very very easy...... finding the level of detail that simulationcraft requires is very very very hard....... long story short: if you (or anybody) finds me the mechanics, it will be present in the code the next day.

BTW...... I encourage you to document bugs (and "features" that are so necessary that they might as well be bugs) at the Issues tab on the project:

Issues - simulationcraft - Google Code

This will make sure your concerns are not lost in the cracks.

Starfox · 12/22/08, 1:13 PM

Originally Posted by Mandalay

2. How can I find a comprehensive list of supported buffs and procs? E.g. potions/flasks/trinkets

I'm trying to make lists of possible options with a little description etc.
Just took the options table from player.cpp and formatted it a little bit and wrote to some options what they do.
PlayerOptions - simulationcraft - Google Code
Hope that helps a bit

If you have a Use:-trinket you can specify them easily, also the engi tinker Hyperspeed Accelerators is just haste_trinket,rating=340,length=20,cooldown=120 in the action string for example

dedmonwakeen · 01/05/09, 1:01 AM

SimulationCraft r1255 now available for download

Many thanks to Starfox for the detailed documentation and to Zakalwe for the tricky Doomguard/Infernal support!

Detail on recent updates: RecentUpdates - simulationcraft - Google Code

Lots of pretty charts: SampleOutput - simulationcraft - Google Code

Tauraherion · 01/06/09, 9:13 AM

Why is that Simcraft returns a scale factor of 0.89 for spell hit even if i set default_hit_rating=446 at the gear point parameters?

Shouldn´t it return a value of 0 for it, assuming 446 means I´m hit capped?
I´m also keeping the "scale_hit_rating=-200" portion of the scale factors parameters. Is that correct?

dedmonwakeen · 01/06/09, 12:10 PM

Originally Posted by Tauraherion

Why is that Simcraft returns a scale factor of 0.89 for spell hit even if i set default_hit_rating=446 at the gear point parameters?

Shouldn´t it return a value of 0 for it, assuming 446 means I´m hit capped?
I´m also keeping the "scale_hit_rating=-200" portion of the scale factors parameters. Is that correct?

The +/- of the scale factors matters.....

For stats that can be "capped", use negative scale deltas.

So..... In your example, you LOSE 0.89dps for every unit of Hit Rating you REMOVE from your gear.