SimulationCraft Model Development - Page 6

dedmonwakeen · 11/06/08, 1:34 PM

Originally Posted by erragal

Dedemonwakeen: I'd suggest running/showing balance numbers without faerire fire. In a balanced raid there should be no reason to cast it, and the DPS hit is fairly significant. For the sample output at least, it'd be appropriate to show the maximum potential DPS similar to affliction locks with and without CoE. It'll make for good comparison in non-feral raids with shadow priests to see whether affliction locks or balance druids take more of a DPS hit by applying the minor armor debuff. Starfire being better than wrath is probably more related to the heavy penalty you get from latency when chain casting wrath.

That's a good point. I'll add a NoFF profile.

I have people waiting in the wings to get started on Druid-Feral, Rogue, Hunter.... waiting for me to put together a template for them. (Actually, Hunter got tired of waiting..... and is already started.) Hopefully, we'll be able to make the Boomkin-vs-Feral tradeoffs soon.

Roywyn · 11/06/08, 7:35 PM

Felguard

Felguard spec DPS is very lacking in the sims, some bugs I've found:
* Pet is unaffected by Windfury Totem (Should be 20% melee haste)
* Effect of Strength Totem looks low to me. Probably correct.
* Strength Totem doesn't add any crit (it should via Agi; unknown Agi:crit?)
* Possible to give the pet another 5% crit (LotP/Rampage)?

dedmonwakeen · 11/06/08, 7:46 PM

Originally Posted by Roywyn

Felguard

Felguard spec DPS is very lacking in the sims, some bugs I've found:
* Pet is unaffected by Windfury Totem (Should be 20% melee haste)
* Effect of Strength Totem looks low to me. Probably correct.
* Strength Totem doesn't add any crit (it should via Agi; unknown Agi:crit?)
* Possible to give the pet another 5% crit (LotP/Rampage)?

Thanks! I've been tearing my hair out over the physical dps.... and you pointed me the perfect direction.

There is a MAJOR problem with haste and attacks in general..... bah, I suck.

Also, it looks like I did not define the Agi:Crit ratio for pets.......... bah, i really suck.

I knew it was time to start giving the melee mechanics some lovin.... No time like the present!

dedmonwakeen · 11/06/08, 10:03 PM

Originally Posted by Roywyn

Felguard
* Strength Totem doesn't add any crit (it should via Agi; unknown Agi:crit?)

Know any Warlocks and Shaman/Paladin who can buff the pet and check out the change in crit stats?

I really have no idea what ratio to use...... For now I'll go with the Warlock ratio.

* Possible to give the pet another 5% crit (LotP/Rampage)?

Forgot to mention: This is already implemented.... I just forgot to put it in the config file. Just add "leader_of_the_pack=1" after each player/pet of interest.

EDIT: Bah.... Well.... I know I never tested this because the released version of the code looks for leader_of_the_pact.....

dedmonwakeen · 11/07/08, 11:35 PM

SimulationCraft r1073 available for download..... SampleOutput has been updated.

A few minor tweaks to modeling.... Revamp of RNG system to improve multi-thread performance.

Kalle · 11/10/08, 8:13 AM

You seem to have a bug somwhere ...

The variance I see when running simcraft multiple times is significantly larger than the predicted error. I tested this with 10 runs of simcraft, 10,000 iterations per run. The deviation I see is larger than the computed error (yes, I divided your error by 2 to get std_dev/sqrt(iterations) ) for all of all raid_70.txt's default players.

I believe this only happens since you introduced multi-threading. However, compiling with -DNO_THREADS doesn't solve the problem.

Genre · 11/10/08, 8:48 AM

Originally Posted by Troffel

The central limit theorem is not applicable, because the conditions a not satisfied.

Mainly, I expect a not necessary a Gaussian distribution, because the random variables are not (in general) independent. It has to be proven that the distributions are Gaussian and this will be a interesting fact.

This would be very similar to stochastic simulations in finance. Basically you are running a multitude of scenarios (say 10000) of 3 minutes fights, pick the worst 10% and find an average DPS for these. So instead of take a blunt average, we would try to find which spec/gear/whatever gives the best worst case scenario. At some point, you have to ask yourself: is higher average DPS better then more stable DPS?

As mentionned above, this is usually how finance people run the liabilities on fund guarantees (spelling on that); for the math savvy the 10% pick would be called CTE(90).

dedmonwakeen · 11/10/08, 10:04 AM

Originally Posted by Kalle

You seem to have a bug somwhere ...

The variance I see when running simcraft multiple times is significantly larger than the predicted error. I tested this with 10 runs of simcraft, 10,000 iterations per run. The deviation I see is larger than the computed error (yes, I divided your error by 2 to get std_dev/sqrt(iterations) ) for all of all raid_70.txt's default players.

I believe this only happens since you introduced multi-threading. However, compiling with -DNO_THREADS doesn't solve the problem.

When it "worked right" were you ever running with multiple threads? (The sfmt package was not previously thread-safe.)

Can you try something: Re-run the experiment with "sfmt=0" in the config file. This will use the libc rand() function. Run with "threads=0" or simply not set because the mutex locking in rand() destroys any benefit you'll get from multiple threads.

I stole the guts of sfmt (with proper acknowledgment of credit, ofc) and made it thread-safe.

I'm not sure if our error-analysis is/was off because the rng package was behaving oddly..... but more unfortunately, in cases like this "more variance" is probably the "more correct" scenario.

Let me know how the rand() experiment goes. I'll do some research on how our "error calculation" is performed. BTW: I welcome your input on this. I make no claims to be being a statistics expert.

Eleven · 11/11/08, 2:22 PM

Are there any plans for a resource timeline? I know I would find this very useful.

dedmonwakeen · 11/11/08, 3:22 PM

Originally Posted by Eleven

Are there any plans for a resource timeline? I know I would find this very useful.

To be honest, I hadn't even thought about it..... but that's a good idea. Roywyn has been asking about reporting the amount of mana left when the target dies and I wasn't sure where I was going to put it..... This expands on that idea very nicely.

I'd promised myself to stop messing around with the existing architecture so that I could focus on adding more class support..... but this one intrigues me.

FYI..... The current DPS timeline is average of all iterations. I would do the same for monitoring resource levels.

dedmonwakeen · 11/12/08, 5:51 PM

SimulationCraft r1100 is available for download.

Full list of modeling tweaks can be found at RecentUpdates - simulationcraft - Google Code

Sample output can be found at SampleOutput - simulationcraft - Google Code

New charts include resource timelines and dps distributions, while the dps timeline has been made easier to read.

Binkenstein · 11/16/08, 7:41 PM

Are you still using your modified Elemental settings, or are they as they are in-game?

dedmonwakeen · 11/17/08, 10:47 AM

Originally Posted by Binkenstein

Are you still using your modified Elemental settings, or are they as they are in-game?

Could you elaborate? I've been a bit distracted lately so I may have missed a key modeling discussion.

If SimulationCraft does not model in-game behavior it is usually due to the following reasons:

(1) The simulator is bugged.
(2) The simulator is executing as intended..... but the author is unaware of a modeling change.
(3) The in-game mechanics do not match tooltip/intent and the simulator chose to model the intended behavior as opposed to the actual behavior.

Which of those categories apply here?

Binkenstein · 11/17/08, 2:30 PM

You changed Chain Lightning to put all jumps into the single target once, just wondering if that is still the case.

dedmonwakeen · 11/17/08, 10:42 PM

Originally Posted by Binkenstein

You changed Chain Lightning to put all jumps into the single target once, just wondering if that is still the case.

Ah.... that. Actually, I implemented it as a "glyph" so that it could be optional.

However..... I just peeked at the code and it looked like I removed that functionality...... I didn't want to give anybody the wrong idea......