SimulationCraft Model Development - Page 3

Tifi · 11/01/08, 5:23 AM

1. The RNG is broken, at least when USE_SFMT is undefined. At an expected resist rate of about 3% the simulation produced almost 100% resists.
Here's the correct code for using rand() in the roll function:

  if( rand() < chance*(RAND_MAX+1) ) 
  {
    return 1;
  }
  else
  {
    return 0;
  }

Please don't use % in conjunction with an RNG. Most of the time it's both unneccessary and detrimental, as it maps a uniform distribution to a non-uniform distribution.

Using rand(), recommended reading

2. My shadowfiend always does 10 swings per summon. At an attack speed of 1.5 this would require the shadowfiend expiration time to be something between 13.5 and 15, not 18.1. (Edit: According to wowhead, the tooltip actually says "Lasts 15 sec.")
Edit: Also, in report_t::print_action(), dodges and parries are not added to the miss % in the displayed results, only RESULT_MISS.

3. SWP deals the same damage whether or not shadow_weaving_wait is set. Is the current behavior of SWP not implemented?

dedmonwakeen · 11/01/08, 6:58 AM

Originally Posted by Tifi

1. The RNG is broken, at least when USE_SFMT is undefined. At an expected resist rate of about 3% the simulation produced almost 100% resists.
Here's the correct code for using rand() in the roll function:

  if( rand() < chance*(RAND_MAX+1) ) 
  {
    return 1;
  }
  else
  {
    return 0;
  }

Please don't use % in conjunction with an RNG. Most of the time it's both unneccessary and detrimental, as it maps a uniform distribution to a non-uniform distribution.

Using rand(), recommended reading

2. My shadowfiend always does 10 swings per summon. At an attack speed of 1.5 this would require the shadowfiend expiration time to be something between 13.5 and 15, not 18.1. (Edit: According to wowhead, the tooltip actually says "Lasts 15 sec.")
Edit: Also, in report_t::print_action(), dodges and parries are not added to the miss % in the displayed results, only RESULT_MISS.

3. SWP deals the same damage whether or not shadow_weaving_wait is set. Is the current behavior of SWP not implemented?

Thanks for the heads up on rand() usage. The code as originally written worked well on my linux laptop, but behaved very poorly on windows as you found out. This is why I introduced SFMT. (I actually made SFMT the default recently but that change has not yet made it into the tar.gz source download.)

SFMT was producing "more" random numbers across the entire spectrum enabling the use of the mod operator. The windows C runtime library version of rand was producing a very poor regression of values. I guess what I'm saying is that I'll certainly change the code, but my trust in windows rand() is so shattered that I'll probably stick with SFMT for a while.

Regarding shadowfiend: I could have sworn seeing a patch note about it getting increased..... but I can't find it now. He does an instant hit upon summoning. Are you including that in your "10 swings"?

Regarding SWP: In general, SimulationCraft simply will call player_buff() whenever DoT-refresh occurs. I really didn't want to make a special case for SWP. It would be easy enough to override the refresh_duration() method on SWP to cache the SW stack...... but I was assuming that Blizzard would be fixing the problem soon. By putting the "wait" conditional in the config I figured to model what people would be doing to get around the bug. However..... since Blizzard still hasn't made the change I suppose I should conform my model to their buggy behaviour.

EDIT: #1 and #2 are checked in. I'll get to #3 later tonight.

Tifi · 11/01/08, 7:36 AM

Originally Posted by dedmonwakeen

SFMT was producing "more" random numbers across the entire spectrum enabling the use of the mod operator.

Even with a greater range, you shouldn't use modulo. Consider this: You have a random number generator X that produces results in the range [0,1234], and you take %1000. Let's call this Y=X%1000. The values of X in [1000,1234] are now mapped to values of Y in [0,234], thus you effectively doubled the probability of values for Y in [0,234]. For example the result Y=100 can be produced by both X=100 and X=1100, with probability 2/1235. The result Y=900, however, can only be produced by X=900, with probability 1/1235.

Edit: Assuming SFMT produces any 32-bit unsigned int, you should use 4294967295.0 instead of RAND_MAX. (2^32-1)

dedmonwakeen · 11/01/08, 9:30 AM

Originally Posted by Tifi

Even with a greater range, you shouldn't use modulo. Consider this: You have a random number generator X that produces results in the range [0,1234], and you take %1000. Let's call this Y=X%1000. The values of X in [1000,1234] are now mapped to values of Y in [0,234], thus you effectively doubled the probability of values for Y in [0,234]. For example the result Y=100 can be produced by both X=100 and X=1100, with probability 2/1235. The result Y=900, however, can only be produced by X=900, with probability 1/1235.

Edit: Assuming SFMT produces any 32-bit unsigned int, you should use 4294967295.0 instead of RAND_MAX. (2^32-1)

Ah.... now I understand. I think I was getting "lucky" by the fact that my modulo divisor was so much smaller than the maximum random integer, significantly reducing the benefit given the low end of the range.

SFMT provides a utility to return a floating point number between zero and one..... so I've switched to use that.

Nicarras · 11/02/08, 1:34 PM

How accurate are the shaman numbers? I have some ele/enh shaman telling me that the modeled rotations in the sample 80 file are way off.

dedmonwakeen · 11/02/08, 6:59 PM

Originally Posted by Nicarras

How accurate are the shaman numbers? I have some ele/enh shaman telling me that the modeled rotations in the sample 80 file are way off.

I would love for the Shaman numbers to be off because they just plain are not competitive and I feel bad for my Shammy guild-mates.

I haven't done a comparison against Enhsim in a while, so I could have drifted off-course.

I did a very detailed comparison against Binke's spreadsheet a while back and found bugs in both our models, but I thought I exited that process with a pretty decent model.

My recommendation to you is to run with "log=1" and "iterations=1" in the config file. This will generate a combat log allowing you to easily examine the spell choices and how much they tick for.

Bottom line: I'm buried right now. I'm just finishing up some massive infrastructure changes and some class polishing. My plan is to put currently supported classes on hold so that I can get Pally/Hunter/Rogue added.

I'm not really out to prove anything..... just doing the best I can, so.........

If someone just tells me "You're wrong." I'm probably not going to pay attention because I'm spread so thin.

If someone tells me "You're wrong because X/Y/Z and here are my counter examples A/B/C", well then I'm going to give them a TON of my attention and pester them with follow-up questions. So...... if your Shaman buddies have some specific data that is well-documented then I would be very grateful if they shared it with me.....

Eleven · 11/03/08, 12:45 PM

I think as far as elemental goes, it was shown that using earth shock in a rotation at 80 was a dps decrease and only a slight increase at 70 when using the ES glyph.

Btw, you still have mages/druids/spriests using wizard oil in the sample 80 file.

Sorry to nitpick, but keep up the good work, this project is amazing.

dedmonwakeen · 11/03/08, 1:51 PM

Originally Posted by Eleven

I think as far as elemental goes, it was shown that using earth shock in a rotation at 80 was a dps decrease and only a slight increase at 70 when using the ES glyph.

Btw, you still have mages/druids/spriests using wizard oil in the sample 80 file.

Sorry to nitpick, but keep up the good work, this project is amazing.

No need to apologize, nitpicking is exactly what the project needs in order to be successful.

Yeah, sorry about the Wizard Oil...... It keeps falling off the end of my list. I'll have it out by next release.

Regarding the Earth Shock testing: Was this a result of theorycraft, or somebody doing in-game testing? ie: Is it because the GCD is not being reduced properly due a bug..... Or do people simply believe it just sucks apriori?

Roywyn · 11/03/08, 1:58 PM

Originally Posted by dedmonwakeen

Regarding the Earth Shock testing: Was this a result of theorycraft, or somebody doing in-game testing? ie: Is it because the GCD is not being reduced properly due a bug..... Or do people simply believe it just sucks apriori?

Earth Shock GCD is hard capped at 1.0s like any other GCD. Tested on PTR 2-3 weeks ago.

GC said that it's intended to reduce the GCD further (to 0.5s or less with haste), and they're trying to fix it.
No news since then. I think it'll a lot of work to override the GCD hard cap.

[Edit]:

Originally Posted by Eleven

Unrelated, for arcane mages, I've been coming up with a 25-50 dps increase using a 57/14/0 build with arcane power/flows and glyphed mage armor over master of elements/ glyphed molten armor.

Arcane is riding the mana limit pretty hard. Whenever you change mana (like when 2T7 bonus gets added or JoW gets changed again), you'd have to flip back and forth with AP/MoW/Armour to see which one combination is better.

They've said that they want to nerf Arcane Barrage due to PvP and change some things to buff Arcane Blast or make it interact with Arcane Barrage, so it's not really worth changing right now.

Eleven · 11/03/08, 2:15 PM

Removing earth shock from the rotation puts elemental shaman back within that 8-10% deficit many people have been coming up with, I don't know if that would also change the scaling numbers you posted recently.

Unrelated, for arcane mages, I've been coming up with a 25-50 dps increase using a 57/14/0 build with arcane power/flows and glyphed mage armor over master of elements/ glyphed molten armor.

Tifi · 11/03/08, 9:11 PM

Are there any plans on modeling different spell type transitons, like channeling->cast, channeling->channeling, etc? I don't know if this is specific to Mind Flay, but I have to wait longer after MF when I cast MB or SWD than when I cast another MF.

I don't know if it's worth it, because a simple priority list for choosing the next spell would probably become less than optimal. If the transition channeling->cast/instant has a large penalty, minimizing the number of those transitions could become important, and a priority list won't do that. I wouldn't know how to compute an optimal strategy then, and I'm not even sure how to specify or model one.

dedmonwakeen · 11/03/08, 9:23 PM

Originally Posted by Tifi

Are there any plans on modeling different spell type transitons, like channeling->cast, channeling->channeling, etc? I don't know if this is specific to Mind Flay, but I have to wait longer after MF when I cast MB or SWD than when I cast another MF.

I don't know if it's worth it, because a simple priority list for choosing the next spell would probably become less than optimal. If the transition channeling->cast/instant has a large penalty, minimizing the number of those transitions could become important, and a priority list won't do that. I wouldn't know how to compute an optimal strategy then, and I'm not even sure how to specify or model one.

I have cast-type specific lag penalties..... but not transition pairs.

Cast Time greater than GCD => random lag centered around 150ms (option: "lag")
Cast Time less than GCD => random lag plus 100ms penalty (option: "gcd_penalty")
Channeled => random lag plus 300ms penalty (option: "channel_penalty")

Latency (and gcd effects) are used to schedule the next "player_ready" event once a cast has completed.

As far as DPET reporting is concerned, latency (and gcd effects) are tacked onto the preceding cast.

Hmm.... I don't think these penalties are in the Windows release at the moment. I'm -close- to a new release.... I keep finding something I want to polish. (most recently: making sure all three major talent calculator tools work)

dedmonwakeen · 11/06/08, 3:01 AM

SimulationCraft r1050 available for download.

Key changes:

* Multi-threading support for all platforms: Just add "threads=N" to your config. For dual-core processors, "threads=2" will result in a 2x performance increase.

* Scale-factor generation is now available. Example usage is in the raid_80.txt config file.

* Talent tree parsing for blizzard/wowhead/mmo for all classes. The raid_80.txt config file has been changed to use wowhead for brevity.

* The ability to set default gear profiles for use with all classes.

There have been a number of architectural changes. If you have been using your own config files, I encourage you to start with a fresh copy of raid_80.txt

We are working on getting the auto-generation of LatestReport wiki going again.

In the meantime, check out SampleOutput: SampleOutput - simulationcraft - Google Code

Roywyn · 11/06/08, 4:55 AM

Originally Posted by dedmonwakeen

SimulationCraft r1050 available for download.

Key changes:

* Multi-threading support for all platforms: Just add "threads=N" to your config. For dual-core processors, "threads=2" will result in a 2x performance increase.

* Scale-factor generation is now available. Example usage is in the raid_80.txt config file.

* Talent tree parsing for blizzard/wowhead/mmo for all classes. The raid_80.txt config file has been changed to use wowhead for brevity.

* The ability to set default gear profiles for use with all classes.

There have been a number of architectural changes. If you have been using your own config files, I encourage you to start with a fresh copy of raid_80.txt

We are working on getting the auto-generation of LatestReport wiki going again.

In the meantime, check out SampleOutput: SampleOutput - simulationcraft - Google Code

1) Awesome!

2) Frost Mage has Summon WE/Icy Veins/Cold Snap in its sequence, but only 42% WE uptime, which is 2 summons.
The fight length allows for 3 summons though, even without Cold Snap.

3) Some nitpicking about hit-rating details and numbers:
* default_hit_rating should be 289
* Both druids need a -26 hit adjustment
* Mage_20_51_0 and Priest 28_33_5 need a +79 hit adjustment
* Mage_0_53_18 needs no hit adjustment

4) I can't execute the simcraft.exe at all, gives some windwos error. It is also much smaller that the previous versions.

5) It still shows Starfire as better than Wrath for Druids oddly.

6) Is it possible to display the amount of mana you end the fight with?
Possibly in the section with all the mana gains?

7) I hate to ask dumb questions, but how can I generate a HTML output *blushes*

8) Thanks again for the great tool.

dedmonwakeen · 11/06/08, 8:08 AM

Originally Posted by Roywyn

2) Frost Mage has Summon WE/Icy Veins/Cold Snap in its sequence, but only 42% WE uptime, which is 2 summons.
The fight length allows for 3 summons though, even without Cold Snap.

Strange. I'll dig in. Thanks for the catch.

3) Some nitpicking about hit-rating details and numbers:
* default_hit_rating should be 289
* Both druids need a -26 hit adjustment
* Mage_20_51_0 and Priest 28_33_5 need a +79 hit adjustment
* Mage_0_53_18 needs no hit adjustment

I keep tweaking that and have yet to get it right. What do you think of the default gear profile? Do the other stats look ok?

4) I can't execute the simcraft.exe at all, gives some windwos error. It is also much smaller that the previous versions.

Grrr...... I switched from the MinGW g++ port to using MS Visual C++ (2008 Express). It was the only way I could get threads working. I wonder if there is some runtime library that I need to force to be included with the app? It ran ok on my Windows XP machine.... Bah.... I hate MS. I'll poke around in that IDE-from-hell and see if I can find some option that refers to runtime libs. Unfortunately, I do not have another Windows machine (without the IDE) to test it on.

5) It still shows Starfire as better than Wrath for Druids oddly.

If I -just- run Starfire vs Wrath in a simple setup then Wrath wins. But given the Glyph of Starfire, extreme levels of Haste gear and raid buffs...... SF wins out.

6) Is it possible to display the amount of mana you end the fight with?
Possibly in the section with all the mana gains?

Excellent idea...... and easy to code I think.

7) I hate to ask dumb questions, but how can I generate a HTML output *blushes*

html=file.html
Sorry.... I suck at documentation.

dedmonwakeen · 11/06/08, 8:33 AM

FYI..... I have uploaded a simcraft.exe built with MinGW. It won't have thread support, but at least I know it works....

Unfortunately, I won't be able to debug the MSVC++ problem until later tonight.

tukez · 11/06/08, 8:42 AM

Originally Posted by dedmonwakeen

FYI..... I have uploaded a simcraft.exe built with MinGW. It won't have thread support, but at least I know it works....

Unfortunately, I won't be able to debug the MSVC++ problem until later tonight.

People could try install Visual C++ Redistributable to get it to work. For example: Download details: Visual C++ 2008 SP1 Redistributable Package (x86)

There are many versions available, so I guess try with the same version dedmonwakeen has.

Troffel · 11/06/08, 8:54 AM

What i really like to see in simulationcraft is the distribution of the dps for each class. It could be easily estimated by a histogram from the iterations. Please include also the randomness of the base spells. That means if the base spell have a range of base damage, the base damage of this spell is random in this range.

dedmonwakeen · 11/06/08, 9:26 AM

Originally Posted by Troffel

What i really like to see in simulationcraft is the distribution of the dps for each class. It could be easily estimated by a histogram from the iterations. Please include also the randomness of the base spells. That means if the base spell have a range of base damage, the base damage of this spell is random in this range.

I don't show this in the google charts, but if you scroll down in SampleOutput you will see the wall-of-text that gets vomits out.

Immediately after the DPS value, there is information on stddev, range, etc. These numbers were generated with "average_dmg=1". If you set "average_dmg=0" then it will produce random base dmg values for direct-dmg spells.

EDIT: Hmmm..... I do have the DPS for every iteration cached away so I actually could generate some form of dps-distribution chart..... bucket the DPS via intervals.... count the number of hits for that interval..... Interesting.

Roywyn · 11/06/08, 11:44 AM

Originally Posted by dedmonwakeen

FYI..... I have uploaded a simcraft.exe built with MinGW. It won't have thread support, but at least I know it works....

Unfortunately, I won't be able to debug the MSVC++ problem until later tonight.

Works, yay! Oh, and more work:
"Flask of the Frost Wyrm" - +125 spell power
"Flask of Endless Rage" - +180 spell power
The mp5 flask is junk, and there is a +HP and a +Resilience flask, oriented for tanking.

The gear stats in the list look quite alright.

Spirit can vary a lot though, depending on your hit rating.
If you manage to cherry-pick all items without spirit, you'll end up with only +138 spirit.
If you take whatever is a decent upgrade when it drops, you'll end up with the +411 in your list.
I'd just leave it at what it is now.

dedmonwakeen · 11/06/08, 12:05 PM

Originally Posted by Roywyn

Works, yay! Oh, and more work:
"Flask of the Frost Wyrm" - +125 spell power
"Flask of Endless Rage" - +180 spell power
The mp5 flask is junk, and there is a +HP and a +Resilience flask, oriented for tanking.

I have plans to move the flask definitions out of the action list (along with weapon buffs).

When that happens I will add all the relevant flasks and elixirs.

I'm probably going to be lazy about the food definition, though. Instead of supporting foods by name, I'm going to simply allow a generic description:

food=stat1=value1,stat2=value2,stat3=value3

Example:
# Blackened Bassilisk
food=spell_power=23,spirit=20

Not sure if I got the values right there.... but you get the idea. That will provide the base functionality and I can add name-specific stuff at a later date.

All of these things could be encoded into the base gear values, but I've really tried hard to keep all stat contributions broken down into their individual components. I think this will be key if I ever get Armory-download working.

Kalle · 11/06/08, 1:43 PM

Originally Posted by Troffel

What i really like to see in simulationcraft is the distribution of the dps for each class. It could be easily estimated by a histogram from the iterations. Please include also the randomness of the base spells. That means if the base spell have a range of base damage, the base damage of this spell is random in this range.

This is not a big deal, but you'll see nothing more than a Gaussian distribution with mean DPS and standard deviation 0.5*sqrt(iterations)*Error (if iterations is large enough).

dedmonwakeen · 11/06/08, 2:12 PM

Originally Posted by Kalle

This is not a big deal, but you'll see nothing more than a Gaussian distribution with mean DPS and standard deviation 0.5*sqrt(iterations)*Error (if iterations is large enough).

I'm actually considering doing this not so much as a helpful tool for the user, but as an additional audit of the simulation results.

ie: If the distribution doesn't look "Gaussian enough" (bell-curvy) it highlights a problem that I need to look into.....

erragal · 11/06/08, 2:14 PM

Dedemonwakeen: I'd suggest running/showing balance numbers without faerire fire. In a balanced raid there should be no reason to cast it, and the DPS hit is fairly significant. For the sample output at least, it'd be appropriate to show the maximum potential DPS similar to affliction locks with and without CoE. It'll make for good comparison in non-feral raids with shadow priests to see whether affliction locks or balance druids take more of a DPS hit by applying the minor armor debuff. Starfire being better than wrath is probably more related to the heavy penalty you get from latency when chain casting wrath.

Troffel · 11/06/08, 2:18 PM

Originally Posted by Kalle

This is not a big deal, but you'll see nothing more than a Gaussian distribution with mean DPS and standard deviation 0.5*sqrt(iterations)*Error (if iterations is large enough).

The central limit theorem is not applicable, because the conditions a not satisfied.

Mainly, I expect a not necessary a Gaussian distribution, because the random variables are not (in general) independent. It has to be proven that the distributions are Gaussian and this will be a interesting fact.