SimulationCraft Model Development

dedmonwakeen · 10/02/08, 3:12 PM

SimulationCraft is an event-driven multi-player DPS simulator with modules available for all classes.

Home Page
Developer IRC channel: irc.stratics.com #simulationcraft
Downloads
Recent Updates
Sample Output
Source Code
- Death Knight
- Druid
- Hunter
- Mage
- Paladin
- Priest
- Rogue
- Shaman
- Warlock
- Warrior
How To Run
How To Build
Formulation vs Simulation

• sp00n · 10/07/08, 12:28 PM

Are there already any melee mechanics available? Or would they have to be completely (re-)written?

I stumbled across the postings in the mage WotLK threads, and I'd really like to see how rogues compare to the caster numbers.

That said, I have exactly zilch knowledge of C++, so unfortunately am of no help you. I'm not sure if Aldriana has any C++ knowledge, but including his RogueCalc - which is written in Python - into a larger simulation tool would be the ideal solution I guess.

Skellum · 10/07/08, 12:40 PM

Is there any chance of formatting the output a bit? I love the bar graphs but I hate the fact that it doesnt include any specific information about the sample data its collected from. By modifying the Simcraft input I can give high crit and haste rates with low +dmg rates and start shunting this all about a bunch of forums,non-EJ, in a rabble rousing attempt.

It would be really amazing if the output automatically provided a link to the data source or was linked to it to where excluding the source data would be difficult to do and be more effort than the average troll would put in to creating a disruption with it.

Kalle · 10/07/08, 12:51 PM

I found some problems with warlocks in r688:

Unstable Affliction was changed to 15s duration, it now has only 5 ticks, new coefficient is 1.0
Shadow Embrace debuff lasts for 12s ingame, not 15s
Improved Soul Leech has two ranks ingame, restoring 1%/2% of total Mana
Duration of Demonic Empowerment is different for imp and for felguard now
In trigger_backdraft you wrote "sim -> add_event( this, 15.0 );" and "e -> reschedule( 12.0 );", the second one should also be 15s, shouldn't it?
Deathcoil is an affliction spell ingame, not destruction
The <25% multiplier for Drain Soul is only present for rank 6 ingame
Haunt's crit bonus was lowered to 50%
Soul Fire doesn't get base_execute_time / 3.5 for its coefficient. Quick testing shows something between 1.112 and 1.182, probably the 1.15 mentioned here.
Dark Pact is not affected by Shadow Mastery ingame
Chaos Bolt cannot miss, but I see partial resits - not sure on the mechanics yet
Metamorphosis doesn't trigger the gcd ingame
Melee attacks from pets may be glancing hits ingame
I retested Master Conjuror in build 9038 and the DD/DoT damage increasing portion of fire-/spellstone is still not affected by it

In imp_pet_t:

    virtual void player_buff()
    {
      imp_pet_t* p = (imp_pet_t*) player -> cast_pet();
      warlock_pet_spell_t::player_buff();
      if( p -> buffs_demonic_empowerment )
      {
	player_multiplier *= 1.20;
	player_crit       += 0.20;
      }
    }

Demonic Empowerment only increases the critchance for the imp. I suppose, the player_multiplier part should be removed.

Regarding your combat rating interpolation, WoWWiki has formulas on that.

In the default raid_80.txt, the destruction warlock does not use Corruption but has glyph_corruption=1. For me it was a dps increase to include Corruption in the action list.

Can you add statistics to the results? Would be nice to know how many iterations will be neccessary to know for sure if talent/glyph/spell x is a damage increase. I would also like an option "simulate until estimated error < x% for all players".

Mearis · 10/07/08, 12:53 PM

Sent you a PM

dedmonwakeen · 10/07/08, 2:21 PM

Originally Posted by sp00n

Are there already any melee mechanics available? Or would they have to be completely (re-)written?

I stumbled across the postings in the mage WotLK threads, and I'd really like to see how rogues compare to the caster numbers.

That said, I have exactly zilch knowledge of C++, so unfortunately am of no help you. I'm not sure if Aldriana has any C++ knowledge, but including his RogueCalc - which is written in Python - into a larger simulation tool would be the ideal solution I guess.

This is actually my third attempt over the past three years to model DPS. The first attempt was for Shadow Priests and it used the standard "formulation" method of enforced rotations, etc. This led me to simulation and I opened up support for all caster classes. For WotLK I did a complete re-write to create an architecture that would support melee classes as well.

So far I only have Enhancement Shaman written........ which is somewhat superfluous because there already exists two very good Enh-Shaman simulators! I do intend, barring personal catastrophe, to have all the classes implemented. The class currently in development is Paladin.... with Hunters to follow. (I wanted to get through all the mana-based classes first.)

Aldriana is a very good developer with exceptional insight into the Rogue class, but I highly doubt he will be interested in contributing to SimulationCraft. There are some things that simulators are good at..... and there are some things that simulations are NOT good at.... and Aldriana understands this quite well. Provided you have invested sufficient thought into your model, formulation is much better at delivering consistent EP values. He and I have had many conversations on this subject and we understand the need for both Formulation and Simulation tools.

My thoughts on the subject can be found here: FormulationVsSimulation - simulationcraft - Google Code

All that being said..... I'd be THRILLED to have someone like Aldriana on the team. Detailed knowledge of WoW mechanics is FAR more important than deep familiarity with obscure/obscene features of C++. Indeed, if I knew there was a motivated individual with sufficient experience waiting in the wings, I would change the order in which new classes are added to the sim just to get them involved sooner...........

Originally Posted by Skellum

Is there any chance of formatting the output a bit?

By default, the tool generates TONS of text output...... but I have no way of enforcing people to post that along with the graphs. The SampleOutput page (found by clicking through my sig) has both the charts and the detailed text, as well as a link to the config file. It is capable of generating HTML and WIKI format for the charts..... and I do intend to add the detailed text output to the HTML and WIKI. However, I still can't guarantee that people will host an entire HTML file as opposed to just posting a GoogleChart http link.

Battlemaid · 10/07/08, 2:26 PM

It would be really amazing if the output automatically provided a link to the data source or was linked to it to where excluding the source data would be difficult to do and be more effort than the average troll would put in to creating a disruption with it.

At least, the ethalon SimCraft report: LatestReport - simulationcraft - Google Code doesn't suffer from such issues since it includes links to input and output artifacts of SimCraft in the wiki page text.

However, it doesn't inline simulation artifacts right into HTML, but rather uploads those to the server and stores them next to the report web page. The simplest idea would be to include input/output files in the HTML text, but toggle its visibility via JavaScript. Tho this still leaves the charts images by themselves being incomprehensive and susceptible to frauds that you described.

dedmonwakeen · 10/07/08, 2:42 PM

Originally Posted by Kalle

I found some problems with warlocks in r688:

Unstable Affliction was changed to 15s duration, it now has only 5 ticks, new coefficient is 1.0
Shadow Embrace debuff lasts for 12s ingame, not 15s
Improved Soul Leech has two ranks ingame, restoring 1%/2% of total Mana
Duration of Demonic Empowerment is different for imp and for felguard now
In trigger_backdraft you wrote "sim -> add_event( this, 15.0 );" and "e -> reschedule( 12.0 );", the second one should also be 15s, shouldn't it?
Deathcoil is an affliction spell ingame, not destruction
The <25% multiplier for Drain Soul is only present for rank 6 ingame
Haunt's crit bonus was lowered to 50%
Soul Fire doesn't get base_execute_time / 3.5 for its coefficient. Quick testing shows something between 1.112 and 1.182, probably the 1.15 mentioned here.
Dark Pact is not affected by Shadow Mastery ingame
Chaos Bolt cannot miss, but I see partial resits - not sure on the mechanics yet
Metamorphosis doesn't trigger the gcd ingame
Melee attacks from pets may be glancing hits ingame
I retested Master Conjuror in build 9038 and the DD/DoT damage increasing portion of fire-/spellstone is still not affected by it

THANK YOU! I will make the requisite changes ASAP!

In imp_pet_t:
Demonic Empowerment only increases the critchance for the imp. I suppose, the player_multiplier part should be removed.

Regarding your combat rating interpolation, WoWWiki has formulas on that.

Only the Imp is getting buffed. Pets are considered "players" by the core architecture. I probably should have used the term "actor". The pet_t class is a sub-class of player_t. The spell_t class holds the calculated set of buffs/debuffs/etc for each spell. The dynamic values associated with an "actor" are all of the form "player_xxx".

I'm afraid I was very cowardly with my implementation of Ratings-conversion with the hope that most people would run with either level 70 or level 80 players. I will look into the information and put it on the TODO list, but it may get prioritized a bit low for now.

In the default raid_80.txt, the destruction warlock does not use Corruption but has glyph_corruption=1. For me it was a dps increase to include Corruption in the action list.

Can you add statistics to the results? Would be nice to know how many iterations will be neccessary to know for sure if talent/glyph/spell x is a damage increase. I would also like an option "simulate until estimated error < x% for all players".

The Destruction profile has been in constant flux due to the recent code reviews to ferret out the remaining problems with Warlock support. Corruption keeps getting put in and out depending upon who touched the profile last and their particular opinion in the "Is the extra spell worth the benefit."

The statistics idea is very interesting. I supposed an option to simulate for a certain number of CPU seconds (as opposed to iterations) might also be useful. I'll see what I can do.

PS: Sorry for the odd "two topics per quote-segmentation". This is already a back-to-back post and if I made it too granular I'd likely pick up another infraction.

dedmonwakeen · 10/07/08, 5:05 PM

@Kalle: I've checked in fixes for all the issues you highlighted. Thanks again for such a detailed review. I'll have a new download available later tonight.

The statistical requests will take a bit longer. I have several ideas bouncing around and I need to decide which avenue to pursue.

Battlemaid · 10/07/08, 6:09 PM

"simulate until estimated error < x% for all players".

Back then we had a discussion about variance on our boards (check the post and several following posts here: shadowpriest.com • View topic - SimulationCraft).

General idea is that currently a sequence of averaged simulation output parameters (be it dps or proc uptime) converges too slowly with increasing number of iterations. Mentioned posts attempt to address this problem, however things didnt move further than some not implemented ideas - we had lots of stuff to do at that time.

By the way, while we're talking LatestReport - simulationcraft - Google Code got an update with the latest fixes by Dedmonwakeen. Check it - maybe you'll find some more corrections.

dedmonwakeen · 10/08/08, 1:28 PM

SimulationCraft version r698 available for download.

Summary of changes: RecentUpdates - simulationcraft - Google Code

Sample Output: SampleOutput - simulationcraft - Google Code

Note: Improved Divine Spirit no longer stacks so DPS will see a minor drop across the board.

dedmonwakeen · 10/08/08, 4:52 PM

Originally Posted by Kalle

Can you add statistics to the results? Would be nice to know how many iterations will be neccessary to know for sure if talent/glyph/spell x is a damage increase. I would also like an option "simulate until estimated error < x% for all players".

I've recently checked in code that reports standard deviation for player DPS.

In addition, I report "convergence" data: I take the average DPS when the sim has executed 90% of the iterations. I compare this to the final average DPS to report a convergence metric.

Shortly I'll add an option for controlling the runtime via this convergence metric.

Kalle · 10/09/08, 12:16 PM

Thanks for the quick implementation of my fixes/suggestions.

My question with the quoted part from imp_pet_t was: Why are you doing multiplier *= 1.2 if the buff only increases the critchance?

When I talked about statistics, I thought about error estimates. Currently you calculate the standard deviation your model produces. With average_dmg=1 I don't find that number particularly interesting, because it is something completely different from the deviation you'll see ingame. Sure, you only have to divide by the square root of the number of iterations to get the estimated error, but writing something like (3000 +/- 10) dps would make it easier to understand for people not familiar with statistics, I think (10 dps being the estimated error of the average value 3000 dps). This Google chart thing doesn't support error bars, does it?

Battlemaid · 10/09/08, 1:00 PM

With average_dmg=1 I don't find that number particularly interesting, because it is something completely different from the deviation you'll see ingame

You'll be surprised but even with average_dmg=0 stddev doesn't differ from average_dmg=1. Results for 9 runs of the same config with 10^2 sims, stddev taken from Druid_Balance stats:
average_dmg=0: 3x times 3.0%, 3x times 3.1%, 2x times 3.2%, 1x time 3.3%
average_dmg=1: 4x times 3.1%, 1x time 3.2%, 3x times 3.3%

This Google chart thing doesn't support error bars, does it?

This is what gcharts can do for emulating error bars:

dedmonwakeen · 10/09/08, 2:47 PM

Originally Posted by Kalle

My question with the quoted part from imp_pet_t was: Why are you doing multiplier *= 1.2 if the buff only increases the critchance?

Why? Because the code is bugged and I failed to understand your very clear explanation the first time.

Fix incoming...........

When I talked about statistics, I thought about error estimates. Currently you calculate the standard deviation your model produces. With average_dmg=1 I don't find that number particularly interesting, because it is something completely different from the deviation you'll see ingame. Sure, you only have to divide by the square root of the number of iterations to get the estimated error, but writing something like (3000 +/- 10) dps would make it easier to understand for people not familiar with statistics, I think (10 dps being the estimated error of the average value 3000 dps). This Google chart thing doesn't support error bars, does it?

Thanks for the feedback.... We're still feeling our way through this stuff.

The standard deviation value I have there is: ( std_dev / avg_dps )

What you are looking for is more like this? ( std_dev / sqrt( iterations ) )

GoogleChart provides a freakish amount of functionality with several options for error bars, I believe. Getting to my current level of understanding almost triggered a brain aneurysm, but I'll jump back into the fray.......

Pitbuller · 10/10/08, 11:40 AM

Dedmowakeen: I suggest you use this spec at 80lv and this at 70lvl. Do shamanist focus work in your sim? It dind't lower downtime at all. How about convection?

Rotation:

actions=flask,type=relentless_assault/windfury_weapon,weapon=main/windfury_weapon,weapon=off/auto_attack/lightning_bolt,maelstrom=5/stormstrike/strength_of_earth_totem/windfury_totem/shamanistic_rage/spirit_wolf/earth_shock/lava_lash/searing_totem/water_shield/bloodlust,target_pct=35

This give 4733dps.
Static shock was dps loss becouse mana starving but I bet there is some mana gain problem and it's usable when those are fixed.

Also lava burst isn't castable with maelstorm weapon anymore.

What wrong with searing totem? Only 1500damage per execute? Should be over 15000/250dps.
Sim cast earth shock once per 12s. Cd is still 6s without reverberation. No flame shcok was used.

Or what I do so wrong?

• Zurm · 10/10/08, 12:22 PM

I'm currently coding the Death Knight DPS module for Rawr... my C++ knowledge isn't to a point where I'd be comfortable owning a module but I'm more than willing to help someone who does with the math/theory and suggestions on how to model certain mechanics.

ravenndude · 10/10/08, 1:20 PM

Any chance you can add Glyph of Icy Veins for mages?

Erdluf · 10/10/08, 2:48 PM

I'd like to see some support for fights which aren't pure tank & spank. For instance, player input containing

movement_period=20
movement_duration=1.7

might mean that every 0 to 40 seconds (zero to movement_period*2), the player must "move" for 1.7 seconds (cancelcast, and then use only instant-cast spells for the next 1.7 seconds).

pushback_period=5

would mean that a caster gets a pushback event (takes a hit causing spell to be delayed) every zero to ten seconds.

Just as latency favors spells with long cast times, movement and pushback favor spells with short (or zero) cast times. I think simcraft could be a great tool for quantifying that benefit.

Discuss? Thanks!

dedmonwakeen · 10/10/08, 3:36 PM

Originally Posted by Pitbuller

Dedmowakeen: I suggest you use this spec at 80lv and this at 70lvl. Do shamanist focus work in your sim? It dind't lower downtime at all. How about convection?

Rotation:

actions=flask,type=relentless_assault/windfury_weapon,weapon=main/windfury_weapon,weapon=off/auto_attack/lightning_bolt,maelstrom=5/stormstrike/strength_of_earth_totem/windfury_totem/shamanistic_rage/spirit_wolf/earth_shock/lava_lash/searing_totem/water_shield/bloodlust,target_pct=35

This give 4733dps.
Static shock was dps loss becouse mana starving but I bet there is some mana gain problem and it's usable when those are fixed.

Also lava burst isn't castable with maelstorm weapon anymore.

What wrong with searing totem? Only 1500damage per execute? Should be over 15000/250dps.
Sim cast earth shock once per 12s. Cd is still 6s without reverberation. No flame shcok was used.

Or what I do so wrong?

Pitbuller, thank you very much for the feedback. Enh Shaman hasn't gotten much oversight compared to the other classes..... and I didn't want to distract to much from Enhsim's efforts.

The plumbing for Shamanistic Focus seems okay...... I'll dig in. Downtime comes into play due to waiting on duration, cooldowns, and resources. Unfortunately, I do not distinguish between those in the report..... so it is unclear just how much is due to lack of mana.

Thanks for the LvB catch. I also need to update the Maelstrom mechanics to PPM. I saw so many reports on the subject..... but I must have missed the final analysis on #PPM and the affects of haste.

Searing Totem: First, I haven't implemented the new ranks. Bah! Second, I neglected to call player_buff() during the execute stage which means it isn't picking up the spell power/crit., just the target_debuff() stuff. Bah! Sorry dude, stuff like that shouldn't escape into the field.

I don't see Flame Shock in your action list...... Also, since Earth Shock has a lower priority than say Stormstrike, perhaps it is possible that there may be stretches of time (between shamanistic rage) where you are severely mana limited??? Basically.... you'd never get to Earth Shock because Malestrom Weapon and Stormstrike are eating up the mana.

I need to do a full review of my mana costs for Shaman spells.... perhaps they are just too high.

Originally Posted by Zurm

I'm currently coding the Death Knight DPS module for Rawr... my C++ knowledge isn't to a point where I'd be comfortable owning a module but I'm more than willing to help someone who does with the math/theory and suggestions on how to model certain mechanics.

Thanks! It may be a while before we get to Death Knight, but I will certainly remember your offer when the time comes.

Originally Posted by ravenndude

Any chance you can add Glyph of Icy Veins for mages?

Definitely. It will be in the next release for sure.

Originally Posted by Erdluf

I'd like to see some support for fights which aren't pure tank & spank. For instance, player input containing

movement_period=20
movement_duration=1.7

might mean that every 0 to 40 seconds (zero to movement_period*2), the player must "move" for 1.7 seconds (cancelcast, and then use only instant-cast spells for the next 1.7 seconds).

pushback_period=5

would mean that a caster gets a pushback event (takes a hit causing spell to be delayed) every zero to ten seconds.

Just as latency favors spells with long cast times, movement and pushback favor spells with short (or zero) cast times. I think simcraft could be a great tool for quantifying that benefit.

Discuss? Thanks!

We've talked about this amongst the development team multiple times. It really isn't very difficult to insert events that cancel (or push back) current casts and resubmit a "player_ready" event some number of seconds in the future.

The only real hard part is coming up with a reasonable set of controls. As you imply, we would want to be able to tie probability to these events. minimum lengths, cooldowns, etc. If you can hammer out a comprehensive set of controls that are not unwieldy, I'm sure we'll find the time to implement them.

dedmonwakeen · 10/10/08, 4:14 PM

Pitbuller, with the fix to Searing Totem and some slight changes to the config:

Config:

shaman=Shaman_Enhancement
level=80
talents=http://talent.mmo-champion.com/?shaman=052030150000000000000000000000000000000000000000003020502300502135331131231051
actions=flask,type=relentless_assault/windfury_weapon,weapon=both/auto_attack/lightning_bolt,maelstrom=5/stormstrike/strength_of_earth_totem/windfury_totem/shamanistic_rage/spirit_wolf/flame_shock/earth_shock/lava_lash/searing_totem/water_shield/bloodlust,target_pct=35
gear_agility=386
gear_stamina=800
gear_intellect=280
gear_attack_power=2000
gear_attack_expertise_rating=185
gear_attack_penetration=0
gear_spell_power=0
gear_hit_rating=280
gear_crit_rating=445
gear_haste_rating=75
chaotic_skyfire=1
# Hateful Gladiator Weapons
main_hand=mace,damage=373,speed=2.6,enchant=mongoose
#off_hand=dagger,damage=258,speed=1.8,enchant=executioner
off_hand=mace,damage=373,speed=2.6,enchant=executioner
# Glyphs
glyph_earth_shock=0
glyph_strength_of_earth=1
glyph_windfury_weapon=1
glyph_stormstrike=1
glyph_lightning_shield=1
# Pets
pet=spirit_wolf
blessing_of_kings=1
blessing_of_might=550
sanctified_retribution=1
swift_retribution=1
quiet=1
active=owner
# Throw in a pair of Paladins.......  Eventually these static parms will be replaced with real actors.
blessing_of_kings=1
blessing_of_might=550
sanctified_retribution=1
swift_retribution=1

Output:

Player=Shaman_Enhancement  DPS=4827.6 (+-10.3)  CS=5  DPR=20.2  RPS=238.8/227.1  (mana)
  Core Stats:  strength=138  agility=501  stamina=1028  intellect=461  spirit=154  health=13290  mana=10766
  Spell Stats:  power=977  hit=10.7%  crit=14.7%  penetration=0  haste=2.3%  mp5=0
  Attack Stats:  power=3257  hit=8.5%  crit=17.4%  expertise=5.6  penetration=0  haste=2.3%
  Actions:
    earth_shock           Count=  8.4|33.2sec  DPE=2953| 2%  DPET=1924  DPR= 5.3  Miss=3.3%  Hit=2137  CritHit=4471|4548|38.0%
    flame_shock           Count= 11.1|25.1sec  DPE=2145| 2%  DPET=1397  DPR= 4.0  Miss=3.3%  Hit= 926  CritHit=1935|1947|35.6%  Tick=272
    lava_lash             Count= 27.6|10.1sec  DPE=1275| 3%  DPET= 750  DPR= 7.3  Miss=0.0%  Hit=1027  CritHit=2054|2070|31.8%
    lightning_bolt        Count= 55.2| 5.1sec  DPE=3590|15%  DPET=2365  DPR=10.5  Miss=3.2%  Hit=2597  CritHit=5421|5628|38.1%
    melee_main_hand       Count=173.9| 1.6sec  DPE=1554|20%  DPET= 973  DPR= inf  Miss=12.5%  Hit=1473  CritHit=2947|3040|31.4%
    melee_off_hand        Count=173.9| 1.6sec  DPE= 776|10%  DPET= 486  DPR= inf  Miss=12.4%  Hit= 736  CritHit=1473|1520|31.3%
    searing_totem         Count=  3.8|73.6sec  DPE=9979| 3%  DPET=8307  DPR=43.5  Miss=3.4%  Tick=268  CritTick=559|564|37.7%
    stormstrike           Count= 27.9|10.0sec  DPE=2658| 6%  DPET=1563  DPR= 2.6  Miss=0.0%  Hit=2368  CritHit=3855|4560|31.7%
    windfury              Count=224.5| 1.2sec  DPE=2225|37%  DPET= inf  DPR= inf  Miss=0.0%  Hit=1797  CritHit=3594|5053|31.4%
   spirit_wolf
    wolf_melee            Count= 59.2| 4.1sec  DPE= 798| 4%  DPET= 532  DPR= inf  Miss=0.5%  Hit= 880  CritHit=1761|1763|6.1%

Perhaps I'm triggering Shamanistic Rage too eary.......

Kalle · 10/11/08, 11:12 AM

For Pandemic you're doing

if( rand_t::roll( s -> player_crit ) )
{
  s -> tick_dmg *= 1.0 + p -> talents.pandemic / 3.0;
}

However, currently the damage added by Pandemic is kind of a seperate spell and is subject to partial resists. I have no idea if this is intended. My combatlog reads (with 3 points in pandemic):

Corruption ticks for 194 (453 resisted)
Pandemic hits for 58 (136 resisted)

ravenndude · 10/11/08, 2:57 PM

Originally Posted by dedmonwakeen

<My IV post>
Definitely. It will be in the next release for sure.

Don't worry about adding glyoh of IV ... I didn't quite read it, hah. For some reason, I thought it elongated IV.

dedmonwakeen · 10/11/08, 4:51 PM

Originally Posted by Kalle

For Pandemic you're doing

if( rand_t::roll( s -> player_crit ) )
{
  s -> tick_dmg *= 1.0 + p -> talents.pandemic / 3.0;
}

However, currently the damage added by Pandemic is kind of a seperate spell and is subject to partial resists. I have no idea if this is intended. My combatlog reads (with 3 points in pandemic):

Corruption ticks for 194 (453 resisted)
Pandemic hits for 58 (136 resisted)

Thanks for the heads up. Very interesting mechanic. I'll set it up as a separate spell such that it gets tracked properly.

Fireye · 10/14/08, 6:35 PM

I saw mention of a raid_70.txt that would be coming soon on the shadowpriest board, any idea when we might see that arrive?