SimulationCraft Model Development

dedmonwakeen · 10/02/08, 2:12 PM

SimulationCraft is an event-driven multi-player DPS simulator with modules available for all classes.

Home Page
Developer IRC channel: irc.stratics.com #simulationcraft
Downloads
Recent Updates
Sample Output
Source Code
- Death Knight
- Druid
- Hunter
- Mage
- Paladin
- Priest
- Rogue
- Shaman
- Warlock
- Warrior
How To Run
How To Build
Formulation vs Simulation

sp00n · 10/07/08, 11:28 AM

Are there already any melee mechanics available? Or would they have to be completely (re-)written?

I stumbled across the postings in the mage WotLK threads, and I'd really like to see how rogues compare to the caster numbers.

That said, I have exactly zilch knowledge of C++, so unfortunately am of no help you. I'm not sure if Aldriana has any C++ knowledge, but including his RogueCalc - which is written in Python - into a larger simulation tool would be the ideal solution I guess.

Skellum · 10/07/08, 11:40 AM

Is there any chance of formatting the output a bit? I love the bar graphs but I hate the fact that it doesnt include any specific information about the sample data its collected from. By modifying the Simcraft input I can give high crit and haste rates with low +dmg rates and start shunting this all about a bunch of forums,non-EJ, in a rabble rousing attempt.

It would be really amazing if the output automatically provided a link to the data source or was linked to it to where excluding the source data would be difficult to do and be more effort than the average troll would put in to creating a disruption with it.

Kalle · 10/07/08, 11:51 AM

I found some problems with warlocks in r688:

Unstable Affliction was changed to 15s duration, it now has only 5 ticks, new coefficient is 1.0
Shadow Embrace debuff lasts for 12s ingame, not 15s
Improved Soul Leech has two ranks ingame, restoring 1%/2% of total Mana
Duration of Demonic Empowerment is different for imp and for felguard now
In trigger_backdraft you wrote "sim -> add_event( this, 15.0 );" and "e -> reschedule( 12.0 );", the second one should also be 15s, shouldn't it?
Deathcoil is an affliction spell ingame, not destruction
The <25% multiplier for Drain Soul is only present for rank 6 ingame
Haunt's crit bonus was lowered to 50%
Soul Fire doesn't get base_execute_time / 3.5 for its coefficient. Quick testing shows something between 1.112 and 1.182, probably the 1.15 mentioned here.
Dark Pact is not affected by Shadow Mastery ingame
Chaos Bolt cannot miss, but I see partial resits - not sure on the mechanics yet
Metamorphosis doesn't trigger the gcd ingame
Melee attacks from pets may be glancing hits ingame
I retested Master Conjuror in build 9038 and the DD/DoT damage increasing portion of fire-/spellstone is still not affected by it

In imp_pet_t:

    virtual void player_buff()
    {
      imp_pet_t* p = (imp_pet_t*) player -> cast_pet();
      warlock_pet_spell_t::player_buff();
      if( p -> buffs_demonic_empowerment )
      {
	player_multiplier *= 1.20;
	player_crit       += 0.20;
      }
    }

Demonic Empowerment only increases the critchance for the imp. I suppose, the player_multiplier part should be removed.

Regarding your combat rating interpolation, WoWWiki has formulas on that.

In the default raid_80.txt, the destruction warlock does not use Corruption but has glyph_corruption=1. For me it was a dps increase to include Corruption in the action list.

Can you add statistics to the results? Would be nice to know how many iterations will be neccessary to know for sure if talent/glyph/spell x is a damage increase. I would also like an option "simulate until estimated error < x% for all players".

Mearis · 10/07/08, 11:53 AM

Sent you a PM

dedmonwakeen · 10/07/08, 1:21 PM

Originally Posted by sp00n

Are there already any melee mechanics available? Or would they have to be completely (re-)written?

I stumbled across the postings in the mage WotLK threads, and I'd really like to see how rogues compare to the caster numbers.

That said, I have exactly zilch knowledge of C++, so unfortunately am of no help you. I'm not sure if Aldriana has any C++ knowledge, but including his RogueCalc - which is written in Python - into a larger simulation tool would be the ideal solution I guess.

This is actually my third attempt over the past three years to model DPS. The first attempt was for Shadow Priests and it used the standard "formulation" method of enforced rotations, etc. This led me to simulation and I opened up support for all caster classes. For WotLK I did a complete re-write to create an architecture that would support melee classes as well.

So far I only have Enhancement Shaman written........ which is somewhat superfluous because there already exists two very good Enh-Shaman simulators! I do intend, barring personal catastrophe, to have all the classes implemented. The class currently in development is Paladin.... with Hunters to follow. (I wanted to get through all the mana-based classes first.)

Aldriana is a very good developer with exceptional insight into the Rogue class, but I highly doubt he will be interested in contributing to SimulationCraft. There are some things that simulators are good at..... and there are some things that simulations are NOT good at.... and Aldriana understands this quite well. Provided you have invested sufficient thought into your model, formulation is much better at delivering consistent EP values. He and I have had many conversations on this subject and we understand the need for both Formulation and Simulation tools.

My thoughts on the subject can be found here: FormulationVsSimulation - simulationcraft - Google Code

All that being said..... I'd be THRILLED to have someone like Aldriana on the team. Detailed knowledge of WoW mechanics is FAR more important than deep familiarity with obscure/obscene features of C++. Indeed, if I knew there was a motivated individual with sufficient experience waiting in the wings, I would change the order in which new classes are added to the sim just to get them involved sooner...........

Originally Posted by Skellum

Is there any chance of formatting the output a bit?

By default, the tool generates TONS of text output...... but I have no way of enforcing people to post that along with the graphs. The SampleOutput page (found by clicking through my sig) has both the charts and the detailed text, as well as a link to the config file. It is capable of generating HTML and WIKI format for the charts..... and I do intend to add the detailed text output to the HTML and WIKI. However, I still can't guarantee that people will host an entire HTML file as opposed to just posting a GoogleChart http link.

Battlemaid · 10/07/08, 1:26 PM

It would be really amazing if the output automatically provided a link to the data source or was linked to it to where excluding the source data would be difficult to do and be more effort than the average troll would put in to creating a disruption with it.

At least, the ethalon SimCraft report: LatestReport - simulationcraft - Google Code doesn't suffer from such issues since it includes links to input and output artifacts of SimCraft in the wiki page text.

However, it doesn't inline simulation artifacts right into HTML, but rather uploads those to the server and stores them next to the report web page. The simplest idea would be to include input/output files in the HTML text, but toggle its visibility via JavaScript. Tho this still leaves the charts images by themselves being incomprehensive and susceptible to frauds that you described.

dedmonwakeen · 10/07/08, 1:42 PM

Originally Posted by Kalle

I found some problems with warlocks in r688:

Unstable Affliction was changed to 15s duration, it now has only 5 ticks, new coefficient is 1.0
Shadow Embrace debuff lasts for 12s ingame, not 15s
Improved Soul Leech has two ranks ingame, restoring 1%/2% of total Mana
Duration of Demonic Empowerment is different for imp and for felguard now
In trigger_backdraft you wrote "sim -> add_event( this, 15.0 );" and "e -> reschedule( 12.0 );", the second one should also be 15s, shouldn't it?
Deathcoil is an affliction spell ingame, not destruction
The <25% multiplier for Drain Soul is only present for rank 6 ingame
Haunt's crit bonus was lowered to 50%
Soul Fire doesn't get base_execute_time / 3.5 for its coefficient. Quick testing shows something between 1.112 and 1.182, probably the 1.15 mentioned here.
Dark Pact is not affected by Shadow Mastery ingame
Chaos Bolt cannot miss, but I see partial resits - not sure on the mechanics yet
Metamorphosis doesn't trigger the gcd ingame
Melee attacks from pets may be glancing hits ingame
I retested Master Conjuror in build 9038 and the DD/DoT damage increasing portion of fire-/spellstone is still not affected by it

THANK YOU! I will make the requisite changes ASAP!

In imp_pet_t:
Demonic Empowerment only increases the critchance for the imp. I suppose, the player_multiplier part should be removed.

Regarding your combat rating interpolation, WoWWiki has formulas on that.

Only the Imp is getting buffed. Pets are considered "players" by the core architecture. I probably should have used the term "actor". The pet_t class is a sub-class of player_t. The spell_t class holds the calculated set of buffs/debuffs/etc for each spell. The dynamic values associated with an "actor" are all of the form "player_xxx".

I'm afraid I was very cowardly with my implementation of Ratings-conversion with the hope that most people would run with either level 70 or level 80 players. I will look into the information and put it on the TODO list, but it may get prioritized a bit low for now.

In the default raid_80.txt, the destruction warlock does not use Corruption but has glyph_corruption=1. For me it was a dps increase to include Corruption in the action list.

Can you add statistics to the results? Would be nice to know how many iterations will be neccessary to know for sure if talent/glyph/spell x is a damage increase. I would also like an option "simulate until estimated error < x% for all players".

The Destruction profile has been in constant flux due to the recent code reviews to ferret out the remaining problems with Warlock support. Corruption keeps getting put in and out depending upon who touched the profile last and their particular opinion in the "Is the extra spell worth the benefit."

The statistics idea is very interesting. I supposed an option to simulate for a certain number of CPU seconds (as opposed to iterations) might also be useful. I'll see what I can do.

PS: Sorry for the odd "two topics per quote-segmentation". This is already a back-to-back post and if I made it too granular I'd likely pick up another infraction.

dedmonwakeen · 10/07/08, 4:05 PM

@Kalle: I've checked in fixes for all the issues you highlighted. Thanks again for such a detailed review. I'll have a new download available later tonight.

The statistical requests will take a bit longer. I have several ideas bouncing around and I need to decide which avenue to pursue.

Battlemaid · 10/07/08, 5:09 PM

"simulate until estimated error < x% for all players".

Back then we had a discussion about variance on our boards (check the post and several following posts here: shadowpriest.com • View topic - SimulationCraft).

General idea is that currently a sequence of averaged simulation output parameters (be it dps or proc uptime) converges too slowly with increasing number of iterations. Mentioned posts attempt to address this problem, however things didnt move further than some not implemented ideas - we had lots of stuff to do at that time.

By the way, while we're talking LatestReport - simulationcraft - Google Code got an update with the latest fixes by Dedmonwakeen. Check it - maybe you'll find some more corrections.

dedmonwakeen · 10/08/08, 12:28 PM

SimulationCraft version r698 available for download.

Summary of changes: RecentUpdates - simulationcraft - Google Code

Sample Output: SampleOutput - simulationcraft - Google Code

Note: Improved Divine Spirit no longer stacks so DPS will see a minor drop across the board.

dedmonwakeen · 10/08/08, 3:52 PM

Originally Posted by Kalle

Can you add statistics to the results? Would be nice to know how many iterations will be neccessary to know for sure if talent/glyph/spell x is a damage increase. I would also like an option "simulate until estimated error < x% for all players".

I've recently checked in code that reports standard deviation for player DPS.

In addition, I report "convergence" data: I take the average DPS when the sim has executed 90% of the iterations. I compare this to the final average DPS to report a convergence metric.

Shortly I'll add an option for controlling the runtime via this convergence metric.

Kalle · 10/09/08, 11:16 AM

Thanks for the quick implementation of my fixes/suggestions.

My question with the quoted part from imp_pet_t was: Why are you doing multiplier *= 1.2 if the buff only increases the critchance?

When I talked about statistics, I thought about error estimates. Currently you calculate the standard deviation your model produces. With average_dmg=1 I don't find that number particularly interesting, because it is something completely different from the deviation you'll see ingame. Sure, you only have to divide by the square root of the number of iterations to get the estimated error, but writing something like (3000 +/- 10) dps would make it easier to understand for people not familiar with statistics, I think (10 dps being the estimated error of the average value 3000 dps). This Google chart thing doesn't support error bars, does it?

Battlemaid · 10/09/08, 12:00 PM

With average_dmg=1 I don't find that number particularly interesting, because it is something completely different from the deviation you'll see ingame

You'll be surprised but even with average_dmg=0 stddev doesn't differ from average_dmg=1. Results for 9 runs of the same config with 10^2 sims, stddev taken from Druid_Balance stats:
average_dmg=0: 3x times 3.0%, 3x times 3.1%, 2x times 3.2%, 1x time 3.3%
average_dmg=1: 4x times 3.1%, 1x time 3.2%, 3x times 3.3%

This Google chart thing doesn't support error bars, does it?

This is what gcharts can do for emulating error bars:

dedmonwakeen · 10/09/08, 1:47 PM

Originally Posted by Kalle

My question with the quoted part from imp_pet_t was: Why are you doing multiplier *= 1.2 if the buff only increases the critchance?

Why? Because the code is bugged and I failed to understand your very clear explanation the first time.

Fix incoming...........

When I talked about statistics, I thought about error estimates. Currently you calculate the standard deviation your model produces. With average_dmg=1 I don't find that number particularly interesting, because it is something completely different from the deviation you'll see ingame. Sure, you only have to divide by the square root of the number of iterations to get the estimated error, but writing something like (3000 +/- 10) dps would make it easier to understand for people not familiar with statistics, I think (10 dps being the estimated error of the average value 3000 dps). This Google chart thing doesn't support error bars, does it?

Thanks for the feedback.... We're still feeling our way through this stuff.

The standard deviation value I have there is: ( std_dev / avg_dps )

What you are looking for is more like this? ( std_dev / sqrt( iterations ) )

GoogleChart provides a freakish amount of functionality with several options for error bars, I believe. Getting to my current level of understanding almost triggered a brain aneurysm, but I'll jump back into the fray.......