Elitist Jerks
Register
Blogs
Forums


Go Back   Elitist Jerks » Public Discussion » Public Discussion

Reply
 
LinkBack Thread Tools
Old 10/02/08, 2:12 PM   #1
dedmonwakeen
Bald Bull
 
dedmonwakeen
Undead Priest
 
No WoW Account
SimulationCraft Model Development

SimulationCraft is an event-driven multi-player DPS simulator with modules available for all classes.

Last edited by dedmonwakeen : 02/06/11 at 1:59 PM.


Offline
Reply With Quote
Old 10/07/08, 11:28 AM   #2
sp00n
Bald Bull
 
Night Elf Rogue
 
Wrathbringer (EU)
Are there already any melee mechanics available? Or would they have to be completely (re-)written?

I stumbled across the postings in the mage WotLK threads, and I'd really like to see how rogues compare to the caster numbers.

That said, I have exactly zilch knowledge of C++, so unfortunately am of no help you. I'm not sure if Aldriana has any C++ knowledge, but including his RogueCalc - which is written in Python - into a larger simulation tool would be the ideal solution I guess.

Stopped Playing

Offline
Reply With Quote
Old 10/07/08, 11:40 AM   #3
Skellum
Bald Bull
 
Skellum's Avatar
 
Worgen Warlock
 
Arthas
Is there any chance of formatting the output a bit? I love the bar graphs but I hate the fact that it doesnt include any specific information about the sample data its collected from. By modifying the Simcraft input I can give high crit and haste rates with low +dmg rates and start shunting this all about a bunch of forums,non-EJ, in a rabble rousing attempt.

It would be really amazing if the output automatically provided a link to the data source or was linked to it to where excluding the source data would be difficult to do and be more effort than the average troll would put in to creating a disruption with it.

My Name Is Skellum, And I wear a Silly Hat.

Offline
Reply With Quote
Old 10/07/08, 11:51 AM   #4
Kalle
Von Kaiser
 
Undead Warlock
 
Wrathbringer (EU)
I found some problems with warlocks in r688:
  • Unstable Affliction was changed to 15s duration, it now has only 5 ticks, new coefficient is 1.0
  • Shadow Embrace debuff lasts for 12s ingame, not 15s
  • Improved Soul Leech has two ranks ingame, restoring 1%/2% of total Mana
  • Duration of Demonic Empowerment is different for imp and for felguard now
  • In trigger_backdraft you wrote "sim -> add_event( this, 15.0 );" and "e -> reschedule( 12.0 );", the second one should also be 15s, shouldn't it?
  • Deathcoil is an affliction spell ingame, not destruction
  • The <25% multiplier for Drain Soul is only present for rank 6 ingame
  • Haunt's crit bonus was lowered to 50%
  • Soul Fire doesn't get base_execute_time / 3.5 for its coefficient. Quick testing shows something between 1.112 and 1.182, probably the 1.15 mentioned here.
  • Dark Pact is not affected by Shadow Mastery ingame
  • Chaos Bolt cannot miss, but I see partial resits - not sure on the mechanics yet
  • Metamorphosis doesn't trigger the gcd ingame
  • Melee attacks from pets may be glancing hits ingame
  • I retested Master Conjuror in build 9038 and the DD/DoT damage increasing portion of fire-/spellstone is still not affected by it

In imp_pet_t:
    virtual void player_buff()
    {
      imp_pet_t* p = (imp_pet_t*) player -> cast_pet();
      warlock_pet_spell_t::player_buff();
      if( p -> buffs_demonic_empowerment )
      {
	player_multiplier *= 1.20;
	player_crit       += 0.20;
      }
    }
Demonic Empowerment only increases the critchance for the imp. I suppose, the player_multiplier part should be removed.

Regarding your combat rating interpolation, WoWWiki has formulas on that.

In the default raid_80.txt, the destruction warlock does not use Corruption but has glyph_corruption=1. For me it was a dps increase to include Corruption in the action list.

Can you add statistics to the results? Would be nice to know how many iterations will be neccessary to know for sure if talent/glyph/spell x is a damage increase. I would also like an option "simulate until estimated error < x% for all players".

Offline
Reply With Quote
Old 10/07/08, 11:53 AM   #5
Mearis
Mr. Sandman
 
Mearis's Avatar
 
Dwarf Priest
 
Defias Brotherhood (EU)
Sent you a PM

Offline
Reply With Quote
Old 10/07/08, 1:21 PM   #6
dedmonwakeen
Bald Bull
 
dedmonwakeen
Undead Priest
 
No WoW Account
Originally Posted by sp00n View Post
Are there already any melee mechanics available? Or would they have to be completely (re-)written?

I stumbled across the postings in the mage WotLK threads, and I'd really like to see how rogues compare to the caster numbers.

That said, I have exactly zilch knowledge of C++, so unfortunately am of no help you. I'm not sure if Aldriana has any C++ knowledge, but including his RogueCalc - which is written in Python - into a larger simulation tool would be the ideal solution I guess.
This is actually my third attempt over the past three years to model DPS. The first attempt was for Shadow Priests and it used the standard "formulation" method of enforced rotations, etc. This led me to simulation and I opened up support for all caster classes. For WotLK I did a complete re-write to create an architecture that would support melee classes as well.

So far I only have Enhancement Shaman written........ which is somewhat superfluous because there already exists two very good Enh-Shaman simulators! I do intend, barring personal catastrophe, to have all the classes implemented. The class currently in development is Paladin.... with Hunters to follow. (I wanted to get through all the mana-based classes first.)

Aldriana is a very good developer with exceptional insight into the Rogue class, but I highly doubt he will be interested in contributing to SimulationCraft. There are some things that simulators are good at..... and there are some things that simulations are NOT good at.... and Aldriana understands this quite well. Provided you have invested sufficient thought into your model, formulation is much better at delivering consistent EP values. He and I have had many conversations on this subject and we understand the need for both Formulation and Simulation tools.

My thoughts on the subject can be found here: FormulationVsSimulation - simulationcraft - Google Code

All that being said..... I'd be THRILLED to have someone like Aldriana on the team. Detailed knowledge of WoW mechanics is FAR more important than deep familiarity with obscure/obscene features of C++. Indeed, if I knew there was a motivated individual with sufficient experience waiting in the wings, I would change the order in which new classes are added to the sim just to get them involved sooner...........

Originally Posted by Skellum View Post
Is there any chance of formatting the output a bit?
By default, the tool generates TONS of text output...... but I have no way of enforcing people to post that along with the graphs. The SampleOutput page (found by clicking through my sig) has both the charts and the detailed text, as well as a link to the config file. It is capable of generating HTML and WIKI format for the charts..... and I do intend to add the detailed text output to the HTML and WIKI. However, I still can't guarantee that people will host an entire HTML file as opposed to just posting a GoogleChart http link.

Last edited by dedmonwakeen : 10/07/08 at 1:27 PM.


Offline
Reply With Quote
Old 10/07/08, 1:26 PM   #7
Battlemaid
Glass Joe
 
Night Elf Priest
 
Jaedenar (EU)
It would be really amazing if the output automatically provided a link to the data source or was linked to it to where excluding the source data would be difficult to do and be more effort than the average troll would put in to creating a disruption with it.
At least, the ethalon SimCraft report: LatestReport - simulationcraft - Google Code doesn't suffer from such issues since it includes links to input and output artifacts of SimCraft in the wiki page text.

However, it doesn't inline simulation artifacts right into HTML, but rather uploads those to the server and stores them next to the report web page. The simplest idea would be to include input/output files in the HTML text, but toggle its visibility via JavaScript. Tho this still leaves the charts images by themselves being incomprehensive and susceptible to frauds that you described.

Offline
Reply With Quote
Old 10/07/08, 1:42 PM   #8
dedmonwakeen
Bald Bull
 
dedmonwakeen
Undead Priest
 
No WoW Account
Originally Posted by Kalle View Post
I found some problems with warlocks in r688:
  • Unstable Affliction was changed to 15s duration, it now has only 5 ticks, new coefficient is 1.0
  • Shadow Embrace debuff lasts for 12s ingame, not 15s
  • Improved Soul Leech has two ranks ingame, restoring 1%/2% of total Mana
  • Duration of Demonic Empowerment is different for imp and for felguard now
  • In trigger_backdraft you wrote "sim -> add_event( this, 15.0 );" and "e -> reschedule( 12.0 );", the second one should also be 15s, shouldn't it?
  • Deathcoil is an affliction spell ingame, not destruction
  • The <25% multiplier for Drain Soul is only present for rank 6 ingame
  • Haunt's crit bonus was lowered to 50%
  • Soul Fire doesn't get base_execute_time / 3.5 for its coefficient. Quick testing shows something between 1.112 and 1.182, probably the 1.15 mentioned here.
  • Dark Pact is not affected by Shadow Mastery ingame
  • Chaos Bolt cannot miss, but I see partial resits - not sure on the mechanics yet
  • Metamorphosis doesn't trigger the gcd ingame
  • Melee attacks from pets may be glancing hits ingame
  • I retested Master Conjuror in build 9038 and the DD/DoT damage increasing portion of fire-/spellstone is still not affected by it
THANK YOU! I will make the requisite changes ASAP!

In imp_pet_t:
Demonic Empowerment only increases the critchance for the imp. I suppose, the player_multiplier part should be removed.

Regarding your combat rating interpolation, WoWWiki has formulas on that.
Only the Imp is getting buffed. Pets are considered "players" by the core architecture. I probably should have used the term "actor". The pet_t class is a sub-class of player_t. The spell_t class holds the calculated set of buffs/debuffs/etc for each spell. The dynamic values associated with an "actor" are all of the form "player_xxx".

I'm afraid I was very cowardly with my implementation of Ratings-conversion with the hope that most people would run with either level 70 or level 80 players. I will look into the information and put it on the TODO list, but it may get prioritized a bit low for now.

In the default raid_80.txt, the destruction warlock does not use Corruption but has glyph_corruption=1. For me it was a dps increase to include Corruption in the action list.

Can you add statistics to the results? Would be nice to know how many iterations will be neccessary to know for sure if talent/glyph/spell x is a damage increase. I would also like an option "simulate until estimated error < x% for all players".
The Destruction profile has been in constant flux due to the recent code reviews to ferret out the remaining problems with Warlock support. Corruption keeps getting put in and out depending upon who touched the profile last and their particular opinion in the "Is the extra spell worth the benefit."

The statistics idea is very interesting. I supposed an option to simulate for a certain number of CPU seconds (as opposed to iterations) might also be useful. I'll see what I can do.

PS: Sorry for the odd "two topics per quote-segmentation". This is already a back-to-back post and if I made it too granular I'd likely pick up another infraction.


Offline
Reply With Quote
Old 10/07/08, 4:05 PM   #9
dedmonwakeen
Bald Bull
 
dedmonwakeen
Undead Priest
 
No WoW Account
@Kalle: I've checked in fixes for all the issues you highlighted. Thanks again for such a detailed review. I'll have a new download available later tonight.

The statistical requests will take a bit longer. I have several ideas bouncing around and I need to decide which avenue to pursue.


Offline
Reply With Quote
Old 10/07/08, 5:09 PM   #10
Battlemaid
Glass Joe
 
Night Elf Priest
 
Jaedenar (EU)
"simulate until estimated error < x% for all players".
Back then we had a discussion about variance on our boards (check the post and several following posts here: shadowpriest.com &bull; View topic - SimulationCraft).

General idea is that currently a sequence of averaged simulation output parameters (be it dps or proc uptime) converges too slowly with increasing number of iterations. Mentioned posts attempt to address this problem, however things didnt move further than some not implemented ideas - we had lots of stuff to do at that time.

By the way, while we're talking LatestReport - simulationcraft - Google Code got an update with the latest fixes by Dedmonwakeen. Check it - maybe you'll find some more corrections.

Offline
Reply With Quote
Old 10/08/08, 12:28 PM   #11
dedmonwakeen
Bald Bull
 
dedmonwakeen
Undead Priest
 
No WoW Account
SimulationCraft version r698 available for download.

Summary of changes: RecentUpdates - simulationcraft - Google Code

Sample Output: SampleOutput - simulationcraft - Google Code

Note: Improved Divine Spirit no longer stacks so DPS will see a minor drop across the board.


Offline
Reply With Quote
Old 10/08/08, 3:52 PM   #12
dedmonwakeen
Bald Bull
 
dedmonwakeen
Undead Priest
 
No WoW Account
Originally Posted by Kalle View Post
Can you add statistics to the results? Would be nice to know how many iterations will be neccessary to know for sure if talent/glyph/spell x is a damage increase. I would also like an option "simulate until estimated error < x% for all players".
I've recently checked in code that reports standard deviation for player DPS.

In addition, I report "convergence" data: I take the average DPS when the sim has executed 90% of the iterations. I compare this to the final average DPS to report a convergence metric.

Shortly I'll add an option for controlling the runtime via this convergence metric.


Offline
Reply With Quote
Old 10/09/08, 11:16 AM   #13
Kalle
Von Kaiser
 
Undead Warlock
 
Wrathbringer (EU)
Thanks for the quick implementation of my fixes/suggestions.

My question with the quoted part from imp_pet_t was: Why are you doing multiplier *= 1.2 if the buff only increases the critchance?

When I talked about statistics, I thought about error estimates. Currently you calculate the standard deviation your model produces. With average_dmg=1 I don't find that number particularly interesting, because it is something completely different from the deviation you'll see ingame. Sure, you only have to divide by the square root of the number of iterations to get the estimated error, but writing something like (3000 +/- 10) dps would make it easier to understand for people not familiar with statistics, I think (10 dps being the estimated error of the average value 3000 dps). This Google chart thing doesn't support error bars, does it?

Offline
Reply With Quote
Old 10/09/08, 12:00 PM   #14
Battlemaid
Glass Joe
 
Night Elf Priest
 
Jaedenar (EU)
With average_dmg=1 I don't find that number particularly interesting, because it is something completely different from the deviation you'll see ingame
You'll be surprised but even with average_dmg=0 stddev doesn't differ from average_dmg=1. Results for 9 runs of the same config with 10^2 sims, stddev taken from Druid_Balance stats:
average_dmg=0: 3x times 3.0%, 3x times 3.1%, 2x times 3.2%, 1x time 3.3%
average_dmg=1: 4x times 3.1%, 1x time 3.2%, 3x times 3.3%

This Google chart thing doesn't support error bars, does it?
This is what gcharts can do for emulating error bars:


Offline
Reply With Quote
Old 10/09/08, 1:47 PM   #15
dedmonwakeen
Bald Bull
 
dedmonwakeen
Undead Priest
 
No WoW Account
Originally Posted by Kalle View Post
My question with the quoted part from imp_pet_t was: Why are you doing multiplier *= 1.2 if the buff only increases the critchance?
Why? Because the code is bugged and I failed to understand your very clear explanation the first time. Fix incoming...........

When I talked about statistics, I thought about error estimates. Currently you calculate the standard deviation your model produces. With average_dmg=1 I don't find that number particularly interesting, because it is something completely different from the deviation you'll see ingame. Sure, you only have to divide by the square root of the number of iterations to get the estimated error, but writing something like (3000 +/- 10) dps would make it easier to understand for people not familiar with statistics, I think (10 dps being the estimated error of the average value 3000 dps). This Google chart thing doesn't support error bars, does it?
Thanks for the feedback.... We're still feeling our way through this stuff.

The standard deviation value I have there is: ( std_dev / avg_dps )

What you are looking for is more like this? ( std_dev / sqrt( iterations ) )

GoogleChart provides a freakish amount of functionality with several options for error bars, I believe. Getting to my current level of understanding almost triggered a brain aneurysm, but I'll jump back into the fray.......


Offline
Reply With Quote
Reply

Go Back   Elitist Jerks » Public Discussion » Public Discussion

Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
Rawr (v2.3.23 released on 9/12) Astrylian Public Discussion 2725 12/10/12 2:32 PM
SimulationCraft: Multi-Player Sim for WotLK dedmonwakeen Class Mechanics 4 07/26/08 7:52 AM
WoW Addon Development - how to? Moogul User Interface and AddOns 13 01/01/08 10:47 AM
The development cycle is... ? Howard Roark Public Discussion 5 10/31/07 6:37 AM
From TheoryCraft to SimulationCraft dedmonwakeen Class Mechanics 18 06/09/07 5:03 PM