SimulationCraft Model Development - Page 7

sjogren · 03/01/09, 5:26 AM

Well, according to Redcape in my JoW thread, the overall proc rate he's seeing is 43%, with a slow weapon, so it can't really be 50% on judgements, can it? Or maybe there's a short internal cooldown for the spell proc explaining why eg judgements get a drastically lower proc rate?

We'd really need some solid testing of proc rate with instant spells, fast spells, slow spells, how haste affects it etc...

dedmonwakeen · 03/01/09, 1:17 PM

Originally Posted by sjogren

Well, according to Redcape in my JoW thread, the overall proc rate he's seeing is 43%, with a slow weapon, so it can't really be 50% on judgements, can it? Or maybe there's a short internal cooldown for the spell proc explaining why eg judgements get a drastically lower proc rate?

We'd really need some solid testing of proc rate with instant spells, fast spells, slow spells, how haste affects it etc...

Misses and dodges could dilute the percentage...... unless these were already pulled out of the sample-set.

EDIT: Just re-read that post in detail. 30 procs in a row with slow weapon auto-attacks should provide a pretty firm MIN value on the PPM. Wish we knew what weapon speed he was using.....

Kalku · 03/02/09, 8:57 PM

Where do you set the scale factor deltas in this new format? I had a look in the obvious places (the sim config files, globals.simcraft, scale_factors.bat) but didn't see it.

dedmonwakeen · 03/02/09, 11:59 PM

Originally Posted by Kalku

Where do you set the scale factor deltas in this new format? I had a look in the obvious places (the sim config files, globals.simcraft, scale_factors.bat) but didn't see it.

Ah.... They default in the code:

scaling_t::scaling_t( sim_t* s ) : sim(s), calculate_scale_factors(0) 
{
  for( int i=ATTRIBUTE_NONE+1; i < ATTRIBUTE_MAX; i++ )
  {
    gear.attribute[ i ] = 250;
  }
  gear.armor_penetration_rating = 250;
  gear.attack_power             = 250;
  gear.crit_rating              = 250;
  gear.expertise_rating         = -150;
  gear.haste_rating             = 250;
  gear.hit_rating               = -200;
  gear.spell_power              = 250;
}

I'll make sure those values are shown in Globals.simcraft so people know how to play with them......

Kalku · 03/03/09, 12:42 AM

Thanks

.

Marco · 03/05/09, 3:26 AM

Has the idea come up before of using identically seeded PRNG streams to get stable scale factors with fewer iterations?

Of course, this isn't trivial, since changing your stats might cause you to cast different spells with the same dice rolls. For example, if you lower your hit rating and that causes you to miss with a DOT, you'll recast the DOT instead of casting a filler spell. Even just raising your spell power will affect when the enemy hits 35%, which might change the spells you cast. So you'd probably want to use a separate PRNG stream for each spell to make this work better, and even then it isn't perfect (e.g. recasting a DOT could cause some amazing crit synergy to happen or not happen, with the same per-spell sequence of dice rolls). But I bet you could get stable scale factors with an order of magnitude fewer iterations this way.

Maybe I'll try this out myself this weekend, if I don't have trouble getting a build working.

(Incidentally, I love the idea of scaling hit and expertise down instead of up to avoid running into the cap. I noticed this issue in conversations about enhsim and I hadn't thought of such an elegant solution.)

Erdluf · 03/05/09, 8:48 AM

Originally Posted by Marco

Has the idea come up before of using identically seeded PRNG streams to get stable scale factors with fewer iterations?

If you use a single seed for the simulation as a whole, you still won't duplicate the sequence as a whole because of timing issues. When scaling for crit, the Druid is going to cast faster (Nature's Grace). A roll that previously went to a Mage's Arcane Blast is now being used for Starfire.

Even if you use independent RNG streams for each toon (or run a single-toon simulation) many scaling factors (haste, crit, damage, hit) will cause cast sequences to be altered. If a Warlock misses with a DoT, he will immediately cast it again. Otherwise he'll use a Shadowbolt. While scaling for hit, the number of Dot's that miss will change, meaning the RNG will get out of sync.

I suppose you could use an independent PRNG stream for each of a toon's spells. Your scale factors might have a lower variance, but I'm inclined to believe the averages would be less trustworthy. As an extreme case, consider a 3s fight where a Moonkin uses a single Starfire. If you always get the same die-roll, a gain of 100 crit rating will always be reported as either a 0% boost to DPS or 100% boost to DPS. The variance will be zero, but the answer will be wrong.

dedmonwakeen · 03/05/09, 11:24 AM

The sim does allow you to override the initial seed...... but that is more of a debug aid than a variance stomper.

To be honest, the easiest way for me to improve scale-factor generation is to improve performance. (The optimal_raid=1 setting was a big part of that.) Periodically, I do a performance review and I think one is past due given the pile of new code dropped in for 3.1.

What is killing me right now are these "downtime" classes. Energy/Focus driven classes have wait periods where nothing in their actions list is "ready". When this happens, the sim puts a "player_ready" event into the queue at current_time+0.1 seconds. This floods the queue, and thrashes the action ready() methods.

I can increase the delta over 0.1sec, but this negatively impacts dps. In some cases (like pets) the action sequence is simple enough that you can put a "wait" action at the end of the list that examines the preceding actions to determine how long to delay for the next player_ready event.

Theoretically, the ready() methods could be changed such that they return a time as opposed a boolean..... but this is more complicated than it sounds......

Marco · 03/05/09, 11:45 AM

Originally Posted by Erdluf

Even if you use independent RNG streams for each toon (or run a single-toon simulation) many scaling factors (haste, crit, damage, hit) will cause cast sequences to be altered. If a Warlock misses with a DoT, he will immediately cast it again. Otherwise he'll use a Shadowbolt. While scaling for hit, the number of Dot's that miss will change, meaning the RNG will get out of sync.

One would definitely want to be careful to use different streams for different decisions to avoid getting out of sync--and on reflection, not just for each spell. For instance, a separate stream for whether a given warlock's shadowbolt misses and whether that warlock's shadowbolt crits, so that you don't get out of sync on whether rolls apply to hit/miss or crit/normal. Admittedly, this is sounding difficult to implement, and the idea of applying "very similar luck" to two different fights is inherently a bit squishy.

Your scale factors might have a lower variance, but I'm inclined to believe the averages would be less trustworthy. As an extreme case, consider a 3s fight where a Moonkin uses a single Starfire. If you always get the same die-roll, a gain of 100 crit rating will always be reported as either a 0% boost to DPS or 100% boost to DPS. The variance will be zero, but the answer will be wrong.

In your extreme case, using consistent rolls doesn't make the result any less trustworthy than using inconsistent rolls. It's wrong because there aren't enough decisions to make it likely that crit (and other) rolls are evenly distributed, but the stability of luck between the more-crit and less-crit simulations isn't degrading the quality of the result.

dedmonwakeen · 03/05/09, 12:15 PM

I keep delaying the release........
My latest excuse: Performance problem in the Hunter module (MM tree related)

In the meantime, check out some PTR data:

SampleOutputPTR - simulationcraft - Google Code

dedmonwakeen · 03/05/09, 2:54 PM

SimulationCraft r1673 available for download.

Still lots of churn going on...... but I wanted to give folks a chance to experiment.

The SampleOutput pages have NOT yet been updated...... since the bulk of the changes were 3.1 related.

Expect to see the wiki pages updated later tonight............

EDIT: One major non-3.1 related addition: Support for Feral Druids (cat only at this point)

• Binkenstein · 03/08/09, 11:23 PM

Thought about shifting the heroism use to the start rather than the end?

dedmonwakeen · 03/08/09, 11:28 PM

SimulationCraft r1754 available for download.

3.0.9 report: SampleOutput - simulationcraft - Google Code

3.1.0 report: SampleOutputPTR - simulationcraft - Google Code

dedmonwakeen · 03/08/09, 11:30 PM

Originally Posted by Binkenstein

Thought about shifting the heroism use to the start rather than the end?

Several classes have damage-enhancing talents tied to target health percentages. (Molten Fury, etc...)

• Zakalwe · 03/09/09, 4:34 AM

Originally Posted by dedmonwakeen

Several classes have damage-enhancing talents tied to target health percentages. (Molten Fury, etc...)

I think he's referring to the math quoted here. The math in the original post has been shown not to accurately model certain types of talents and abilities, but a good point is certainly made - that the presence of percentage-based execute talents doesn't necessarily mean you should pop heroism towards the end of the fight.

The discussion seems to still be ongoing, but I think it would be very worthwhile to at least make the bloodlust timing configurable when optimal_raid=1.

EDIT: For what it's worth, rerunning Raid_T8.simcraft after having changed one of the shaman profiles to pop bloodlust at the start yields only about 450 RDPS less than the default run. That's a very tiny .28% difference, pretty much a confirmation for those arguing that it's a wash.

dedmonwakeen · 03/09/09, 9:26 AM

Originally Posted by Zakalwe

I think he's referring to the math quoted here. The math in the original post has been shown not to accurately model certain types of talents and abilities, but a good point is certainly made - that the presence of percentage-based execute talents doesn't necessarily mean you should pop heroism towards the end of the fight.

The discussion seems to still be ongoing, but I think it would be very worthwhile to at least make the bloodlust timing configurable when optimal_raid=1.

EDIT: For what it's worth, rerunning Raid_T8.simcraft after having changed one of the shaman profiles to pop bloodlust at the start yields only about 450 RDPS less than the default run. That's a very tiny .28% difference, pretty much a confirmation for those arguing that it's a wash.

Interesting...... While popping cooldowns right away (which is what most simcraft AI do) guarantees that you maximize the number of times they come up, a fight with a reasonably well known (+/- 1min) fight length will let good players "have their cake and eat it, too". If you imagine that these large DPS enhancing CDs are the tines of a fork, you can slide the whole fork back and forth...... and get that last tine to land in BL territory. This gives you both the CD and the Execute.

And you don't need to be omniscient, either: Pop the CD right at the start just like normal..... but as you get close to that last 1min of the fight, you may choose to delay the next use of the CD-ability such that it lands inside BL.

• Zakalwe · 03/09/09, 9:59 AM

It's still easier to synchronize CDs with BL if you just pop everything at the beginning of the fight. And because the math quoted in that blog post seemed to show that the results will be exactly equal no matter when the BL occurs (given perfect CD/BL timing) it was concluded that one should always do it right away.

But in the comments to that blog post we've shown that some classes have execute rotations that scale better with haste than their regular rotations, and that this isn't accounted for in the original math. So the actual answer to the question "when should we pop BL?" turns out to depend on exactly how much of your DPS is provided by affliction warlocks and TG warriors. In other words, something you might want to analyze with Simulationcraft.

EDIT: Regarding the difficulty of timing CDs to occur during BL, keep in mind that there's an increasing number of trinkets that have a random buff with a relatively high proc chance and a relatively long cooldown. It's very likely for such buffs to fall entirely within the first 40 seconds of a fight, and much less likely for such buffs to fall entirely within the last 40 seconds of a fight.

dedmonwakeen · 03/09/09, 12:01 PM

Originally Posted by Zakalwe

EDIT: Regarding the difficulty of timing CDs to occur during BL, keep in mind that there's an increasing number of trinkets that have a random buff with a relatively high proc chance and a relatively long cooldown. It's very likely for such buffs to fall entirely within the first 40 seconds of a fight, and much less likely for such buffs to fall entirely within the last 40 seconds of a fight.

Good point. I believe one of the two trinkets for each base setup (caster/melee) has one of those trinkets and so was included in your test.

• frmorrison · 03/09/09, 11:19 PM

Originally Posted by dedmonwakeen

If you assume that the Judgments are "spells" and pull those out of the sample set, then 14PPM starts to look like a pretty solid model.

Late answer, but Judgements for Paladins could as ranged attacks (like hunter attacks). So if a spell can proc for ranged and melee attacks seperately, a Paladin would get extra procs.

• Binkenstein · 03/10/09, 8:58 PM

Originally Posted by dedmonwakeen

Interesting...... While popping cooldowns right away (which is what most simcraft AI do) guarantees that you maximize the number of times they come up, a fight with a reasonably well known (+/- 1min) fight length will let good players "have their cake and eat it, too". If you imagine that these large DPS enhancing CDs are the tines of a fork, you can slide the whole fork back and forth...... and get that last tine to land in BL territory. This gives you both the CD and the Execute.

And you don't need to be omniscient, either: Pop the CD right at the start just like normal..... but as you get close to that last 1min of the fight, you may choose to delay the next use of the CD-ability such that it lands inside BL.

It's more a test of the theory than anything else.

dedmonwakeen · 03/10/09, 9:26 PM

Originally Posted by Binkenstein

It's more a test of the theory than anything else.

Actually...... You've convinced me..... Zak did some runs popping BL early and the raid dps was virtually the same.

Kalle · 03/14/09, 7:28 PM

I did some pet tests at the training dummies on the testrealm and noticed that recount's ability breakdown differs from simcraft's summary, at least for the succubus. The level 83 dummy tests were done in build 9658, the tests with the level 80 dummies in build 9684.

Melee_lvl80 (1), LoP_lvl80 (1), Melee_lvl80 (2), LoP_lvl80 (2), Melee_lvl83, LoP_lvl83

Lash of Pain only crits for 1.5x damage
Lash of Pain cannot be dodged
Lash of Pain Damage seems to be lower than in simcraft, I think you forgot the (1.5/3.5) multiplier to the succubus' spell power (casting speed adjustment).
Although Lash of Pain looks like a spell, it doesn't seem to use spell hit mechanics. I see only 1.6% miss for the level 83 dummy while having 211 hit rating (= 8.04% spell hit). It doesn't simply use the melee cap (8% according to [FAQ]Working theories of raiding at level 80) either. Probably 8% cap with melee hit rating formula (1% hit = 32.79 rating): 8% - 211*(819/26855) = 1.57% miss.
Normal melee attacks (from the succubus) however seem to use the spell hit rating formula (1% hit = 26.23 rating), because I didn't see a single miss in 6007 attacks with 211 hit rating. With 200 hit rating, I see misses.
I have no idea how critchances for the succubus are calculated. But you clearly see the critrate depression against the level 83 dummy for her melee attacks. The 13.3% crit simcraft reports seems quite low considering the fact that I had zero raidbuffs during my tests.
Oh, and I observed 25.3% glancing strikes which doesn't support the 24% chance most people here seems to agree on (at least this was my impression from looking at some of the search results for "glancing", however I don't play a melee class and don't follow their discussions) and what is used in simcraft. I think I saw a test with 7000+ attacks that resulted in 24%, maybe this has changed? Or it is different for pets? I don't know.

If you still need data on pet mana regeneration, I think I have enough free time now.

dedmonwakeen · 03/14/09, 11:26 PM

Originally Posted by Kalle

I did some pet tests at the training dummies on the testrealm and noticed that recount's ability breakdown differs from simcraft's summary, at least for the succubus. The level 83 dummy tests were done in build 9658, the tests with the level 80 dummies in build 9684.

Melee_lvl80 (1), LoP_lvl80 (1), Melee_lvl80 (2), LoP_lvl80 (2), Melee_lvl83, LoP_lvl83

Lash of Pain only crits for 1.5x damage
Lash of Pain cannot be dodged
Lash of Pain Damage seems to be lower than in simcraft, I think you forgot the (1.5/3.5) multiplier to the succubus' spell power (casting speed adjustment).
Although Lash of Pain looks like a spell, it doesn't seem to use spell hit mechanics. I see only 1.6% miss for the level 83 dummy while having 211 hit rating (= 8.04% spell hit). It doesn't simply use the melee cap (8% according to [FAQ]Working theories of raiding at level 80) either. Probably 8% cap with melee hit rating formula (1% hit = 32.79 rating): 8% - 211*(819/26855) = 1.57% miss.
Normal melee attacks (from the succubus) however seem to use the spell hit rating formula (1% hit = 26.23 rating), because I didn't see a single miss in 6007 attacks with 211 hit rating. With 200 hit rating, I see misses.
I have no idea how critchances for the succubus are calculated. But you clearly see the critrate depression against the level 83 dummy for her melee attacks. The 13.3% crit simcraft reports seems quite low considering the fact that I had zero raidbuffs during my tests.
Oh, and I observed 25.3% glancing strikes which doesn't support the 24% chance most people here seems to agree on (at least this was my impression from looking at some of the search results for "glancing", however I don't play a melee class and don't follow their discussions) and what is used in simcraft. I think I saw a test with 7000+ attacks that resulted in 24%, maybe this has changed? Or it is different for pets? I don't know.

If you still need data on pet mana regeneration, I think I have enough free time now.

Kalle, thanks (as usual) for the exceptional level of detail. Looks like we need revisit Succy.....

And YES we definitely need hard data on pet mana regen. That is a big glaring hole in our model right now......

• Zakalwe · 03/15/09, 5:49 AM

My tests several months ago seemed to indicate PetMP5 ~= 0.3*WarlockIntellect. I couldn't quite get the numbers to match up to that simple equation, though it was hard to figure out if that was because of inaccurate testing methodology or because the equation is actually more complex, maybe involving spirit or some other factor.

I know very little about regen mechanics in general, but my initial reaction was that pet Mp5 scaling directly off the master's Intellect seemed pretty odd. And since pets running out of mana practically never happens in 3.0.X, I kind of left it at that. I do imagine pet mana issues might become more pressing in 3.1, though, so if anyone could independently confirm my result, we should implement it ASAP.

dedmonwakeen · 03/16/09, 5:27 PM

SimulationCraft r1851 available for download.

3.0.9 report: SampleOutput - simulationcraft - Google Code

3.1.0 report: SampleOutputPTR - simulationcraft - Google Code

A couple things to note:
(1) JoW is being modeled as 15PPM for both weapons and spells
(2) BoW and Mana Spring do not stack when patch=3.0.1