SimulationCraft Model Development - Page 11

sjogren · 03/01/09, 4:26 AM

Well, according to Redcape in my JoW thread, the overall proc rate he's seeing is 43%, with a slow weapon, so it can't really be 50% on judgements, can it? Or maybe there's a short internal cooldown for the spell proc explaining why eg judgements get a drastically lower proc rate?

We'd really need some solid testing of proc rate with instant spells, fast spells, slow spells, how haste affects it etc...

dedmonwakeen · 03/01/09, 12:17 PM

Originally Posted by sjogren

Well, according to Redcape in my JoW thread, the overall proc rate he's seeing is 43%, with a slow weapon, so it can't really be 50% on judgements, can it? Or maybe there's a short internal cooldown for the spell proc explaining why eg judgements get a drastically lower proc rate?

We'd really need some solid testing of proc rate with instant spells, fast spells, slow spells, how haste affects it etc...

Misses and dodges could dilute the percentage...... unless these were already pulled out of the sample-set.

EDIT: Just re-read that post in detail. 30 procs in a row with slow weapon auto-attacks should provide a pretty firm MIN value on the PPM. Wish we knew what weapon speed he was using.....

Kalku · 03/02/09, 7:57 PM

Where do you set the scale factor deltas in this new format? I had a look in the obvious places (the sim config files, globals.simcraft, scale_factors.bat) but didn't see it.

dedmonwakeen · 03/02/09, 10:59 PM

Originally Posted by Kalku

Where do you set the scale factor deltas in this new format? I had a look in the obvious places (the sim config files, globals.simcraft, scale_factors.bat) but didn't see it.

Ah.... They default in the code:

scaling_t::scaling_t( sim_t* s ) : sim(s), calculate_scale_factors(0) 
{
  for( int i=ATTRIBUTE_NONE+1; i < ATTRIBUTE_MAX; i++ )
  {
    gear.attribute[ i ] = 250;
  }
  gear.armor_penetration_rating = 250;
  gear.attack_power             = 250;
  gear.crit_rating              = 250;
  gear.expertise_rating         = -150;
  gear.haste_rating             = 250;
  gear.hit_rating               = -200;
  gear.spell_power              = 250;
}

I'll make sure those values are shown in Globals.simcraft so people know how to play with them......

Kalku · 03/02/09, 11:42 PM

Thanks

.

Marco · 03/05/09, 2:26 AM

Has the idea come up before of using identically seeded PRNG streams to get stable scale factors with fewer iterations?

Of course, this isn't trivial, since changing your stats might cause you to cast different spells with the same dice rolls. For example, if you lower your hit rating and that causes you to miss with a DOT, you'll recast the DOT instead of casting a filler spell. Even just raising your spell power will affect when the enemy hits 35%, which might change the spells you cast. So you'd probably want to use a separate PRNG stream for each spell to make this work better, and even then it isn't perfect (e.g. recasting a DOT could cause some amazing crit synergy to happen or not happen, with the same per-spell sequence of dice rolls). But I bet you could get stable scale factors with an order of magnitude fewer iterations this way.

Maybe I'll try this out myself this weekend, if I don't have trouble getting a build working.

(Incidentally, I love the idea of scaling hit and expertise down instead of up to avoid running into the cap. I noticed this issue in conversations about enhsim and I hadn't thought of such an elegant solution.)

Erdluf · 03/05/09, 7:48 AM

Originally Posted by Marco

Has the idea come up before of using identically seeded PRNG streams to get stable scale factors with fewer iterations?

If you use a single seed for the simulation as a whole, you still won't duplicate the sequence as a whole because of timing issues. When scaling for crit, the Druid is going to cast faster (Nature's Grace). A roll that previously went to a Mage's Arcane Blast is now being used for Starfire.

Even if you use independent RNG streams for each toon (or run a single-toon simulation) many scaling factors (haste, crit, damage, hit) will cause cast sequences to be altered. If a Warlock misses with a DoT, he will immediately cast it again. Otherwise he'll use a Shadowbolt. While scaling for hit, the number of Dot's that miss will change, meaning the RNG will get out of sync.

I suppose you could use an independent PRNG stream for each of a toon's spells. Your scale factors might have a lower variance, but I'm inclined to believe the averages would be less trustworthy. As an extreme case, consider a 3s fight where a Moonkin uses a single Starfire. If you always get the same die-roll, a gain of 100 crit rating will always be reported as either a 0% boost to DPS or 100% boost to DPS. The variance will be zero, but the answer will be wrong.

dedmonwakeen · 03/05/09, 10:24 AM

The sim does allow you to override the initial seed...... but that is more of a debug aid than a variance stomper.

To be honest, the easiest way for me to improve scale-factor generation is to improve performance. (The optimal_raid=1 setting was a big part of that.) Periodically, I do a performance review and I think one is past due given the pile of new code dropped in for 3.1.

What is killing me right now are these "downtime" classes. Energy/Focus driven classes have wait periods where nothing in their actions list is "ready". When this happens, the sim puts a "player_ready" event into the queue at current_time+0.1 seconds. This floods the queue, and thrashes the action ready() methods.

I can increase the delta over 0.1sec, but this negatively impacts dps. In some cases (like pets) the action sequence is simple enough that you can put a "wait" action at the end of the list that examines the preceding actions to determine how long to delay for the next player_ready event.

Theoretically, the ready() methods could be changed such that they return a time as opposed a boolean..... but this is more complicated than it sounds......

Marco · 03/05/09, 10:45 AM

Originally Posted by Erdluf

Even if you use independent RNG streams for each toon (or run a single-toon simulation) many scaling factors (haste, crit, damage, hit) will cause cast sequences to be altered. If a Warlock misses with a DoT, he will immediately cast it again. Otherwise he'll use a Shadowbolt. While scaling for hit, the number of Dot's that miss will change, meaning the RNG will get out of sync.

One would definitely want to be careful to use different streams for different decisions to avoid getting out of sync--and on reflection, not just for each spell. For instance, a separate stream for whether a given warlock's shadowbolt misses and whether that warlock's shadowbolt crits, so that you don't get out of sync on whether rolls apply to hit/miss or crit/normal. Admittedly, this is sounding difficult to implement, and the idea of applying "very similar luck" to two different fights is inherently a bit squishy.

Your scale factors might have a lower variance, but I'm inclined to believe the averages would be less trustworthy. As an extreme case, consider a 3s fight where a Moonkin uses a single Starfire. If you always get the same die-roll, a gain of 100 crit rating will always be reported as either a 0% boost to DPS or 100% boost to DPS. The variance will be zero, but the answer will be wrong.

In your extreme case, using consistent rolls doesn't make the result any less trustworthy than using inconsistent rolls. It's wrong because there aren't enough decisions to make it likely that crit (and other) rolls are evenly distributed, but the stability of luck between the more-crit and less-crit simulations isn't degrading the quality of the result.

dedmonwakeen · 03/05/09, 11:15 AM

I keep delaying the release........
My latest excuse: Performance problem in the Hunter module (MM tree related)

In the meantime, check out some PTR data:

SampleOutputPTR - simulationcraft - Google Code

dedmonwakeen · 03/05/09, 1:54 PM

SimulationCraft r1673 available for download.

Still lots of churn going on...... but I wanted to give folks a chance to experiment.

The SampleOutput pages have NOT yet been updated...... since the bulk of the changes were 3.1 related.

Expect to see the wiki pages updated later tonight............

EDIT: One major non-3.1 related addition: Support for Feral Druids (cat only at this point)

Binkenstein · 03/08/09, 10:23 PM

Thought about shifting the heroism use to the start rather than the end?

dedmonwakeen · 03/08/09, 10:28 PM

SimulationCraft r1754 available for download.

3.0.9 report: SampleOutput - simulationcraft - Google Code

3.1.0 report: SampleOutputPTR - simulationcraft - Google Code

dedmonwakeen · 03/08/09, 10:30 PM

Originally Posted by Binkenstein

Thought about shifting the heroism use to the start rather than the end?

Several classes have damage-enhancing talents tied to target health percentages. (Molten Fury, etc...)

• Zakalwe · 03/09/09, 3:34 AM

Originally Posted by dedmonwakeen

Several classes have damage-enhancing talents tied to target health percentages. (Molten Fury, etc...)

I think he's referring to the math quoted here. The math in the original post has been shown not to accurately model certain types of talents and abilities, but a good point is certainly made - that the presence of percentage-based execute talents doesn't necessarily mean you should pop heroism towards the end of the fight.

The discussion seems to still be ongoing, but I think it would be very worthwhile to at least make the bloodlust timing configurable when optimal_raid=1.

EDIT: For what it's worth, rerunning Raid_T8.simcraft after having changed one of the shaman profiles to pop bloodlust at the start yields only about 450 RDPS less than the default run. That's a very tiny .28% difference, pretty much a confirmation for those arguing that it's a wash.