SimulationCraft, a tool Mograin, Rivendare, Gorefiend and Thassarian would have used - Page 3

tripp6sic6 · 12/01/09, 12:48 PM

I see a possible bug in the 3.3 Scourge Strike. On the pie chart that shows the damage sources of the player, scourge_strike and scourge_strike_shadow are both colored purple instead of scourge_strike being brown and scourge_strike_shadow being purple. Does this mean that the physical portion is doing shadow damage or is it simply a visual bug?

Starfox · 12/01/09, 2:01 PM

Originally Posted by tripp6sic6

I see a possible bug in the 3.3 Scourge Strike. On the pie chart that shows the damage sources of the player, scourge_strike and scourge_strike_shadow are both colored purple instead of scourge_strike being brown and scourge_strike_shadow being purple. Does this mean that the physical portion is doing shadow damage or is it simply a visual bug?

Ah, just a visual bug, I only set the school on the action itself to physical. Forgot to also change it in the stats for SS.
Fixed in r4027

♦ Darkside · 12/01/09, 2:07 PM

Originally Posted by Starfox

Edit: Tundra Stalker was not in. Added it :/

Whoa, that amped up Frost Damage by quite a bit. The damage from Oblit is still several thousand lower than what it is on live though

I've been doing a few sims to get an idea of stat weights, but I've noticed that there can be absolutely enormous variance in the values produced. For example:

3.2104 1.1999 1.0000 4.1707 1.8148 2.9310 2.0621 1.9866 8.2241 0.0000 0.0000 0.0000

2.0750 1.0161 1.0000 1.7200 1.3800 1.1491 1.4473 1.3451 5.3868 0.0000 0.0000 0.0000

I got those values on back to back simulations. Is such an incredible amount of deviation commonplace for such calculations or unique to the Death Knight model?

dedmonwakeen · 12/01/09, 2:14 PM

I have found that it takes about 10k ( or even 20k ) iterations to get reasonable scale factors.

We've experimented a ton with contorting the various RNG packages (aka smooth_rng=1). This will provide repeatability, but there are some stats such as Crit Rating where I'm just happy with results yet.

Starfox · 12/01/09, 2:15 PM

Originally Posted by Darkside

Whoa, that amped up Frost Damage by quite a bit. The damage from Oblit is still several thousand lower than what it is on live though

I've been doing a few sims to get an idea of stat weights, but I've noticed that there can be absolutely enormous variance in the values produced. For example:

3.2104 1.1999 1.0000 4.1707 1.8148 2.9310 2.0621 1.9866 8.2241 0.0000 0.0000 0.0000

2.0750 1.0161 1.0000 1.7200 1.3800 1.1491 1.4473 1.3451 5.3868 0.0000 0.0000 0.0000

I got those values on back to back simulations. Is such an incredible amount of deviation commonplace for such calculations or unique to the Death Knight model?

Oh, that is clearly an issue here.
It is pretty common when you forget to reset things, don't initialize variables etc. :S
One of the included profiles? Or was it just out of line for every DK profile?

♦ Darkside · 12/01/09, 2:21 PM

Those values were calculated using my character's gear. I'll check again later with the provided profiles.

I've noticed that the error decreases if I run more simulation iterations, but it is still too large for comfort (on the order of 10-15%).

Originally Posted by dedmonwakeen

I have found that it takes about 10k ( or even 20k ) iterations to get reasonable scale factors.

We've experimented a ton with contorting the various RNG packages (aka smooth_rng=1). This will provide repeatability, but there are some stats such as Crit Rating where I'm just happy with results yet.

That seems odd, especially since I noticed that my damage output for both simulations was essentially the same (very minor deviance, maybe 2-3%). What is happening to produce such wild swings in the stat weights?

Starfox · 12/01/09, 2:42 PM

Originally Posted by Darkside

Those values were calculated using my character's gear. I'll check again later with the provided profiles.

I've noticed that the error decreases if I run more simulation iterations, but it is still too large for comfort (on the order of 10-15%).

That seems odd, especially since I noticed that my damage output for both simulations was essentially the same (very minor deviance, maybe 2-3%). What is happening to produce such wild swings in the stat weights?

There is clearly an issue, consecutive iterations get slower and slower.
For scale factors you need very high iteration counts, that's when I noticed the issue (I rune scalefactors with iterations=50k)

dedmonwakeen · 12/01/09, 3:42 PM

Originally Posted by Darkside

Those values were calculated using my character's gear. I'll check again later with the provided profiles.

I've noticed that the error decreases if I run more simulation iterations, but it is still too large for comfort (on the order of 10-15%).

That error value is not only outside of "comfort" but well into "bug" range. We will dive in.

EDIT: I solved the runtime issue, but I doubt this will solve the variance problem. I will continue analysis.

tripp6sic6 · 12/02/09, 11:15 AM

I edited the current Unholy profile to use the rotation that will be used in 3.3 and am getting the same variance he talked about. In three different runs at 10,000 iterations with no smooth RNG, here were the results.

2.0116 1.7981 1.0228 0.0827 1.6544 1.4802 2.0179 1.8565 5.7330
2.0116 1.5881 0.7054 0.5889 1.3504 1.7968 1.6783 1.7021 4.7066
1.6978 0.8282 0.7130 1.6894 1.3522 2.5918 1.3952 1.1694 3.7944

death_knight=Death_Knight_T9_17_00_54
origin=http://profiler.wowhead.com/?profile=18127877
level=80
race=tauren
talents=http://www.wowhead.com/?talent#jfVMVZZfMbI0cdg0ux0uIqo
glyphs=the_ghoul/dark_death/icy_touch/horn_of_winter/pestilence/raise_dead
actions=flask,type=endless_rage
actions+=/food,type=dragonfin_filet
actions+=/presence,choose=blood
actions+=/snapshot_stats
actions+=/auto_attack
actions+=/raise_dead
actions+=/plague_strike,blood_plague<=2
actions+=/icy_touch,frost_fever<=2
actions+=/blood_strike,death=0
actions+=/scourge_strike
actions+=/summon_gargoyle,time<=20
actions+=/summon_gargoyle,bloodlust=1
actions+=/death_coil
actions+=/horn_of_winter
actions+=/empower_rune_weapon
ranged=sigil_of_virulence
# Gear Summary
gear_strength=1843
gear_agility=118
gear_stamina=1706
gear_intellect=20
gear_spirit=20
gear_attack_power=901
gear_expertise_rating=214
gear_armor_penetration_rating=94
gear_hit_rating=264
gear_crit_rating=1085
gear_haste_rating=342
gear_armor=14987
meta_gem=relentless_earthsiege
tier9_2pc_melee=1
tier9_4pc_melee=1
trinket1=deaths_choice
trinket2=deaths_verdict,id=47131
main_hand=dualblade_butcher,weapon=axe2h_3.60speed_811min_1217max,enchant=rune_of_the_fallen_crusader

dedmonwakeen · 12/02/09, 12:00 PM

Originally Posted by tripp6sic6

I edited the current Unholy profile to use the rotation that will be used in 3.3 and am getting the same variance he talked about. In three different runs at 10,000 iterations with no smooth RNG, here were the results.

2.0116 1.7981 1.0228 0.0827 1.6544 1.4802 2.0179 1.8565 5.7330
2.0116 1.5881 0.7054 0.5889 1.3504 1.7968 1.6783 1.7021 4.7066
1.6978 0.8282 0.7130 1.6894 1.3522 2.5918 1.3952 1.1694 3.7944

Is it possible for you to run the CLI? The GUI is running with Qt+MinGW and the CLI is using MSVC++.

Running on my development platform (Linux) using the profile above I get a nice tight result with minimal variance. I am unable to test it on Windows until much later tonight.

To use the CLI, take your profile and add the following to the top:
patch=3.3.0
calculate_scale_factors=1
iterations=10000

If you want to speed it up, you can just analyze a subset of the stats:
scale_only=str,agi

simc.exe your_profile output=output.txt html=output.html

Starfox · 12/02/09, 12:26 PM

Originally Posted by dedmonwakeen

Is it possible for you to run the CLI? The GUI is running with Qt+MinGW and the CLI is using MSVC++.

Running on my development platform (Linux) using the profile above I get a nice tight result with minimal variance. I am unable to test it on Windows until much later tonight.

To use the CLI, take your profile and add the following to the top:
patch=3.3.0
calculate_scale_factors=1
iterations=10000

If you want to speed it up, you can just analyze a subset of the stats:
scale_only=str,agi

simc.exe your_profile output=output.txt html=output.html

SimulationCraft for World of Warcraft release 3.2.2 ( iterations=999, max_time=300, optimal_raid=1, smooth_rng=0 )

>:C

tripp6sic6 · 12/02/09, 2:40 PM

There is much less variance between runs with the CLI, but there are still some stats that have major outliers. Is this just a statistical problem and require more iterations? Here is the data that I have.

Using the CLI in Windows with the same profile, but with the default 20k iterations.

1.9901 1.1468 0.7071 1.7490 0.6151 2.4641 1.7375 1.2576 4.8160
2.0010 1.1403 0.7076 1.7060 1.3537 2.4700 1.7374 1.2739 3.6265
1.6122 1.1358 0.7022 1.7828 1.3260 2.4531 1.7275 1.2589 3.6601

Using the CLI in Windows with the same profile, but with 10k iterations.

1.9965 1.1733 0.7177 1.7291 1.3698 2.4444 1.7197 1.2701 4.7621
1.9802 1.1676 0.7079 1.7825 1.3504 2.4781 1.3668 1.2622 4.7899
1.9946 1.1685 0.7012 1.7727 1.3584 2.4738 1.7532 1.2432 4.7134

dedmonwakeen · 12/02/09, 2:55 PM

Originally Posted by tripp6sic6

There is much less variance between runs with the CLI, but there are still some stats that have major outliers. Is this just a statistical problem and require more iterations? Here is the data that I have.

Using the CLI in Windows with the same profile, but with the default 20k iterations.

1.9901 1.1468 0.7071 1.7490 0.6151 2.4641 1.7375 1.2576 4.8160
2.0010 1.1403 0.7076 1.7060 1.3537 2.4700 1.7374 1.2739 3.6265
1.6122 1.1358 0.7022 1.7828 1.3260 2.4531 1.7275 1.2589 3.6601

Using the CLI in Windows with the same profile, but with 10k iterations.

1.9965 1.1733 0.7177 1.7291 1.3698 2.4444 1.7197 1.2701 4.7621
1.9802 1.1676 0.7079 1.7825 1.3504 2.4781 1.3668 1.2622 4.7899
1.9946 1.1685 0.7012 1.7727 1.3584 2.4738 1.7532 1.2432 4.7134

Since I am seeing variance "spikes" in your data as opposed to just "spreads" I am inclined to believe there is a bug rattling around in there. I'll take a peek over Starfox's shoulder......

Starfox · 12/02/09, 4:39 PM

Seems like we found the cause, uninitialized variable :O

tripp6sic6 · 12/02/09, 5:33 PM

I have found a bug with my 3.3 Unholy profile (above). It seems that Blood Strike and Plague Strike are being treated as if they are attacking from the front, but only those abilities. There is a roughly 7.7% parry rate on those abilities.