Retesting hit table assumptions

drumbum · 12/04/08, 12:21 AM

So far, I believe the rogue community has been working off the assumption that the chance to miss melee attacks against raid bosses has not changed since The Burning Crusade. However, after coming across some convincing evidence that this might not be the case (from here and here, specifically), I decided to test the dual wield miss chance to see if I could confirm those findings, and to see if it can be confirmed to also impact dual wielding autoattacks.

For my test, I used [Poniard]x2 because of its speed. My dagger weapon skill is maxed out at 400, and I did not equip any proc based effects of any sort. All attacks were made against the Heroic Training Dummy in Stormwind.

My stats were as follows:

2412 AP
258 hit rating (7.87% physical, 9.84% spell)
18.03% character sheet crit chance, 4.57% spell crit chance
8 expertise (2.00%)

I made a total of 11,500 autoattacks (all of them from behind) over the course of 3 sessions against the training dummy. The WWS reports are here:

Edit: WWS reports and log files are no longer available.

Observations:

Out of 11,500 attacks, 2201 (19.14%) of them missed. Since I did not have any points in Precision, and I had 258 hit rating (7.87%), this would estimate the white miss chance at 19.14% + 7.87% = 27.01%. (My statistics is a bit rusty. Could someone please calculate a confidence interval for this data?) Assuming that the correct value is 27%, this is a decrease by 1% from the Burning Crusade value.
Out of 11,500 attacks, 1557 (13.54%) of them crit. My character sheet physical crit chance was 18.03%, so this suggests that crit chance was reduced by 4.51% against a raid boss mob. I'm not sure what the previously assumed value for this was, but by dissecting Vulajin's spreadsheet, it looks like his spreadsheet assumes crit chance is only reduced by 0.6% against boss mobs. Is this a significant change since BC, or is there a tooltip error? Or did I make a mistake?
Out of 11,500 attacks, 512 (4.45%) of them were dodged. I had 2% worth of expertise, so this seems to be consistent with the previously found values of anywhere between 6.25% and 6.5% dodge chance.

In addition, during these tests, I had [Anesthetic Poison] applied to both weapons. I do not have any points in Improved Poisons. Some further observations based on this:

Out of 4449 poison procs, 328 (7.37%) of them missed. Since I did not have any points in Precision, and I had 258 hit rating (9.84% spell), this would estimate the poison miss chance at 7.37% + 9.84% = 17.21%. This appears to be consistent with the assumption that poisons behave like regular spells (at 17% miss rate).
Out of 4121 successful poison procs, 1864 (45.23%) of them were partially mitigated.

Can anyone else confirm the findings of this test? Are there any other mechanics that should be retested now?

Antiarc · 12/04/08, 1:07 AM

Interesting thread.

I did a bit of testing as well, but was pulled into a heroic CoT run before I could get a ton of data. Neverthless, here's the WWS:

WWS Loading...

My base stats:

182 hit + 5% precision

The character sheet reads as:
11.94% spell hit
10.55% melee hit vs level 80

571 crit rating
32.64% melee crit
12.44% spell crit

22 expertise

I was using a pair of the Librarian's Papercutters, again for the speed. I was dual wielding wound poison, as well.

I haven't done any analysis on it yet, and the amount of data is a bit scarce, but it's a data point. On a side note, one question I do have is why my spell crit is so much lower than my physical crit - I presume that's due to Malice and CQC not applying to the crit rate, but that still leaves 10% unaccounted for. What's the missing "duh" factor here?

Latito · 12/04/08, 1:17 AM

Originally Posted by Antiarc

On a side note, one question I do have is why my spell crit is so much lower than my physical crit - I presume that's due to Malice and CQC not applying to the crit rate, but that still leaves 10% unaccounted for. What's the missing "duh" factor here?

Agility only counts towards melee crit. Crit rating and talents apply to both.

Antiarc · 12/04/08, 1:38 AM

Originally Posted by Latito

Agility only counts towards melee crit. Crit rating and talents apply to both.

Aha. Thank you.

Antiarc · 12/04/08, 3:57 AM

Okay. Second test run, 10,001 autoattacks made, with Wound mainhand and Anesthetic offhand, just for giggles. The poisons did fall off for a few minutes (I was playing Play Auditorium and forgot to check it), but they aren't really the point of the test.

182 hit + 5% precision, 22 expertise, 32.63% physical crit, 12.44% spell crit.

WWS is here: http://wowwebstats.com/rv3g5uoomzpfk

17.2% miss - With my 10.55% hit, this puts the boss miss rate at 27.7%, which is rather close to the previously-known 28%.
0.9% dodge - With my expertise, I would expect 1% dodge; no change here.
27.37% crit - this would seem to corroborate your experience regarding crit; this is 4.93% lower than my character sheet crit rate, which is certainly surprising.

I will note that I am specced for both Malice and Close Quarters Combat, so I tested the possibility that one of those talents isn't working properly. Versus a normal level 80 dummy, after 700 attacks, I'm showing a 32.4% crit rate - within 0.2% of my character sheet crit rate. It's hovered there for the duration of that test, as short as it is, which would seem to indicate that Malice and CQC are working properly, but that bosses have a 5% melee crit penalty built in.

Mazz · 12/04/08, 5:23 AM

What was your missrate at the lvl 80 dummy? I am trying to exclude the possibility that they actually changed the single roll system... Drumbums data matches quite close with a dualroll system (I get an expected crit of 13.3% with dual roll) but your crit on the lvl 80 dummy contradicts that if your missrate was significant.

PessimiStick · 12/04/08, 11:10 AM

Originally Posted by Mazz

What was your missrate at the lvl 80 dummy? I am trying to exclude the possibility that they actually changed the single roll system... Drumbums data matches quite close with a dualroll system (I get an expected crit of 13.3% with dual roll) but your crit on the lvl 80 dummy contradicts that if your missrate was significant.

His expected miss would be 13.45% against the even-level dummy, which should drop his expected crit by about 4% if white attacks were in fact moved to two-roll. Either his 80-dummy was really lucky, or that isn't the case.

Edit: While I don't have any hard numbers for it (I wasn't logging at the time), I ran some decent-length tests a few days ago when playing around with the MP bug, and I was also seeing crit rates several percentage points under what I was expecting on the heroic dummy. In fact, I ended up moving to the level 80 dummy to see if Wound poison counted as another stack for the bugged MP (it did), simply because my observed values on the heroic dummy while naked were far enough "off" that I didn't trust them.

Asuah · 12/04/08, 11:32 AM

To preclude the possibility of a two-roll system operating here and fudging the numbers, the logical thing to do is to spam Shiv on a Heroic Training Dummy (as Shiv cannot be dodged or parried, the only two outcomes are hit and crit). I am currently in the process of doing so, and though the count is low (238 hits so far - these samples take some time to gather :x), my recounted crit is 24.0% when my character sheet chance is 32.71% (CQC spec'd - using dual [Poniard], will test with [Dalaran Knuckles]/[Left-Handed Brass Knuckles] next). I assume (for now) that the high 7.7% crit difference is because of random chance due to the small sample size, but it is still reason for more investigation.

For comparison purposes, a quick 100 yellow hits on the level 80 dummy showed a 30.8% crit, which is closer to my 32.71% character sheet crit.

nelalas · 12/04/08, 1:35 PM

Originally Posted by drumbum

258 hit rating (7.87% physical, 9.84% spell)
8 expertise (2.00%)

Out of 11,500 attacks, 2201 (19.14%) of them missed. Since I did not have any points in Precision, and I had 258 hit rating (7.87%), this would estimate the white miss chance at 19.14% + 7.87% = 27.01%. (My statistics is a bit rusty. Could someone please calculate a confidence interval for this data?) Assuming that the correct value is 27%, this is a decrease by 1% from the Burning Crusade value.
Out of 11,500 attacks, 512 (4.45%) of them were dodged. I had 2% worth of expertise, so this seems to be consistent with the previously found values of anywhere between 6.25% and 6.5% dodge chance.
Out of 4449 poison procs, 328 (7.37%) of them missed. Since I did not have any points in Precision, and I had 258 hit rating (9.84% spell), this would estimate the poison miss chance at 7.37% + 9.84% = 17.21%. This appears to be consistent with the assumption that poisons behave like regular spells (at 17% miss rate).

Some proportional z-tests of your data:

$\alpha=0.05$
$Z_{critical,\alpha/2}=1.96$

1. Miss Chance

$n=11500$
$p_\mu=0.28$
$q_\mu=1-p_\mu=1-.28=0.72$
$p_sample=0.2701$

$s_p = \sqrt{\frac{p_\mu q_\mu}{n}} = \sqrt{\frac{.28(.72)}{11500}} = 0.004187$
$Z=\frac{p_s-p_\mu}{s_p}=\frac{.2701-.28}{.004187}=-2.36$
$|Z|>1.96$

2. Dodge Chance

$n=11500$
$p_\mu=0.065$
$q_\mu=1-p_\mu=1-.065=0.935$
$p_sample=0.0645$

$s_p = \sqrt{\frac{p_\mu q_\mu}{n}} = \sqrt{\frac{.065(.935)}{11500}} = 0.0645$
$Z=\frac{p_s-p_\mu}{s_p}=\frac{.0645-.065}{.0645}=-0.22$
$|Z|<1.96$

$p_\mu=0.0625$
$q_\mu=1-p_\mu=1-.0625=0.975$

$s_p = \sqrt{\frac{p_\mu q_\mu}{n}} = \sqrt{\frac{.0625(.975)}{11500}} = 0.002257$
$Z=\frac{p_s-p_\mu}{s_p}=\frac{.0645-.0625}{.002257}=0.89$
$|Z|<1.96$

3. Poison Miss Chance

$n=4449$
$p_\mu=0.17$
$q_\mu=1-p_\mu=1-.17=0.83$
$p_sample=0.1721$

$s_p = \sqrt{\frac{p_\mu q_\mu}{n}} = \sqrt{\frac{.17(.83)}{4449}} = 0.0372895$
$Z=\frac{p_s-p_\mu}{s_p}=\frac{.1721-.17}{.000032}=0.37$
$|Z|<1.96$

Your data show sufficient evidence to conclude that the miss chance observed at the target dummy is different than 28%, but that the chance to be dodge is not different than either 6.25 and 6.50%, and that the observed poison miss chance is not different than 17%. Keep in mind that this is one sample derived from a target dummy.

Leto · 12/04/08, 1:52 PM

It would be understandable that the dual-wield miss rate was lowered by 1% in wrath, as others are observing the 1h miss rate was lowered by 1%. It wouldn't make sense for them to lower one but not the other, balance wise.

Rambaral · 12/04/08, 2:06 PM

So this means what, we need to adjust the soft caps on hit/crit/exp?

Leto · 12/04/08, 2:12 PM

Originally Posted by Rambaral

So this means what, we need to adjust the soft caps on hit/crit/exp?

Adjust melee hit caps, indicate the reduced chance to crit bosses, and leave exp assumptions the way they are.

drumbum · 12/04/08, 2:20 PM

Originally Posted by nelalas

Your data show sufficient evidence to conclude that the miss chance observed at the target dummy is different than 28%, that the chance to be dodge is neither 6.25 and 6.50% but somewhere in between, and that the poison miss chance was different than 17%. Keep in mind that this is one sample derived from a target dummy.

Thanks for running the numbers, but I think you forgot to take the square root while calculating the $s_p$ values for dodge and poison miss chance.

Also, forgive me if I'm wrong here, but wouldn't it be more accurate to compare the observed and expected values without the hit rating added in? For example, in the case of physical miss chance, the expected value was actually 20.13% and observed value was 19.14%, not 28% and 27%. Again, this is just my understanding, so correct me if I'm wrong, but I believe this is why you can get a better confidence interval if you are closer to the hit cap when running the tests.

Originally Posted by Leto

It would be understandable that the dual-wield miss rate was lowered by 1% in wrath, as others are observing the 1h miss rate was lowered by 1%. It wouldn't make sense for them to lower one but not the other, balance wise.

Agreed. To be honest, I was fully expecting to find that the miss chance would be 27% now after reading about the 1H tests, so I'm not surprised by my results (although I still think more testing would be good to verify this, especially since Antiarc's data came up with 27.7% miss chance).

What was more surprising to me was the large change in crit chance. This makes me wonder if there's anything else we could could be missing that was changed in WOTLK.

Roywyn · 12/04/08, 3:07 PM

Originally Posted by drumbum

What was more surprising to me was the large change in crit chance. This makes me wonder if there's anything else we could could be missing that was changed in WOTLK.

For spells, it was assumed that there is a crit depression of 3% vs. boss mobs.

If you raided with a Ret paladin in 2.4 (+3% spell and melee crit), you'd get poison crit vs. trash mobs but never vs. bosses.
That common explanation was that your crit chance vs. bosses was lowered by 3%, which eats up the +3% from a ret paladin.
There also was one longer test of spamming arcane missiles vs. a mob 3 levels above you which showed that spell crits are on a 2-roll system, and that there is a ~3% crit depression vs. mobs 3 levels above you.
I think those 3% have only been eyeballed, without an exact statistical analysis.

Drumbum had 4121 poison hits+crit with 115 crits, for a 2.79% poison crit chance.
That's 1.78% below the listed spell crit chance of 4.57%

Adrine had 88 crits, 946 hits+crits, for 9.30% spell crit. 3.12% below the listed chance of 12.44%.

The natural test would now be to reduce crit rating on gear to ~3.5%, check if you still get crits, and see when you stop getting them.

The same procedure would work for melee crit, but requires a respec (no malice and no other crit talents).

Hit caps

A natural test would be to equip 3% in hit rating (assuming everyone has 5% precision).
That would cover specials and whites while not dual-wielding if he cap is 8%, or give 1% miss chance otherwise.

Then the DW miss cap is either 8%/9%+19%, or they changed the 19% penalty as well. Sounds like headache.

Antiarc · 12/04/08, 6:41 PM

Okay, been doing some more testing.

I forgot to get a combat log for WWS purposes, but I have Recount running.

With 2.52% hit and no offhand, I was able to produce a miss with both Sinister Strike and white swings in about 300 white swings. This was accidental, since I was looking at my spell hit (which was just over 3%), but I included it anyhow. It demonstrates that the special and white hit cap is more than 7.52%

99 hit rating + precision = 8.02% hit
334 crit rating for 26.57% crit
30 expertise (7.5%, quite capped)

Out of 1000 melee swings:
560 (56%) were hits
221 (22.1%) were glances
219 (21.9%) were crits

In that time, I produced 241 Sinister Strikes:
178 (73.9%) were hits
63 (26.1%) were crits

I've had no misses for the duration of the test, and would have expected about 10 white swing misses so far, if the cap was indeed 9%. The sample size is too small to be statistically conclusive, but it's looking pretty good that the special and single-wield hit cap is 8%.

Edit: Continuing the test...

2004 melee swings

1079 (53.8%) were hits
467 (23.3%) were glances
458 (22.9%) were crits

480 Sinister Strikes

353 (73.5%) were hits
127 (26.5%) were crits

Still no misses observed. At this point I'd have expected about 25 misses, so I'm pretty confident in saying that the new hit cap is 8%.

This would mean that you'd want 262 hit without precision, or a measly 98 hit with precision. After that, its value significantly diminishes, though you will still be under the poison hit cap.

Further edits: I decided to test if the spell hit cap has been lowered, as well. I'm dual wielding Wound Poison, with 290 hit + precision, which is 11.06% hit from gear, and 5% hit from precision, resulting in a total of 16.05% hit. Within about 100 Wounding hits, I did observe 2 misses, which would seem to indicate that the spell hit cap has not been lowered accordingly.