Soul

So, I followed Zaldinar's advice and got myself hit-capped through a combination of gear swaps, food and elixirs. I also respecced to drop Arcane Instability to see the effect of a lower crit rate on the crit suppression. As a result, my crit rate went down to 8.48% on the paper doll.

The results were interesting.

So, obviously, the crit suppression is still in effect at this lower crit rate, even while hit capped. However, the magnitude of the crit suppression seems smaller. The observed crit rate is 7.07%, a drop of significantly less than the 2% drops observed in the two previous tests.

At this point, I'd like to propose a set of hypotheses in order of increasing strength and difficulty of testing:

1. Crit rates are reduced from those on the paper doll versus boss level targets.
2. A naked character that has no flat crit rate modifers, either from talents or buffs/debuffs will not experience any significant crit suppression.
3. A naked character will not experience significant crit suppression, regardless of whether or not it has flat crit modifiers arising from talents, buffs or debuffs. In other words, crit suppression only applies to crit from stats.
4. Crit suppression depends on the crit rate listed on your paper doll in some fashion that is easy to express.

Does anyone think this is a worthy progression of hypotheses to investigate?

Your tests show that is is 98.4% likely that you are showing crit suppression, but you may want to run a few more trials because with a crit rate so close to 50%, the variance is very large. In general, the randomness (and hence variation) of this sort of distribution peaks when when the crit chance is at 50%. Trials tend to converge faster for crit values that are further away from 50% in either direction.

It is looking for now that there is no significant difference in crit reduction between our AM tests at 16% crit and your scorch test at 50% right now, but it would be good if you collected more samples.

 

galzohar

You need 2,500 casts to get 1% deviation with 50% crit (and not a lot less with different crit rates). To get 0.1% deviation you need 250,000 casts.

EDIT: Corrected the number of casts needed. That's what you get for calculating quickly. But the idea is the same.

 

Soul

Actually, it's 2,500 and 250,000 respectively, *cough*.

But yeah, I eliminated a large part of that post that had detailed math as to how much casting you'd need to do to get a decent amount of accuracy in a crit estimate because it came out very cranky sounding.

 

diag

If a level 60 mage wearing something like [Naglering] (or molten armor) and some hp5 gear duels with a level 63 bm hunter, one can collect 1800 data point per hour.

 

Jonny_Monroe

Is there any evidence that would imply crit suppression is active in PvP?

OMNOMNOM.

 

Zaldinar

Not that we've seen yet, but it could be an easy way to show it. The other concern is whether or not your level + 3 == Boss, which unfortunately is not necessarily an assumption that we can make in this case. Would be interesting to see though. Anyone have a level 77 caster?


If I were to approach this entire scenario in an exhaustive testing binge, I'd be mixing differing levels of hit and crit gear and talents against the boss dummy and level 80 target dummy to see what I could get from it.

This is what I got against a level 80 yesterday:

and the beginings of a naked run on the boss dummy:

Obviously need way more datapoints on the naked trial. Something else we should keep in mind when considering this, I bring up the question of hit rate for two reasons, it eliminates the question of one roll or two roll by effectively isolating one of the rolls (if its maxed), and if we can come up with an effective model for showing the reduction against an 83 target, we can backcheck it against a level 80 target, 70, 60, and 1, to see if we can reapply it to explain crit inflation.

Something Muphrid had brought up at some point (I think when we were trying to explain crit inflation), what if your crit rating was being calculated when spells are cast against a level 83 target as if you were level 83? As in, using the Rating to percent conversion for level 83. Exclude crit from talents, int, and base, and recalculate it based on that? If anyone is familiar with how the rating conversion formulas extend into the post 80 realm it would be interesting to see those predicted formulae.

To truly model the game, we first must research it.
http://forums.worldofwarcraft.com/th...icId=109841969
Proven TheoryCrafting Stuff, chain casting in a PTR near you soon.

 

Shaewyn

I'm not 100% up on the maths we're looking at here, but would it be useful for a couple people to help provide data sets?

Say, shooting at a target dummy naked and untalented (and with various levels of gear/talents) for a while and then posting the resulting numbers along with screenshots for verification? Would that be useful for investigative purposes?

 

TigaFin

That is an interesting theory. It would probably also increase the number of crits on lower level targets and I think it feels like we crit more on lower level targets. So test on level 60 dummies next and see if the number of crits is higher than indicated on character panel?

 

Zaldinar

Here are some test conditions that I'd be interested in seeing the result of:

Naked Mage VS 80 target dummy, no spec.
Hit capped mage VS 80 target dummy, no spec (hit capping without talents).
Hit capped mage Vs 80 target dummy with talents.

Naked Mage VS 83 target dummy, no spec.
Hit capped mage VS 83 target dummy, no spec (hit capping without talents).
Hit capped mage Vs 83 target dummy with talents.

All tests would need around 2500+ points to be convincing, preferably a /combatlog (If hosting files is an issue I can set up an upload center on my bounceme domain), but a recount screenshot should be fine, along with either a screen shot of tooltips/spec or just text identifying them for each test.

I say a /combatlog, by the way, because not that I don't trust people to do the math correctly, or don't trust Recount, or any of that... Having a paper trail for testing has done me no end of good in recent times, being able to reuse logs to verify other things, so if you provide enough data about what went into the test, then the combat log itself, the data can potentially be reused elsewhere.


Not necessarily, we know they get inflated, that's really easy to show in Theramore with the AoE dummies. It was around 7% more damage on the level 1-3 Troggs and Wolves that I observed the inflation on doing Ice Lance and crit gain tests ( World of Warcraft - English (NA) Forums -> So I chaincast Ice Lance on the PTR today... ). The problem is if we use level 1 conversions for the crit rating I should have seen way way more than 7% gain, which is why I tended to think it was hit influenced vice rating influenced. But all the avenues are worthy of exploration.

To truly model the game, we first must research it.
http://forums.worldofwarcraft.com/th...icId=109841969
Proven TheoryCrafting Stuff, chain casting in a PTR near you soon.

 

Setia

I cast 3586 Arcane Missiles at the level 70 training dummy. Out of these, 695 crit (19,38%, while 615 were expected (paper doll crit with Mage Armor is 17,15%, as Arc/Fire), so for 10 levels under, there's a 99,7% chance there is a positive crit difference. In my case, it was around 2,2%. In case crit rating and other sources of crit behave differently relative to mob level, I had 496 crit rating (10,80% crit) and 906 int (6,34% crit) - AI was off.

Perhaps with enough data points, both below and over 80, we could find the formula for crit% depending on mob level relative to player level to be similar both over and under the player's level (assuming bosses simply behave as level 83). However, it would not be linear.

Recount screenshot

 

Soul

I was wondering if it would be better to settle the whole "one-roll, two-roll or something else" business once and for all before we do any further crit testing.

Here is how I propose to do it:

Trial 1: Naked, untalented mage versus level 80 target dummy.
Trial 2: Nearly naked, untalented mage buffed to be hit capped with food + elixir+hit rating trinket versus level 80 target dummy.
Trial 3: Naked mage hit-capped through talents versus level 80 target dummy (this is trivial to achieve for a level 80 mage versus an equal level target).

You'd be at the naked crit rate for all your trials (about 2%), so in order to get a deviation of 0.1% in your measured crit rating, you'd need at least 19,600 (landed) casts. To get a 95% confidence interval of plus or minus 0.1% in your crit rating, you'd need rather more casts: at least 76,000.

After getting decently accurate values for crit rating on all three tests, we can test the following hypotheses:

1. The difference in crit rating between Trials 2 and 3 is zero.

If this is true, then we can treat hit from hit rating as equivalent to flat hit modifiers. Otherwise, we can't.

2. The difference in crit rating between Trials 1 and 2/3 is zero assuming one-roll/two-roll/whacky roll systems.

This allows us to figure out how exactly crit and hit interact: whether it's a table like melee, two independent rolls or some other whacky system.

This is exceedingly tedious, but it's the only real way to completely account for the interaction of hit with crit once and for all.

We could do the same thing with a boss-level dummy, but hit-capping on the heroic dummy with gear and food will almost certainly increase the crit chance on your paper doll with no real clue of what it actually should be and then, you end up having to do (many) more trials to get a decent measurement of your crit rating due to the effect of crit chance on randomness.

As an aside, it took me a little under two hours to do the test in post 1001. So, this is pretty hard core and I'm really looking for a way to do this rigourously without having to spend 8 or 32(!) hours getting a decent dataset (I'd like to actually, like, play the game in my free time). Has anyone got any suggestions?

EDIT: So I started on the level 80 dummy runs. Naked with no buffs except mage armor and mp5 food. Talents taken were Arcane Stability, Arcane Subtlety and Clearcasting because... it's shocking how tedious having a lack of Clearcasting is. The results were as follows:



One quarter of the way through. I think I'm going to quaff an mp5 flask as well for the remainder of this test. Totals so far:

I did a test where I pounded on the level 70 dummies with Blizzards (they're clumped up in fours in Orgrimmar). After about 500 Blizzard strikes, I realized that I hadn't missed a single one, which seems to mean that the minimum miss chance without +hit talents is now zero.

 

Jonny_Monroe

Hmm... I have a level 76 priest I've been camping at dragonshrines for cloth transmutes. He has a minimal amount of crit rating, I could take him to some target dummies and smite spam a level 80 dummy. If the crit rating conversion is calculating at the target's level it should be evident after a decent sample is taken. I'll spam some smites later today and see what kind of sample I can get.

OMNOMNOM.

 

semanteme

I bought Rides a Dread Legion and figured I'd spam Arcane Missile on a lvl 80 dummy while reading it. I'm not sure how useful these data actually will be though, because my mage was talented. So clearcasting proc'ed and gave the potency buff, but it looks to me like I'm critting more than I should be.

 

Saratoga

@semanteme
As you said before, your mage was talented. What means: if clearcasting procs, more crit possibilties.
10% chance that it procs, 30% to get a crit with clearcastings, what ups about 3% the chance to have a crit.
Adding your crit score those 3% and you are quite close to what is on your recount =)

 

Lhivera

Zaldinar, don't I remember you doing some research on whether crit rate is affected by caster/target relative level some time back? Focusing, IIRC, on crit rates vs. much lower-level targets?

At Veridian Dynamics, we can even make radishes so spicy, people can't eat them. But we're not, because people can't eat them.

 

Shaewyn

3 Tests completed:

Test 1: level 80 target (FLAWED)
- talented (57/3/11), hit capped through talents and gear
- 500 casts
- Hitcapped, with talents and gear (286 hit rating)
- Buffs of Arcane Intellect, Molten Armour
- Glyph of Arcane Missiles, Arcane Blast and Molten Armour

WARNING: Clearcasting and arcane potency may invalidate results.

Test 2: naked mage, talentless, level 80 target
2000 arcane missile casts
39 crits

tooltip crit rate: 2%
Observed Crit Rate: 2%

Notes:
- naked
- talentless
- no buffs
- Glyph of Arcane Missiles, Arcane Blast and Molten Armour
- Dranei, so heroic presence is in effect.

Test 3: Naked, talentless mage vs boss level target
2500 arcane missile casts
27 crits

tooltip crit rate: 2%
Observed Crit Rate: 1.1%

Notes:
- naked
- talentless
- no buffs
- Glyph of Arcane Missiles, Arcane Blast and Molten Armour
- Dranei, so heroic presence is in effect.


Combatlogs and screenshots are available here
(If the downloads are exceeded, let me know and I'll re-upload it.)

 

solbergb

I have to question the methodology of testing with extremely low base crit rates (2%). Your population is so low in crits that even a very small amount of "streaky" behavior will taint the results and you need to measure very fine differences in sample sizes for the measurement to have any meaning. A 2% crit rate is 50 crits in 2500 trials. A 50% crit rate is 1250 crits in 2500 trials. A variance of even one from the expected mean of 50 crits will show a 2% shift from the mean.

A 16% crit rate is 8 times less sensitive to a single crit high or low, a 50% crit rate is 25 times less sensitive.

I'm not saying that there isn't something real going on here. I'm saying that you will have an easier time measuring it with crit rates closer to 50% than to 2%.

 

Soul

We've sort of noted this isn't the case. As an example, we can consider Setia's data:

496 crit rating at level 70 translates into 22.5% crit at level 70. With INT and base crit, Setia would do at least 27% crit if the stats and rating were done at the target's level.

On a side note, I did a quick test on level 70 target dummies, naked with no buffs save Mage Armor over a coffee break. Here are the results:



Note that there were no misses. The crit rate is significantly higher than 1.98% as well (95% confidence), but that's sort of expected.

We've more or less verified that crit depression exists. The probability of obtaining the results we have in this thread without there being any crit depression is miniscule. We are now trying to figure out a way to characterize this crit depression.

We've more or less been going on the assumption from measurements that crit depression is on the order of 2%. It's a lot easier to distinguish small levels of crit depression when your crit rate is itself small. Distinguishing between a 4% crit rate and a 2% crit rate is easier than distinguishing between a 47% crit rate and a 49% crit rate. It also takes less time to get a certain amount of accuracy when your crit rate is small because your variability is smaller too (although 'less time' is really a relative thing).

 

galzohar

Actually, if your only problem was 2% from the mean, you'd have 2% of 2% = 0.04% crit error, which is very good. Of course randomness will make much bigger deviations than this.

Testing at very low crit rates means that your error in crit count is equal to the square root of the number of crits. So if you cast enough to average 1000 crits, expect a deviation of 100 (so if you were doing it with 1% crit you casted 100,000 spells and can expect a 0.1% deviation from the expected crit rate).

It doesn't make a whole lot of a difference what crit rate you're testing on when it comes to how many casts are required, you'll always need hundreds of thousands of casts to get a good result.

 

Saruk

As Soul said in the post above, it's easier to distinguish between smaller (or higher) crit rates than it is so distinguish between crit rates approaching 50%. This is because of the way random sampling works. It also means you don't need "hundreds of thousands of casts" to prove anything. Sampling confidence intervals have severe diminishing returns on 'confidence' the more sample you add. It's why political polsters typically stop at 1000 or 1500 respondents to predict what 100milion voters will do - you don't get much more accurate over and above that for the time and money it costs to poll people.

Assuming an infinite population, an observed 2% crit rate on 2,500 casts has a 99% confidence interval of 0.72% (we are 99% sure the 'real' crit rate lies +/-0.72% above or below the observed of 2%).

Assuming an infinite population, an observed 50% crit rate on 2,500 casts has a 99% confidence interval of 2.58% (we are 99% sure the 'real' crit rate lies +/- 2.58% above or below the observed of 50%)

 

Shaewyn

Ok, ran another test:

Test 4: Hitcapped with talents and gear.

2501 arcane missile casts
513 crits

tooltip crit rate: 23.07%
Observed Crit Rate: 20.5%

Notes:
- geared, hitcapped
- Talented (57/3/11) MINUS ARCANE POTENCY
- Buffs of Arcane Intellect, Molten Armour
- Glyph of Arcane Missiles, Arcane Blast and Molten Armour
- Dranei, so heroic presence is in effect.

NOTE: I avoided Arcane Potency so that the crit bonus of clearcasting was avoided.

Zip file with combat logs and screenshots is here.

 

galzohar

Deviation of crit rate assuming it's the expected crit rate: 0.84%

Do like 2-3X more casts and assuming you don't have any other factor messing it up, you might have something. At least this shows that it's rather reasonable you're losing crit.

 

Soul

Hmm... this might be interesting. The drop measured so far is 2.5% at 23% paper doll crit. At 16% crit, Zaldinar saw a 2% drop in crit. At 8.48% paper doll crit, I saw a 1.4% drop in crit.

I wonder if I could trouble you to do maybe another 10000-15000 or so casts with the same setup (you don't need to do them all at once). Then do another 12500 casts but without Arcane Instability and with Mage Armor instead of Molten Armor (an 8% difference in crit).My point here is to see whether the drop in crit is significantly different between the two scenarios and isolate whether a change in crit from flat modifiers results in a change in the amount of crit suppression we see.

 

Zerstorung

Are you saying that perhaps, crit reduction is based on a percent? So the more crit you have, the greater the expected loss of crit is? Though, I doubt the programming/formula is as simple and straight forward as a flat % (and it's proven it's not if you take values from the above results).

Why do you think there is crit suppression in the first place? Intended, or some left over old school programming (like how some classic mobs still have vulnerability bonuses).

 

Soul

Crit reduction is obviously not flat because people have noticed crit reductions on the order of 2% while geared, but naked characters, with ~2% base crit, can still crit. In fact, Manly claims that a naked character with 2% base crit achieves exactly 2% crit versus a boss-level mob in post 980 on this thread.. So, yes, I'm thinking that the higher your paper doll crit chance, the greater the drop in crit will be. The question is how is it applied?

As for why there is crit suppression in the first place, I haven't the foggiest. All I know is that we did the Dr. Boom tests to try and determine a one or two-roll system back in September of 2007 and incidentally discovered crit suppression with spells.