Miles from World of Logs - Real Time Raid Analysis was kind enough to implement an overview of each spec's top DPS/healing, per boss. The numbers represent the median of the top 20 effective DPS/effective HPS numbers from players of each spec, per fight.

Below is his data presented in a few crude tables for ICC 25 normal. The last column compares each spec's average DPS with the best DPS from that table.











Issues:
- The highest DPS specs are a bit of a self-fulfilling prophecy: the "best" players (meaning the minmaxers) will migrate toward that spec and they will push the numbers higher because (a) they're the minmaxers and (b) because there will simply be more of a selection base for that spec.

- In the last column I should probably average rank accross all ICC fights, rather than DPS since numbers vary so greatly between say Blood Queen and Valthiria.

- The tables will become more interesting in the following weeks when I plan to add week-on-week trends to the graph.

- I am not an expert in statistics! I am not even an initiate. So please present any scientific objections to this "analysis" in a gentle and easy to understand way

This some very cool information. Great work putting it together.

I'm curious about why you picked a median of the top 20. If you're doing this by hand then yea, 20's a lot. But if you're gathering this automatically then maybe taking more samples would be a helpful comparison.


Or you might argue that 20 is too many. I'm not a statistics guy, but when I pull up the top dps on Festergut World of Logs - Real Time Raid Analysis and I just look at the top 25 dps, it consists of just 5 feral druids, 7 mages, and 8 rogues. However, in the list above feral druids are listed at #7 in the dps list for Festergut. So I would presume the next 15 feral druids performed so poorly that hunters, fury warriors and ret pallies blew past them in the averages.

The values are taken (automatically, thanks Miles) from top dps lists per spec, for example World of Logs - Real Time Raid Analysis . In my table you see the median of the top20 values from that list.

Why 20? Historically I did the first graphs on WoW Meters Online - Combatlog Replay data, and that site has only 20 results in its tops. WoL has 200, but in order to answer to "what's about the best dps this spec does currently?" averaging top 200 seems too much. Taking the top value also seems too fluky. So there's my scientific answer

World of Logs separates Cats and Bears, and some druids will tank the first portion of Festergut in cat dps gear/spec, and then switch to cat after the tank swap, and get a salvation, and pull amazing damage with the 90% damage increase, but WOL will regard them as bears. Only the last graph will have these ferals numbers in them, since they are recorded as tanks, the 1st and 3rd graph pull the numbers listed as cats.

Pashaman, have the WoL folks publicly released the raw data? I'm sure I'm not alone in saying that I would love to play around with the full top 200 dataset in a google docs spreadsheet to generate graphs, etc. CSV format would be ideal.

Oh, this topic again.

So, let's go over how the entire concept of this is wrong-- specifically, this isn't measuring what you think it's measuring.

- Sample bias. Say there's 2 rogues in every raid and 1 feral in every raid. Half of those ferals are tanks, half are cats. The 20th druid, then, is closer to the average, because there's 4x as many rogues as cats. So, you'll want to count the 20th cat against the 80th rogue. (In other words-- percentile comparisons, not absolute rank comparisons.) How many feral druids have DPSed on a lich king kill?

- Gear bias. What percentage of loot council guilds give preference on gear to the highest DPSer? If a feral gets defaulted a DPS staff but rogues have to compete, what does that mean?

- Buff bias. Gaps aren't as large as they first appear. People more likely to get TotT/Hysteria casts, for instance, will be skewed higher than their "raw" DPS. Additional damage via TotT/Hysteria also don't get allocated to their casters. Inevitably, the buff receivers are either high performers or receive the buff from social status (from being officers, being friends with the buffer, being the rumored best class for the buff, etc).

- RNG bias. Classes that are more likely to have RNG impact their final DPS will tend to have higher maximum numbers but lower average numbers. Certain best-stats also skew RNG-- ferals can get higher maximum DPS by gearing for agility (more crits), but higher average DPS by gearing for ArP.

This is just the beginning of what to take into account. Which is why Blizzard dismisses such statistics so easily-- are they actually measuring anything worthwhile? Instead of the graphs above, give me percentile stats (99th/90th/80th/50th), accomodate buff imbalance, use average performances by player X instead of maximum performances to account somewhat for RNG, and so on.

These numbers don't necessarily say anything about class/spec balance.

The chart is exactly like he describes. And to say its useless i think is a bit unfair. From a game designer point of view i would assume it would be very useful. At the very least it is a snapshot of how the game is played, which in my opinion is much more useful then how it could be played (sims). This information could also be used to show trends. Things like the weather, economy and stock market are all based off trends. This would allow the designers to foresee problems before they occur, and hopefully adjust accordingly. It also does show class balance, if only for the 440ish participants involved, but it does. How and what to get out of the information is the difficult part. I would assume though there are many more uses for this chart then the few i listed.

Partially, read this thread: World of Logs • View topic - Suggestion: Class Rankings . It's not the full top 200, it's just what I'm showing in the tables (median of top 20).

Or maybe I don't think it's measuring what you think I think it's measuring

I started these tables for my guild in 3.2 when we didn't have as much raid DPS as we wanted. The question was, who's not performing? What DPS output should we expect from everyone? Clearly it's not as simple as "your bottom 5 DPSers are bad" because each spec potential DPS is different. But what is each spec's potential?

There are sims of course, but I felt that having a snapshot of what's out there in the real raids is a good anchor in reality (thanks Batar, you made that point very well). Are these numbers an accurate/absolute measurement of spec potential? Probably not. Are they a good indication? Probably yes.

That data would be great to have, but I don't have it atm.

Without knowing anything about statistics, don't these things "even out" overall? Aren't cats as likely to be friends with the officers/buffers as rogues are?

They don't say everything, granted, but say nothing? that seems too much.

In the thread I linked, Miles talks about working on it yes.

I'm not a statistic master so perhaps some people may be able to describe it with more details but I'm going to give a brief explaination of why the median of "top" 20 is a totaly biased estimation.

You can describe the total damage done by 1 player(class) on one given encounter as a single sample (measure) of the quantity you are trying to estimate.

The quantity beeing a Random variable it follows a Probability distribution.

When you want to compare dps among different classes you want to compare their Probability distribution. The mean and variance are parameters that describe the probability distribution (but are enough only for Gaussian/Normal distribution), which usually fit many random events in the world thanks to the Central limit theorem.

But in our case the normal distribution isn't a good model, first because normal distribution has an infinite support, (can take negative value regardless of the mean). And it's definitivly a non symmetric distribution as everyone aim for the "best dps".
More complex distributions, as it's the case for us, usually require more parameters than mean and variance to be described. And most people understand that the mean isn't what interest us here (but can be interesting for Blizzard who care about the whole playerbase).
Modeling the right distribution to fit the scenario can be a really interesting problem, but would require of course a lot of data.

Given the measurements avaiable (lets say top 200 damage by spec), you cannot estimate distribution parameters, as some people pointed top 200 damage done by 1 spec can be the whole data for one class (including the worst players), but the 1%top for an other class.

You may eventually be able to make an estimation based on top N of your samples only if you artifically reduce all your base sample to the same size.
Let's say you have 2000 samples for Arcane Mage, but 200 for feral druids, you pick 200 samples out of your 2000 Arcane Mage samples (using a uniform random distribution). Then take the top 20 out of your 200 samples for both classes. That would be a non biased estimation of the top 10%, but once again that wouldn't describe the whole distribution.

If you don't want to discard your 2000 Arcane Mage sample, you simply have to take a fixed % top. You will average the dps of the top 10% mage (200) and the top 10% of the druids (20), your estimation for the druids would of course be much more noisy in this case.

If you want to make a robust estimation of the distribution you need the whole data set (not the top N/%), having less sample for one class/spec isn't that bad in this case as you can estimate how robust your estimation was given the number of sample.

Overall the top N/% is probably extremly sensitive to the variance of the distribution.

A bit more details on the bias estimation of your measurements,

Considering you are measuring the average of the top 10% of samples. Which corresponds to the Cumulative distribution function beeing above or equal 0.9 (1-10%).

Assuming normal distribution (as I said it's a bad model but can give you an idea) :



The erf function is a bit complicated so I just gave a numerical solution you can check on wikipedia if you want more details.

What does this mean?

Class X with 9000 average dps and 1000 dps sigma, your measure will show 10281 dps.
Class Y with 9000 average dps and 100 dps sigma, your measure will show 9102 dps.

As everyone could "feel" that measurement is biased and highly favor variance, just tried to express by how much

I see what you're saying about variance, but I'll just note that the OP actually said


Calculating the median is not the same as calculating the average. (Though I don't know what calculation was used in generating the charts.)

The median of top 200 will still be greatly affected by the variance of dps. However ncreasing the sample to at least top 100 would help without losing the information about what "best" dpsers of each spec are capable of. Also you could actually calculate the variance of top 200 for each spec, which would give you a better picture of the possible size of the bias.

Used as a reference for classes, you're still not getting real reference data. Again, RNG bias. Fight length bias (you get different results from a 3:30 fight versus a 5:45 length fight, and depends on individual cooldowns). The correlation between faster kill times and higher DPS given (caused by heroism/lust and effective cooldown usage) means that your number might not match. Either you're in a guild competitive with the best guilds in the game, or you won't see similar numbers. And these numbers say nothing of consistency-- player X might reach 12000 DPS one week and 9000 the next. Is he playing optimally in both cases?

The term "median of the top 20" simply means halfway between 10th and 11th place. Why is that number a better estimate of DPS than #1? Than #100? Than #1000?

The point is, you need to do a lot of extrapolation and simply unscientific analysis to get any conclusions from the data presented. This is the kind of data that is responded to by blue posts saying "we have better data, we know more than you, we don't think there's evidence to merit a change." From a game designer perspective, it's not a good summary. It's only part of picture, and a much smaller glance than you'd think.

Rogues are more likely to talk to other rogues about TotT strategy. So, they're more likely to give them to rogues. Rogues are more likely to be established DPS in a 5+ year old game than a feral in a long-standing guild because ferals haven't always been viable DPS. That gets mitigated the longer time goes on, but history doesn't disappear.

So in the end you're claiming these numbers as representative of what you should expect to see out of an elite player; but your definition of elite poor statistically speaking; they aren't representative of the individual; they aren't expected. Again, you can't draw consistent conclusions across classes. You can't draw consistent conclusions of gear. You have no indication of what those top 20 players are doing to get scores that high.

In other words, these numbers clustered together don't give you much of an indication of anything except that, for good or ill, some players got that number. Maybe they got lucky and rode threat so tightly that another crit would've pulled aggro and wiped the raid. Maybe they blew cooldowns on Blood Queen to get the first bite, giving them another minute of buffed damage (instead of letting it go to someone who'd get more benefit, but didn't use cooldowns). Maybe they're AOEing on adds instead of focus-targeting a more dangerous enemy. Maybe all of the top 20 rogues got to stay on Deathwhisper during the mana shield, but only the top 10 ferals did.

You're right in that these mean more than nothing, and I didn't say that, exactly. But in a table that went so far as to rank every single spec (and PER FIGHT rankings as well), can you really draw firm conclusions other than basic things like "locks are weak for DPS" or "good players play spec X"?

Also, some classes (like balance druids, feral druids, locks) make liberal use of DOTs, and some of the fights (ok, most of them in ICC) have periods where the DPS is unable to attack. The arcane mage dps remains high when they stop attacking. The DOT classes have their DOTs ticking, which is increasing total damage done (a stat that really matters IMO), but decreasing their personal dps.

Is it possible to get the same tables listed for total damage done on boss pulls in ICC?

The WoL DPS ranking uses effective DPS, which means your dps goes down even if you do zero damage, so even if you don't have any dots and are not attacking you lose dps.

That's already taken into account. World of Logs rankings use effective DPS - damage done divided by fight length.

I do think that to some extent, all the ones on top are cheating, like staying on Lady Deathwhisper, as you said. It'd be impossible to get to the top of the chart if you ran from adds to boss, back to adds. Hell, even just switching targets hurts your damage, due to lack of debuffs on the new targets. Similarly on Blood Queen, all the people at the top are getting the bite first, or nobody really plays that spec. But, well, if that's the case, you couldn't draw any conclusions from the data anyway, since you had so few points.

Not saying your points are invalid, on the contrary. You have to ignore fight mechanics in order to get to the top, or at least work around them. So the numbers don't really mean much in terms of actual usefulness of a certain spec/class.

Being a mage I'll comment a bit on arcane mage.

I don't know what the norm is, or if there even is a norm, to how many you bring of each class. Pure math says that you should bring 2,5 players of each class to a 25 man raid. You obviously can't bring half a player so some classes will have 3 members, some 2 members, some 1 member and some might not even be there. All depending on guild preferences, who can come and fight etc.

We usually run with 3 mages. Arcane mage really shines when they get a priest to throw a shield on them because of [spell]44396[/spell]. But there are only so many priests around (albeit other classes can cast other shields to proc it also but I've never seen them do it) so not every mage with the talent will get it every time they can.

And all the players on the top spec list must have had perfect setups, ie they get all the buffs they can and probably doesn't even have to do the dirty work. For example, maybe they ignored the spikes on Lord Marrowgar and figured someone else will do it and just kept up their rotation on the boss. Which is a personal DPS gain in order to flex their e-penis, while slightly penalising the raid by not doing the fight correctly. So, you can't tell all the mages to swap to arcane and expect them to crush everything compared to fire. Albeit arcane is better at the moment than fire, but it doesn't have the 5% crit buff for example and it takes more buffing to really shine.

I guess these kind of lists are interesting, but they shouldn't be taken too seriously. Definitly not as "omg! Let's bring 23 arcane mages, 1 tank and 1 healer. We'll kill the boss in 60 seconds!"

I severely doubt every feral druid in the top 20 has Hysteria, even though it's an ideal buff (and arguably ideal for the raid, although I don't agree). I've browsed the top 20 parses looking for that very thing. While you might count on special buffs for some classes, you can't count on it for all of them. Do Arms warriors get Hysteria? Is there a Hysteria in every endgame raid?

I've also browsed logs where a few ferals use a more effective strategy at the top, followed by a "typical" strategy by later ones.

Now, this might be different for more popular classes, but less popular classes/specs will rarely be in those ideal situations. Ideal raiding conditions exist much more rarely than you might think.

Does it really matter how often they exist? They *can* exist. If people want to be perform at the top level they need to work towards making those conditions happen. I remember when you used to be able to stack as many elixir buffs as you could carry in your bag. There were a lot of comments back then about "well, how often do people really do that?" It doesn't matter.


The subject of the thread is "top performers", not "average schmucks". Yes RNG plays a big factor. Not getting Vile gassed on Festergut or having to keep up a kinetic bomb on Princes makes a difference. But a lot of the key to being a top performer is knowing how to (and being able to) put yourself in a position where you can perform in as near ideal raiding conditions as possible.


With that said, I do think it would be useful to see a variance number. Sometimes you look at the top dps and the number 1 guy is like 20% above the next closest dps.

I'm saying that there might not be 20 players playing in ideal raid conditions in more than just the terrible specs. While they can exist, there's no guarantee that the people making up that graph really are what they are assumed to be-- the best players playing the best they possibly can.

There might not be a single feral druid playing in a top-level DPS-stacking guild, with top-level gear, who plays at a high level, who gets ideal buffs, and reports it publically on WoL. Particularly because the correlation between top-level guilds and public reports is small, and a DPS-stacking guild might not give buffs to a certain spec. Then after that, you still need those players to get "lucky" fights.

There has been some talk about buffs and rng, but there's one equally important factor: adapting the fight length.

For some classes, in order to maximize effective dps, a short fight is perfect, as it increases the impact of heroism/bloodlust and haste or wild magic potions. 30s of bloodlust is a far larger increase of eDPS in a 2 min Marrowgar fight than in a 3 min Marrowgar fight. These classes need to go with the minimum of healers and optimized raid dps.

Other classes need a nice long execute phase to increase eDPS or a good alignment of cooldowns with the fight length.

Think Lich King. You could AoE the adds in phase 1 and get to the top of the meters, but that'd be worse than doing it right. The problem isn't that they stacked the raid to get there, it's that they ignored the mechanics of the fight/did it wrong. It's that doing the most damage isn't always equal do doing the most useful damage, or doing the most beneficial thing for the raid, which means you cannot really use it to judge anything.