I think what you'll find is that you're encouraging people to focus on personal gains rather than raid gains.

On Putricide I could focus on the boss in order to maximize my personal damage and DPS, or I could take the time to swap to adds so that the raid doesn't wipe. There are similar situations all throughout ICC (Bonespikes, Deathwhisper, Saurfang beasts, etc.), and if you start chastizing members because they're not matching some numbers that you pulled off the internet (that in absolutely no way directly correlates to their personal potential); you're doing your raid a disservice.

Not only will those players be more tempted to tunnel-target their DPS, but it'll foster hostility. Sure, a little bit of competition is good and helps us all improve, but taken to the extreme it'll very quickly take the fun out of the game. Last I checked that's why we were playing. :c)

Not a sermon, just a thought!

This idea is overblown. If the raid wipes, the raid is no longer concerned about padding/improving personal stats, but about killing the boss. And, to be purely Machiavellian, if the raid doesn't wipe, the raid needs to make the determination whether your not-kill-adds strategy is helping or hurting efforts (if you push Putricide to phase three, there are less adds to kill). There are already plenty of tools that allow raid leaders to see who is attacking the adds and who is not.

Why does the answer to the question of "how do I improve?" have to be "not switch to adds/abuse game mechanics/cheat in some fashion at the expense of other members"? Sure, it's the easy answer. And I guess the hard answer (get better at the game) can be tough to swallow. But, remember that "matching" the numbers pulled randomly from the internet is not the goal. Explaining why you aren't within, say, 75% of them is. It could be for any number of reasons: survivability, gear, rotations, cd usage. If your goal is improve damage in a certain fight, or even overall, this data can help narrow down which people look at.

That's not to say this data cannot be improved. Variance or standard deviations would be a good start.

Speaking of taking things to the extreme, your post is a great example I'm just going to answer one thing: we're not using just these tables to assess performance. For example when we were wiping on Putricide we looked at damage done to adds, damage taken from each source of avoidable damage, death order, ... Use different tools for different jobs, friend.

I agree, I'll try to find a way to implement the percentile-ranking idea (first suggested by Allev). That's going to take a bit of work, especially from the busy WoL developer, we'll see if/when it will be possible.

Those tables look great, do you plan to publish the application/source code to generate them?

While I agree that such measurements are not the all-end answer, they certainly help in asserting the performance of your raid members, and find individuals who underperform relative to your whole setup.
And as already mentioned, such things are only tools to help you in your further decisions.
You can never have enough tools.

Yup, I agree, the median 20 number is just a pretty small sample out of the data set. It isn't representative at all for how the average player is doing with a specific class and spec, but is an indicator of what number is possible if you're perfectly geared, have a good rotation and get lucky. We just skip 10 (a semi random choice) rows to loose the outliers all the way on the top and present a realistic target you can work towards.

The full data set including the old bosses is at 872k rows at the moment, so I doubt we'll be releasing all of that just because of performance/bandwidth issues, but I'll look into extending the data a bit more. One of the pre-requisites of near real time updates of the rankings is saving of the lower limit, I've made that table a bit wider to contain also the 10/30/50/70/90-th percentile of the top 200 itself (read the #(2*nth) value), so publishing that data is the next thing on the planning.

Saving and using top n-th% would be even better, but as top 200 is already this heavy, it isn't feasible at the moment. It's a luxury problem, I know, having too many samples that you have to drop things in order to get an answer, but we're working on it :P

I think taking a percentile from a fixed number is just bad practice, though. Taking a percentile from the entire data set would be quite interesting.

Edit: wtb reading comprehension.