[Mage] Rawr Spec Shootout

MyrddinE · 06/09/08, 3:53 PM

So I just burnt three or four weeks of time (mostly while goofing off at work) running hundreds of tests with Rawr to see which Mage spec beats all.

I initially started this test to prove that Frost mages are the best DPS at lower gearings, but I found some surprising results... but why don't you look for yourself. The results are here: Mage Spec Spreadsheet

From the spreadsheet:

Overview
This spreadsheet was imported from Excel, so obviously some formatting was lost. In particular, the cell backgrounds, which were a dynamic conditional format in Excel, were manually added by me in the Google version.

The first tab, Shadowpriest, shows the maximum DPS achieved by the various talent specs at each gearing level. I cleared Rawr's item cache, and repopulated it with equipment capped at the given iLevel. Then I asked Rawr to optimize the equipment for the best gear it could, given the spec and available gear.

Other than iLevel, there are a few other differences between the gearing. iLevel 114 (pre-Kara) sockets only green quality gems, and only the basic global buffs are applied; a player at that level is most likely doing 5 man content. iLevel 125 (Kara gear) sockets only blue quality gems, and most global buffs are applied. iLevel 135 gear and above sockets purple quality gems, and all global buffs are applied. Only one group-specific buff (paladin auras, Draenai auras, Totems, etc) is ever included: Heroism. This helps eliminate discussion of group makeup on the Mage's DPS.

Obviously, the most important tab is the Shadowpriest tab. That shows the DPS of the various specs with different levels of equipment, assuming you have a Shadowpriest with you. This is the tab where where I ran a full Optimize for every single value on the list (often several) except for the very last one (164 + all buffs).

Then I took every one of those builds, no change in equipment, and saw how well they did without a shadowpriest. That's on the NoShadowpriest tab.

Finally, I took all of those builds (with Shadowpriests), and ran them through different fight lengths (with no change in equipment). That's all the Time tabs, one tab for each level of equipment. This shows the burst damage capability of a build, as well as how mana-efficient it can be for long endurance fights.

Highlighting
The highlighted boxes on each screen are in the top 10% of damage for the given row, using the 0/0/0 talentspec as the baseline.

For example, on the Shadowpriest tab, at the 125 iLevel gearing, the highest DPS spec is 18/0/51 with 1420.72dps. 0/0/0 clocks in with 587.71dps. 1420.72 - 587.71 = 833.01 dps from the highest to the lowest. So the top 10% of the specs are within 83.30dps from the top spec... or anything over 1337.42dps. That is what is highlighted in red.

Results
Frost builds did surprisingly well. Though some Fire builds were able to beat Frost at the very end gearing (this is the best possible gear in the game, assuming you have anything you want out of Sunwell, badge gear, etc), Frost was in the lead by a tiny margin the entire way up. The advantage of Frost is even more evident when you look at different fight lengths... short fights really favor Frost, where it can pop out two Elementals; long fights favor Frost because it has better mana efficiency. I expected Frost to win at low gearing levels (in fact, proving that was why I started this project) but I did not expect it to do so well at the end.

Arcane did susprisingly poorly. In fact, I was so convinced that something must be wrong with the optimizer that I worked on the 164 gearing manually, trying to find a set of gear that got Arcane to do better... I was able to beat the optimizer, but unable to get Arcane even close to Fire or Frost. This contradicts conventional wisdom that at very high gearing, Arcane's large mana pool enables it to shine. Even in very short fights Arcane is unable to beat Fire or Frost. This either indicates that Arcane proponents are wrong, or perhaps that Rawr is not modeling Arcane rotations well.

Fire did about what I expected... not so well early on, but a strong finish. No real surprises there. It's worthwhile to point out that Fire gains a larger benefit from Shaman totems than any other spec... Fire noticably pulls ahead in peak theoretical DPS when put in a group with a Shadowpriest, Moonkin, and two Shamans in order to get every possible buff in the game (note the last row on the Shadowpriest tab).

Further Improvements
In the Elitist Jerks thread, I posted all the files used to construct this data, including the slightly modified version of Rawr (I added two mage specs to the drop-down list, Fire/Meditation and Frost/Meditation). Please feel free to take those files and find better optimized sets of gear for a given spec and gearing, and I will update this spreadsheet with the best results. Make sure you use the same version of Rawr (only available in that thread), unless you feel like re-doing ALL of the DPS numbers in a more recent version of Rawr. Also make sure you use the included itemCache files... don't accidentally include higher iLevel gear on a lower-iLevel character.

If you do post a better DPS number for a given spec and iLevel, you need to provide the character file that created it. Attach it to the post or something. I will not update DPS numbers unless I can verify that the gear is of the proper level, the talents match the listed specs, and the fight and buff options were not altered.

You can download the Rawr build I used to create this data, the source code, and all the saved characters and ItemCache files for the various item levels. The data files also includes the original Excel spreadsheet that the online spreadsheet was created from.

Kavan · 06/09/08, 6:17 PM

There's a very important problem with these results. All the characters have race set to night elf. This means there are 0 base stats which hurts arcane quite a bit since it's more stats based.

MyrddinE · 06/09/08, 6:33 PM

That was deliberate, though I forgot to mention it. Some races have int bonuses, some spirit, some have haste buffs... I didn't want to tie the computation to any particular mage build, so I removed all racial benefits to mages by setting it to a race that has none. There is no other way to model all the talent specs evenly.

MyrddinE · 06/09/08, 6:40 PM

Aww crap... now I see what you mean. The base stats are not included at ALL... as in, no gear means 0 int. Well shit.

In that case, which race do you believe has the worst racial ability, or one that can be disabled/ignored? I'd really hate to have to run the numbers 5 times.

Kavan · 06/09/08, 6:43 PM

You could easily take Undead that has no mage specific racials.

• Wizeowel · 06/10/08, 10:22 AM

Indeed, the file "135+200=1581.44@40-00-21.xml" jumps from 1581 dps to 1831 dps just by selecting undead

Sancus · 06/10/08, 10:48 AM

Only one group-specific buff (paladin auras, Draenai auras, Totems, etc) is ever included: Heroism. This helps eliminate discussion of group makeup on the Mage's DPS.

This doesn't make any sense. You assume Heroism but no totems? That's a nonsensically unrealistic assumption. There is absolutely no reason not to include Wrath of Air if you're going ot assume Heroism.

Somegnome · 06/10/08, 12:14 PM

Actually, it does make sense.

Heroism is a once in ten minutes push of a button, totems require the shaman to be alive and to keep them up every two minutes, unless they get destroyed for some reason which makes it even less likely they are up all through the fight.

Last but not least, heroism doesn't have a range limit, in many fights it can happen more or less easily for a mage to be seperated from the totems, in terms of range, though.

However, I'd like to inquire about how rawr models heroism in the mage module. Does it take into account that for the sake of maximizing dps of a fire mage, it's best to be used at 20%? Apologies if this seems a stupid question, I just found no mention of this in the sheet and think it's important enough to ask about.

Thank you very much for this sheet anyways, I found it very interesting to skim through.

Sancus · 06/10/08, 12:21 PM

Heroism is a once in ten minutes push of a button, totems require the shaman to be alive and to keep them up every two minutes, unless they get destroyed for some reason which makes it even less likely they are up all through the fight.

Last but not least, heroism doesn't have a range limit, in many fights it can happen more or less easily for a mage to be seperated from the totems, in terms of range, though.

False. Heroism requires the shaman to be alive when the boss hits 20-30% or thereabouts, not "once in 10 minutes", because mage-group shaman are required to use it at that time. Any assumption which assumes your shaman is dying before 2 minutes into the fight is a bad one, since no one benchmarks dps assuming their group is going to die. You may as well add death chances in for VT/VE and for YOU, to average out your dps assuming you die some of the time.

Shaman should always be in range of their group, if they're not, your raid leader is incompetent or your shaman is. Not a valid reasoning.

Any benchmark that includes Heroism but not totems is wrong and not worth looking at.

Otterpop · 06/10/08, 1:21 PM

*edit* my point was irrelevant, since Rawr spec comparisons are for high-end endgame. I'm pretty casual, as is most of my guild, and rarely get heroism when I should anyway. Mods, feel free to delete this post if need be.

MyrddinE · 06/10/08, 1:57 PM

I included Heroism because it's a core mechanic of high end raiding. Stacking Heroism + Icy Veins + etc when the boss is at 20% is such a powerful mechanic that raids will go to a lot of effort to hit that. I've seen raids swap a melee Shaman into the caster group at 25%, just so he could pop heroism for the mages.

That, plus Heroism's longer range (100 yards rather than 20ish for totems) means that Heroism is much more likely to be available than any particular totem. In addition, unlike the other totems which have a flat addition to damage, the Heroism totem's haste effect alters rotations more... Arcane specs can benefit very little, sometimes, at high ends from heroism.

Because it's a powerful, often used buff that alters the dynamics of one spec vs another, I thought it was important to include it despite being a group buff.

Lysistrate · 06/11/08, 6:02 AM

Hi and thank you for your work.

For sure you are right that for swapping, a shaman can basically sit anywhere. in high end raiding group composition is crucial. with max 3 mages it looks like this:
Shadow
Mage
Mage
Mage
Shaman

Do the calculations with a resto shaman.. I bet this is the most common raid setup.

Setup with 2
Shadow
Mage
Mage
Shaman
additional healer due to shadow support

So your statement is kind of contradictory... high endgame raiding with shadow support and heroism but nothing useful fills the free slot?

Feel free to browse the brutallus logs... I bet you will find in every mage grp a shadow and at least a resto shaman.

Ranec · 06/11/08, 9:50 AM

I have recently become rather suspicious of Rawr. Based on the results you found through an exhaustive search of viable specs for mages, I don't know that Rawr is correctly modeling certain aspects of the mage class correctly. I became convinced of this last night, when a tier 4 mage using pure arcane blast was #1 raid dps during trash, beating a tier 6 rogue (normally #1 dps) wielding the Illidan offhand. Granted, the mage had a shaman popping heroism and drums, and a shadow priest restoring mana. However, the rogue also had two shamans popping totems. Based on the above Rawr results, this should not have happened, but it was a legitimate occurrence (no random disconnects, etc). Of course, this was only during trash, arcane blast didn't seem to be as viable during long boss fights due to mana problems.

Also, I examined the source code to Rawr. Results are built using a rather long list of shot rotations that the author developed, probably borrowing from personal experience as well as the existing knowledge base. However, I was able to identify a few obvious rotations that were not present (or I at least didn't see). Hopefully the author will recognize the advantage of building an algorithm that seeks out the most efficient shot rotations, perhaps using a genetic algorithm. Given the plethora of available talents, it would be difficult to enumerate all possible shot rotations manually, and Rawr isn't there yet.

In summary, take what you get from Rawr with a grain of salt. It is extremely difficult to do what Rawr is trying to do, and it needs some additional work to get there.

Pintofbrew · 06/11/08, 10:02 AM

Originally Posted by Ranec

I have recently become rather suspicious of Rawr. Based on the results you found through an exhaustive search of viable specs for mages, I don't know that Rawr is correctly modeling certain aspects of the mage class correctly. I became convinced of this last night, when a tier 4 mage using pure arcane blast was #1 raid dps during trash, beating a tier 6 rogue (normally #1 dps) wielding the Illidan offhand. Granted, the mage had a shaman popping heroism and drums, and a shadow priest restoring mana. However, the rogue also had two shamans popping totems. Based on the above Rawr results, this should not have happened, but it was a legitimate occurrence (no random disconnects, etc). Of course, this was only during trash, arcane blast didn't seem to be as viable during long boss fights due to mana problems.

Firstly, you must be refering to T5 mage, not T4. T4 with Arcane Blast would net you a great big bowl of Fuck-all. Secondly, AB is massively powerful on Trash. I am never beaten by any rogue on trash, and this is purely because trash is so short I can manage to 100% rely on AB, which is nigh-on 2300dps. Thirdly, the way trash timing is staggered, I get one set of CDs every other pull, meaning that 2kDPS just hitched a ride to 3200DPSville for 20sec. (by stacking a 20% haste buff, a +436dmg and a global +30% damage, all with no costly consumables.)

For arcane you mention T5 as an indication of gear superiority, not realizing that it means a 20% buff to 100% of the damage out. This does not hold true on Bosses, of course, because on Bosses AB can only account for 60-80% of your output, the rest being a low-DPS rotation (~1500DPS)

Lastly, a lot of Rogue output is due to CD stacking of their own. I'm pretty positive no rogues do Haste pots on trash and I'm equally sure running around on trash is a lot more time-wasting than Tabbing. Not to mention Blade Flurry is a no-no due to Poly.

There is nothing wrong with Rawr, there is nothing wrong with your Trash DPS observations. Both are WAI and wearing a Glave OH will not help when on trash I hit every 1.3sec for 2400 ABs and every other pack with 1sec 4k ABs, before taking crits into account at 37%.

Maledict · 06/11/08, 10:10 AM

I have recently become rather suspicious of Rawr. Based on the results you found through an exhaustive search of viable specs for mages, I don't know that Rawr is correctly modeling certain aspects of the mage class correctly. I became convinced of this last night, when a tier 4 mage using pure arcane blast was #1 raid dps during trash, beating a tier 6 rogue (normally #1 dps) wielding the Illidan offhand. Granted, the mage had a shaman popping heroism and drums, and a shadow priest restoring mana. However, the rogue also had two shamans popping totems. Based on the above Rawr results, this should not have happened, but it was a legitimate occurrence (no random disconnects, etc). Of course, this was only during trash, arcane blast didn't seem to be as viable during long boss fights due to mana problems.

Why is this a surprise? Arcane spec is perfect for trash - it can blow all it's mana every pull, it has incredibly low threat thanks to talents, and requires no build up unlike fire or frost to reach maximum dps on each mob. Even tier 6 rogues have to accumulate combo points and lose them swapping targets.

The more important question is, why are you parsing and comparing trash?

For the original post, ignoring all group buffs apart from Bloodlust makes the comparisons fairly worthless I'm afraid. The advantage fire spec has over frost is that fire spec scales much better, thanks to the pet not gaining many of the mages abilities. Also, it might be best if you considered times in terms of mobs, rather than pure numbers - apart from the Shade of Akama final burn, for example, I cannot think of where I'd use a 100 second parse.