I looked at both r4491, and r4492. I couldn't find any issue directly, or indirectly with how simulation craft is grading spirit.
While this is just guessing at this point, I can only think that SC is organically increasing the value of spirit BECAUSE it is a listed as a dps stat, and as a result, would be requesting a certain amount. Much like how haste values differently at various gear levels (haste out weighing spellpower in 251, but is more or less tied at 264). I admit I am no expert on code and so my hope is that someone well versed in the SC code could enlighten me on how to check this.
"Better to keep your mouth closed and be thought a fool than to open it and remove all doubt"
SC isn't supposed to do any kind of guessing or estimating, it's supposed to operate using the same algorithms it is believed WoW uses, then it does simulated combat X number of times and takes the average of the results. In other words, it shouldn't be "organically" increasing the value of spirit. It just simulates the mana regen and the spellpower bonus from spirit with the various procs related to spirit (glyph of shadow, imp spirit tap if you have it).
There could still be a bug related to spirit or how these things stack with each other that manifests from certain combinations of buff, talents, gear, glyphs or something, but it's not like Rawr where it just takes a spreadsheet-derived value of spirit and tacks that onto your DPS (not denigrating Rawr, I use both tools). It should be simulating the real contribution from spirit on every iteration and then taking the average.
Where did you read that Rawr uses a spreadsheet-derived value of spirit?
Where SC simulates ShadowPriest play, Rawr models ShadowPriest play. Simulating is by design slower than modelling, but might obtain increased accuracy.
When compared to actual real life results, both methods will always be off. Where SC might be a 5-10% off compared to what you see in raid situations, Rawr will probably be around 10-15% off. This rarely affects your gear choices a lot however.
I cannot say what happens in SC, but I know Rawr is starting to like Spirit quite well compared to Crit Rating now that you can get 1k+ Haste Rating. I have not analyzed why, but I was assuming that the Spell Power gains from Spirit do more for you than extra Crit Rating, and due to much higher mana consumption with such amounts of Haste, Spirit can extend your longevity.
Rawr - Coder of HolyPriest (Healer) and ShadowPriest (DPS) Modules. Get Your Rawr 2.3.x!
I'll try to run iterations tonight using my current gear set. I forgot to save my previous sims (which I meant to post in an earlier reply), so I will need to be more diligent in getting those out. It will be late though, since tonight we have HLK....sigh.
Where did you read that Rawr uses a spreadsheet-derived value of spirit?
Where SC simulates ShadowPriest play, Rawr models ShadowPriest play. Simulating is by design slower than modelling, but might obtain increased accuracy.
When compared to actual real life results, both methods will always be off. Where SC might be a 5-10% off compared to what you see in raid situations, Rawr will probably be around 10-15% off. This rarely affects your gear choices a lot however.
I cannot say what happens in SC, but I know Rawr is starting to like Spirit quite well compared to Crit Rating now that you can get 1k+ Haste Rating. I have not analyzed why, but I was assuming that the Spell Power gains from Spirit do more for you than extra Crit Rating, and due to much higher mana consumption with such amounts of Haste, Spirit can extend your longevity.
Spreadsheets also "model" the behavior. Don't take it as an attack. Rawr and a spreadsheet use the same method to reach their result, regardless if one is more rigorous or not. e.g. both Rawr and a spreadsheet will likely both take the average damage dealt by mindblast (1020) while simcraft iterates multiple random casts at the true range (998-1048) then takes the average (which will eventually close on 1020 with enough iterations). To be honest I haven't looked at Rawr's code, but modeling is modeling, whether done by a spreadsheet or by Rawr the inherent weaknesses are the same. I still consider it a useful tool as long as its weaknesses are known by the user, just like simcraft (which has its own inherent weaknesses) or any other tool.
Regardless, both tools have inherent margins of error, and when you're talking about 8-20 DPS increases when your total DPS is in the 12k range, you have to wonder if it's even within the tool's capability to measure that kind of tiny increase in the real world. I think it's more likely a small latency spike during the fight will have a bigger impact.
Yes, model or simulation will never match reality perfectly. And yes, you are correct, spreadsheets model the same way I do in Rawr. However, the way you wrote it, I was afraid it would be interpreted as Rawr having fixed values. Rawr produces dynamic values based on your current gear and spec. When it comes to model vs simulation, it is as you say, a simulation needs to do many iterations to get an approximate result due to random factor, while a model will use absolute averages from the start. Which is significantly faster. (Needed for gear optimizer)
Rawr - Coder of HolyPriest (Healer) and ShadowPriest (DPS) Modules. Get Your Rawr 2.3.x!
About this SC/Rawr discussion, I'd like to point out that both SC and Rawr use model, precisely (let's hope) the same model, which should be (at least their author think) Blizzard model and code.
Now, the difference is not in the fact to model or not model, but what you do with the model.
SC simulates it, and averages different simulations. Compared to Rawr, it has as advantage to take into account the full randomness of some event (such as spell base damage), and has the drawback to add some more simulation noise which is then tricky to properly remove (especially if you're interested only in the mean, not in the distribution of dps: even if averages are good at doing that, some noise is still there).
Rawr (attemps to) solve it, in a mathematical approach. Or maybe sometimes solve an approximate model, because that's easier. Hence, it does not add the additional noise of random simulation. But the simplification (such as taking mean base spell damage) induces also some lost precision.
If you don't want to use any model at all, all what you can do is to use Blizzard code, ie. Wow, to get information, mostly playing the same fight in different conditions / gear set, ideally keeping your precise spell selection and lag sequences identical. That can be useful to choose between two gear sets for example. But if you want to have some predictive power (for example, to avoid choose Strength gear), you will then use a model. For all what we know for sure, Blizzard could have decided to make your spell hit for 1000% more if you have precisely 3420 Str and 1932 Agi. Only our models, backed on verified prediction and Blizzard own's public announcement about their design, exclude such options. Predicting always require to model.
If it was only a case of taking the average of say the damage range of Mind Blast then, yes, it might be easy to underestimate the differences in effectiveness between a simulator and an averaging-modeler.
With the exception of rage using classes the amount of damage you do with a certain doesn't tend to have an impact on your rotation at all.
However, where you run into issues is when you have things that *can* effect your rotation. For example, say you had a simple spell power trinket proc. Something like Rawr would model this as an average spell power amount over the course of entire fight and would determine it's rotation based off this average spell power amount. A simulator however might well decide that having that trinket procced means you might wise to cast different spells than you would otherwise. They could especially be true in the cast of haste procs.
The same applies with talents that proc on may for example give some effect when you crit. It may cause a change in the rotation for some period. It would be rather hard to model this in the form of a spreadsheet type approach.
Relying too much upon averages, while it does make things like gear optimizers far more possible, means you're not really reflecting what spells the player will be casting and it also can mean you can suffer from some serious banding of item weights.
Now, in a number of cases a Rawr style approach is quite adequate for what a lot people want to know. i.e. What gear should I wear or aim for.
But there are some items or procs that really have no good implementation in such an approach and can lead to incorrect choices being made.
Given that there are a limited number of items actually in the game and likely to be relevant at any particular tier it's possible that some form of a gear optimizer could be made for SimulationCraft that wasn't horribly slow.
Maybe something that started off with a limited number of iterations on different gear sets to throw out those that are obviously worse while slowly increasing the number of iterations for increased precision as the differences between the gear sets decreases. It still would not be as fast as a Rawr style approach but it would have the other advantages that a simulation has.
Well I have had several people gripe at me for having to use Wowhead profiler to Simcraft different gearsets. Something that added item swapping into Simcraft would make it pretty OP tbh.
I would really like to comment on this specifically:
Originally Posted by Althor
Given that there are a limited number of items actually in the game and likely to be relevant at any particular tier it's possible that some form of a gear optimizer could be made for SimulationCraft that wasn't horribly slow.
Maybe something that started off with a limited number of iterations on different gear sets to throw out those that are obviously worse while slowly increasing the number of iterations for increased precision as the differences between the gear sets decreases. It still would not be as fast as a Rawr style approach but it would have the other advantages that a simulation has.
How can I say "fucking awesome" only with more feeling? Basically take all the usable gear from an ilvl range, Sim every combo, then spit out each gearset from best to worst? Oh, and looking forward to new and more racial bonuses you'd have to be able to select races to run the sets through. Mmm would the gemming be dynamic as determined by each gearset though or would there be static gemming? Well, I guess we'll find out when you finish, huh? (:P)
Yeah, doing that Frmercury would be pretty "fucking awesome" only with more feeling. However assuming you use 0.1 second to simulate 1 gearset, having a quad cpu core with hyperthreading (8 cores effectively), dealing with only 5 gearchoices / slot 17 slots. 17^5*0.1/8 seconds to go through all options = 17748,2125, or (/3600) 4.93 hours with 100% cpu usage. And I did not include enchants & gemming. Good luck simming every combo.
I could change Rawr to handle procs more dynamically so that you start doing spells differently or otherwise, but the loss of performance vs gain in accuracy is not worth it considering the goal of Rawr.
But as I've said earlier, neither Simcraft nor Rawr will at any time perfectly reflect reality.
Rawr - Coder of HolyPriest (Healer) and ShadowPriest (DPS) Modules. Get Your Rawr 2.3.x!
Well only one full round would have to be done for each content release then published. After that the functionality could be used for one gearset with maybe 2-3 slots with 2 choices for each. That would go much, much quicker obviously.
As Not-So-Evil mentions, it is the gems that make a difficult job nearly impossible. Some classes have truly lop-sided scale factors which enables gem-pruning down to a single choice.... but not all.
While there are 17 slots there are far fewer stat types. The list of stat types is further reduced when one only considers the relevant stats (between 6 and 8 depending upon the class). Given a collection of gear and gems, one can quickly determine the min-max for each stat. Choose a few sample points and you can create an interesting 6-8 dim matrix in a reasonable amount of time. That matrix can be used to drive the traditional DP/MC search engines that Rawr employs. The problem with this approach is that procs need to be reduced to static gear amounts, requiring a final brutish analysis to choose trinkets.
Alternatively, provided you start with a particular gear point, you can simply drop the steel ball on the wavy surface and let it roll where it wants to go. (ie: Calculate scale factors for a point in the state-space and then jump to a new point, repeat, etc.) Obviously, there are local-minima problems associated with such an approach that can be problematic with stats like ArPen. This approach handles trinket procs better, but still not great.
Personally, I prefer to think that TC users are not stupid. I know there terabytes of forum posts that might indicate otherwise, but in general the folks that really care about tweaking tend to be a cut above those "just playing".
I plan to provide a simple paper-doll interface to SimC. It will allow users to set ilevel/gem filters and scale factor sliders. Based upon those values, gear will be simply ranked in drop-down lists. Talents/Glyphs will be adjusted in a very simplistic fashion. While finding BiS sets from scratch is an interesting intellectual exercise, real player toons do not have every item in the game immediately available. My intent is to provide a simple user-assisted gear exploration tool. Minimize clicks and maximize control.
It is an engineering solution that will lack in elegance, but one that may prove useful nonetheless. This interface, along with other architectural changes, will be available prior to Cata reaching the PTR. Once Beta info starts pouring in, I want to make sure we have a firm foundation upon which to build.
If it was only a case of taking the average of say the damage range of Mind Blast then, yes, it might be easy to underestimate the differences in effectiveness between a simulator and an averaging-modeler.
With the exception of rage using classes the amount of damage you do with a certain doesn't tend to have an impact on your rotation at all.
However, where you run into issues is when you have things that *can* effect your rotation. For example, say you had a simple spell power trinket proc. Something like Rawr would model this as an average spell power amount over the course of entire fight and would determine it's rotation based off this average spell power amount. A simulator however might well decide that having that trinket procced means you might wise to cast different spells than you would otherwise. They could especially be true in the cast of haste procs.
[...]
Relying too much upon averages, while it does make things like gear optimizers far more possible, means you're not really reflecting what spells the player will be casting and it also can mean you can suffer from some serious banding of item weights.
What you point as differences is precisely what I had in mind. This are the models simplification / averages that leads to imprecision, not the fact that you solve the model mathematically and not in simulation. In theory, nothing prevents you from solving the model without avering trinket proc effects.
In fact, I really think that Rawr correctly handles procs for some classes which tends to stack procs (warriors, for example, or Firemage), as the impact between stacking procs and just using them independently (ie. average it) is not negligible.
I ran a series of tests today to work out the spirit scaling issue. As the numbers were within +/- 2dps in each iteration, I simply chose the first sim of each test. These first two use 23sp gems in blue sockets (minus belt/helm for meta). 1st iteration uses +10stats to chest, with JC/Engineering.
Notice the scaling? Sp is highest still at 1.44, followed by crit at 1.25|| 1.08 Spirit|| 0.94 Haste
...Why is spirit out-scaling haste? Why is it soo close to crit?
edit test1: in this sim, spirit is STILL out-scaling haste, though haste/crit have slightly higher numbers. Introducing int scaling seems to have had in a minimal effect vs other stats. For example, in the original test, haste was scaling at 0.93, now at 0.94
Test 2.
Changed enchant to chest from +10stats to +15spirit, gems remain the same.
The addition of both int scaling and changing chest enchant to +15spi to chest slightly reduces spirit scaling (we go from 1.08 to 1.06), SP/crit/haste increase in scaling. Haste see the smallest gain (from .9363 to .9425). Spirit STILL out-scaling haste.
Test 3: Enchant= +10 stats to chest, Gems= 10spi_12sp in ALL blue sockets (total of 5, 2 base+3 added). Professions: JC/Engineering still.
swapping 23sp gems in cloak/gloves/ring/weapon for 10spi_12sp_5sp gem/socket brings spi scaling below haste scaling for the first time. Also worth noting, that vs the original test3 (where int had zero scaling) spirit scaling was 0.9944, in this test3...spirit scaling is .9940. SP and haste are the only stats that see an INCREASE in scaling vs previous test3.
Test 4: 10spi_12sp still used in blue sockets, enchant to chest changed from +10stats to +15 spirit
Here is where it seems weird. In test 3, the scaling reversed from the first 2 tests.. spirit began dropping while sp/crit/haste increased. This test, we resume the pattern of increased spirit = increased scaling. Why?
I wanted to quote my previous statement because it was on the original test4 where things became awkward. In the new sim where int scaling had been introduced, we see that once again spirit scaling, begins to increase again. Crit also takes an increase (vs test3). I am considering running a series of sims where I would replace 60 haste worth of gems (6x10haste_12sp) with 60 crit (6x10crit_12sp) and see how that changes both haste and crit/spirit scaling.
Last edited by Bowchikabow : 06/16/10 at 3:33 PM.
"Better to keep your mouth closed and be thought a fool than to open it and remove all doubt"
WoW. How did I miss that. Going back to the save logs, I neglected to check-box "analyze intellect". I will fix this and repost. Thank you for the spot.
"Better to keep your mouth closed and be thought a fool than to open it and remove all doubt"
Are you sure that mana is a non-issue ?
I have difficulties to understand how loosing 10 Int and gaining 5 spirit (your first - second comparison) could increase your spirit scaling.
The other possibility I see is that socket bonuses, Are they only spellpower ? Or do yhey have haste/ crit ? But they should change spirit scale factor the other way around: currently, going from +23 sp to +12SP/+10 spirit gems in blue seems to reduce the interest of spirit. However, spirit doesn't scale with itself or spellpower at all : they just add eachother, they don't interact. But it scales with haste and crit.
Are you sure that mana is a non-issue ?
I have difficulties to understand how loosing 10 Int and gaining 5 spirit (your first - second comparison) could increase your spirit scaling.
The other possibility I see is that socket bonuses, Are they only spellpower ? Or do yhey have haste/ crit ? But they should change spirit scale factor the other way around: currently, going from +23 sp to +12SP/+10 spirit gems in blue seems to reduce the interest of spirit. However, spirit doesn't scale with itself or spellpower at all : they just add eachother, they don't interact. But it scales with haste and crit.
First. It was a mistake on my part for not selecting "analyze Intellect". I should have that checked. Yes, all the blue socket bonuses that I filled in Test 3/4 were +5sp socket bonus. I'll edit the post later today when I get a chance to re-run the sims. Just finished raiding and am quite tired. While I am resting, something I want to ask: Would int scaling affect the spirit scaling?
"Better to keep your mouth closed and be thought a fool than to open it and remove all doubt"
I'm not sure to what extent SimCraft simulates this, but from a mechanics point of view, crit gains from int make spellpower gains from spirit worth more, and probably in a non-linear way. Also, in case in the simulated scenario you are mana limited, the int/spirit interaction from meditation may influence spirit scaling as int is a nonlinear factor in the meditation formula.
Originally Posted by Pewsey
Many of our snakes are 3m+ in size. They'll just take the lawnmower off you and beat the shit out of you with it to make you tender, then bite you and eat you.
He's definitely mana-limited in the sims: there's mana gain from Dispersion, but no damage from SW. So Dispersion is being used not because of movement, but because of running dry. If you are running mana-limited sims, it will greatly inflate the value of spirit and intellect relative to a full-raid-buffs scenario. Similarly, haste will make you burn through your mana faster while crit is a pure DPM increase, so the relative scaling of crit and haste rating makes sense as well.
Try running again with optimal_raid=1 and see how the numbers change.
Crit (and hence Int) has two direct effect on dps scaling for spirit : first, more crit means that any spellpower gain will be multiplied by a greater constant. In simple example, you gain twice more absolute dps when you have 100% crit chance (at 200% crit) than when you have 0% crit. That effect is linear in absolute dps gain, Second, crit increases the uptime of IST / Glyph of Shadow, and hence increases your spirit -> spellpower conversion. That second effect is 1 - (1-%crit)^{#chances}, where #chances is the number of "eligible" cast during the proc time. It is sub-linear (going from 0 to 10% crit is better gain in uptime (and dps) than going from 90% to 100%).
Now, most of your troubles can be explained because of mana limitations, If you lack mana, Int is the best regen stat (not only because of meditation effect, but because also of mana pool size / shadowfiend / dispersion / resplenishment). Hence, trading 10 Int for 5 spirit is a bad deal. Similarly, trading 11sp for 10 spirit and 5 spellpower (socket bonus) is maybe a slight spellpower loss (*), but a greater mp5 gain.
* : Assuming glyph of shadow and twisted faith, you gain permanently 20% of your spirit as spellpower, and 50% every non-periodic crit for 10s.
I'll be doing back-of-envelop estimation of uptime : you have 50% total crit, 20% total haste, and spend half of the time casting dots. MF is doing only 2 ticks (1 per GCD). The GCD is 1.5/1.20 = 1.25s, and you can fit 4 non-periodic GCD every 10s. Hence, the uptime for glyph of shadow is 1 - (0.5)^4 = 87.5%. The total spirit -> spellpower conversion is (.2 + .3*.87) = .461, ie 46% of spirit converted (in average) into spellpower. Let's say it's 50% (haste is pretty conservative, MF reduced to 2 ticks in my model, and BoK or IST is not taken into account) : that means that 12sp+10spi + 5 sp bonus gem is about 22sp finally, to be compared to 23sp pure gem. For 6 sp socket bonus, it is a straight gain to gem for the bonus. For 5, that might be the case, depending on the uptime. And if you lack mana, it will be in all cases.
I will make the change list in this post for the sims. The updated sims themselves will be edited into my previous post. I will also make commentary edits where needed in the original post.
Added int scaling to all 4 sims. 1 item swap (helm, from healer T10264 to dps T10277), gear used in simulations now match gear shown on US Armory.
As both iterations of test3 (with and without int scaling) showed the highest comparative dps, I have since swapped my gems full-time to 10spi_12sp_5sp and I am seeing a positive difference not only in how long I last in a fight before popping fiend/disp but also dps. Understand that gains from such gem choices will vary player to player.
edit: the poster below does a much better job explaining the spirit conversion than I do. I made an error by using the spell power from my total spirit both before and during IST/GoShadow, which skewed my numbers.
Last edited by Bowchikabow : 06/16/10 at 6:04 PM.
Reason: Spelling, added gear edit, removed unnecessary section.
"Better to keep your mouth closed and be thought a fool than to open it and remove all doubt"
As both iterations of test3 (with and without int scaling) showed the highest comparative dps, I have since swapped my gems full-time to 10spi_12sp_5sp and I am seeing a positive difference not only in how long I last in a fight before popping fiend/disp but also dps.
IST does need to be taken into account though. at 498 raid-buffed spirit, IST would give me 547spirit (or 273.5 spell power... 547*50=27350/100=273.5 with GoShadow which would activate at the same time as IST, this would give both IST/GoShadow equal up-time). That seems pretty significant to me.
No, IST gives you 10%*498 = 49.8 spirit, or 24.9 spellpower. With 90% uptime, that's 90%*24.9 = 22.4 spellpower on average. 11.2 spellpower per talent point is not significant, there is simply no better place to put those points.
It seems like bad science to me to base your raid gearing decisions around the results of 500 second solo training dummy simulations. I'm not saying it's a bad idea to match blue +5 sockets, just that this work doesn't support that conclusion. In any case, the scaling numbers you found were around 1.1 spirit / 1.4 spellpower, so you should be seeing a DPS increase - on 500 second training dummy runs - of about 1.1*10 - 1.4*6 = 2.6 DPS per socket. In full-raid-buffs with your gear the numbers are 1.27*10 - 2.20*6 = -0.5 DPS per socket.
It really doesn't make sense to claim that at high gear levels spirit somehow gains vs. raw spellpower: in a non-mana-limited scenario, the DPS gain of spirit is purely from its conversion into spellpower.
Edit: Analytically, 1 spirit on gear is scaled by Blessing of Kings (10%) and Improved Spirit Tap (10% * ISTUptime). That scaled spirit is then converted to spellpower from Twisted Faith (20%) and Glyph of Shadow (30% * GoSUptime). So 1 spirit provides on average 1 * 110% * (1 + 10% * ISTUptime) * (20% + 30% * GoSUptime) spellpower. For you in a full-raid-buffed Patchwerk scenario, the uptimes from Simulationcraft are 94% (Improved Spirit Tap) and 99% (Glyph of Shadow) resulting in 0.598 spellpower per spirit. This analytic value is slightly higher than simulationcraft's reported spirit scale factor of 0.56 PP, but within a reasonable margin of error.
It is notable that even with 100% uptime on both procs, spirit is at best worth 0.605 spellpower. It will never be a pure DPS increase to match blue +5 sockets, but as our crit rates increase, we approach break-even.
Last edited by CaseyTheRetard : 06/16/10 at 5:40 PM.