Rogue DPS Simulation Spreadsheet - Page 2

Shaithis · 04/03/09, 9:21 AM

By the way, I will point out that there is no real statistical value of ever going over 35 iterations.

biberon · 04/03/09, 7:32 PM

While this spreadsheet is interesting to play with, I'm seeing considerable variability in terms of the output dps. Running the dps simulation 20 times I've observed the finished dps swing from 4500 to 5200. To eliminate potential user error, I downloaded the spreadsheet again under a different file name. Making no changes to default settings I ran 20 simulations and came up with a 600 dps spread; 5,333 for a high, 4,724 for a low.

Your sheet is no small undertaking and I certainly mean no disrespect, but the output variability is far greater than the dps contribution of any given piece of gear. I feel that to obtain consistent results the number of iterations would need to be very high.

Genre · 04/03/09, 7:58 PM

Originally Posted by biberon

While this spreadsheet is interesting to play with, I'm seeing considerable variability in terms of the output dps. Running the dps simulation 20 times I've observed the finished dps swing from 4500 to 5200. To eliminate potential user error, I downloaded the spreadsheet again under a different file name. Making no changes to default settings I ran 20 simulations and came up with a 600 dps spread; 5,333 for a high, 4,724 for a low.

Your sheet is no small undertaking and I certainly mean no disrespect, but the output variability is far greater than the dps contribution of any given piece of gear. I feel that to obtain consistent results the number of iterations would need to be very high.

The number of iterations per run will without a doubt increase the stability of your results. Just so we are clear on what we are talking about. A simulation is composed of a series of iterations (realisations if you will); taking the average of these should be what a simulation returns. Obviously, if you are running with a very low iteration count, you will induce volatility and your values will be all over the place between 2 different simulations.

sp00n · 04/03/09, 8:00 PM

Originally Posted by biberon

While this spreadsheet is interesting to play with, I'm seeing considerable variability in terms of the output dps. Running the dps simulation 20 times I've observed the finished dps swing from 4500 to 5200. To eliminate potential user error, I downloaded the spreadsheet again under a different file name. Making no changes to default settings I ran 20 simulations and came up with a 600 dps spread; 5,333 for a high, 4,724 for a low.

Your sheet is no small undertaking and I certainly mean no disrespect, but the output variability is far greater than the dps contribution of any given piece of gear. I feel that to obtain consistent results the number of iterations would need to be very high.

I assume this is why Malvanas stated that higher number of iterations = more precise results in the 'How to Use' section of the sheet?

Increasing number of iterations (max is 1000) will increase running time but can increase precision of the results. 300 iterations appeared practical for my previous calculations.

Of course, if you're seeing a 600 DPS spread while doing 20 simulations with 300 iterations each, that sounds like a real problem. If you're just running 1 simulation with 20 iterations though, this is an expected result. Just like in WoW itself, RNG > any gear upgrade in a relative small amount of swings (which honestly, 20x3minutes or whatever still is).

biberon · 04/03/09, 8:08 PM

Originally Posted by Genre

The number of iterations per run will without a doubt increase the stability of your results. Just so we are clear on what we are talking about. A simulation is composed of a series of iterations (realisations if you will); taking the average of these should be what a simulation returns. Obviously, if you are running with a very low iteration count, you will induce volatility and your values will be all over the place between 2 different simulations.

Agreed, however I did use the default of 300 iterations, so it was 20 runs @ 300. On second look I'm not so sure it's as simple as running this particular simulation enough times to find a solid mean, as I just bumped the iterations up to 2,000 and the first two runs produced 4,780 dps followed by 5,527. No further runs at 2000 iterations were made.

I have no mind for programming and only tinker with excel so I'm afraid I'm only as useful as reporting observations.

Edit: I didn't catch the max iterations being 1,000. Re-ran 10 runs @ 1,000 iterations:
No changes were made to the downloaded spreadsheet aside from setting iterations to 1000.

Max dps - 5411
Min dps - 4803

Individual results:
0) 5411
1) 5184
2) 4803
3) 5035
4) 5270
5) 5092
6) 4927
7) 4852
8) 5099
9) 5181

Mavanas · 04/03/09, 10:43 PM

Biberon-

I am not observing what you are reporting. The differences from run to run that you are reporting are not normal, so you need to explain to me which setup you are using. Even though I have ran the file several thousand times by now and I have not seen anything like that, I just ran it 3 more times out of curiosity and got three results within 10dps of each other which is totally normal.

Does anyone else see the kind of behavior Biberon is observing?

Also I disagree with Shaithis. More iterations decrease variance of the mean value, so it does help to run more, but as I said 300 has been a good enough number for my purpose. The variance of the mean value is proportional to 1/n where n is the number of iterations. At some n the variance is low enough so that further increases of n are not practical because they increase calculation time too much.

biberon · 04/04/09, 1:20 AM

Originally Posted by Mavanas

Biberon-

I am not observing what you are reporting. The differences from run to run that you are reporting are not normal, so you need to explain to me which setup you are using. Even though I have ran the file several thousand times by now and I have not seen anything like that, I just ran it 3 more times out of curiosity and got three results within 10dps of each other which is totally normal.

Does anyone else see the kind of behavior Biberon is observing?

<snipped for clarity>

Edit: I carried my testing over onto the laptop which is running Excel 2002 and found much more consistent results (albeit at a snails pace). Turns out the spreadsheet doesn't play nicely with Excel 2003 in its release version, and fleshing it out with SP3 has cured the anomoly.

Manigra · 04/04/09, 6:55 AM

Is it not possible in excel to provide a seed for the random number generator in excel?

For instance for iteration 1 you seed the rng with the value 1, for iteration 2 seed it with value 2 and so on. This would provide you with the same result each time you calculated your dps (that is until you changed a stat)

Mani

sp00n · 04/04/09, 7:38 AM

Originally Posted by biberon

Edit: I carried my testing over onto the laptop which is running Excel 2002 and found much more consistent results (albeit at a snails pace). Turns out the spreadsheet doesn't play nicely with Excel 2003 in its release version, and fleshing it out with SP3 has cured the anomoly.

The RAND function was changed in Excel 2003, and there was a bug in early versions which could result in negative numbers. Your observations may be related to that.

Description of the RAND function in Excel 2007 and in Excel 2003
The RAND function returns negative numbers in Excel 2003

Mavanas · 04/04/09, 9:23 AM

Originally Posted by Manigra

Is it not possible in excel to provide a seed for the random number generator in excel?

For instance for iteration 1 you seed the rng with the value 1, for iteration 2 seed it with value 2 and so on. This would provide you with the same result each time you calculated your dps (that is until you changed a stat)

Mani

No, excel does not do that. Also I now remember that at the very beginning of my programming of RNG in Excel, I observed those negative random numbers and found the same hotfix on Excel website. Once you apply it, your Excel should work perfectly fine. Thanks Sp00n.

Mavanas · 04/08/09, 11:32 AM

I implemented more 3.1 changes in my new version of the spreadsheet:
- added T8 pieces and their bonuses
- new armor penetration and reduction mechanics based on latest PTR findings
I also fixed the envenom bug and added socket bonus values to the gear sheet to make it a bit more user friendly. You still have to activate the socket bonuses manually, but do not have to look them up on wowhead anymore.

I updated the main post with new links.

Mavanas · 04/09/09, 10:12 AM

I'll post the findings based on the simulation in this thread as opposed to 3.1. The program still has occasional bugs and until it's been peer reviewed and tested I don't want people in the 3.1 thread to start learning new cycles for 3.1 based on it.

I fixed two bugs yesterday. One is related to wound poison, where it was not affected by Vile Poisons in error. Second was with templates of combat specs, two of which did still did not contain lightning reflexes. With those changes done, combat has pulled ahead of mutilate and 15/51/5 has pulled ahead of 18/51/2. I also note larger variance in dps of the 18/51/2 spec, which has been anecdotically confirmed in the 3.1 thread.

I have replaced the current files with the fixed versions without changing the file name. So please update your versions by clicking the links above.

Now to the current findings about 3.1 dps:

Tree		Spec	DPS	Gear Notes				Poisons
Combat		15/51/5	6573	5/5 T8 poison hit capped, AG gems		WP/DP
Combat		7/51/13	6543	5/5 T8 poison hit capped, AG gems		WP/DP
Combat		18/51/2	6524	5/5 T8 poison hit capped, AG gems		WP/DP
Mutilate		51/13/7	6507	5/5 T8 poison hit capped, exp capped, AG gems	IP/DP
vHaT 2c/s		8/20/43	7660	5/5 T8 poison hit capped, exp capped, AG gems	WP/DP
vHaT with LR 2c/s	8/20/43	7726	5/5 T8 poison hit capped, exp capped, AG gems	WP/DP
nvHaT 2c/s	23/5/43	7706	5/5 T8 poison hit capped, exp capped, AG gems	IP/DP

Compared to the previous results posted in 3.1 thread, the changes include the 2 bug fixes, using full t8 instead of just 4 pieces, using agility gems as fillers for all specs and also adding a vHaT spec with LR currently appears to be highest dps spec out of all specs.

testor · 04/09/09, 10:59 AM

I get the same ratios on the numbers as well. And I find the modified 8_20_43 (LR) to be slightly ahead. However, by playing with the combat CPs (5/4) you can get them to flip flop position mildly.

We need to see how it actually performs, but I'm still thinking 8_20_43 (LR) is going to be significantly better due to the dynamics of moving around and paying attention to the 23_5_43. Of course the RNG plays such a huge factor, it will take a little time to get solid multiple encounter results that can be honestly compared.

MentalPROblem · 04/09/09, 12:17 PM

Thanks a lot for your work!
Is there any chance you could post ptr.wowhead.com or just wowhead.com links to the exact talent builds and names of the weapons you were using? I'm afraid I might be thinking of a wrong variation of the spec, is this the spec you are referring to as vHaT with LR 8/20/43 with Webbed Death/Webbed Death, WP/WP? thanks in advance.

ieatpaperbag · 04/09/09, 12:24 PM

Originally Posted by MentalPROblem

Thanks a lot for your work!
Is there any chance you could post ptr.wowhead.com or just wowhead.com links to the exact talent builds and names of the weapons you were using? I'm afraid I might be thinking of a wrong variation of the spec, is this the spec you are referring to as vHaT with LR 8/20/43 with Webbed Death/Webbed Death, WP/WP? thanks in advance.

The right-hand column says what poisons are being used, the two 8/20/43 specs use WP/DP and the 23/5/43 spec uses IP/DP.