I made a post on the shadowpriest.com forums, but thought I would post it here as well in case you didn't see it there. Basically I'm having a problem in that the results given out by simcraft are exactly he same every time.
Even if I reduce the simulation to 1 iteration, I still get the exact same result every time, both for the dps done and the scaling factors (although the results for 1 iteration are different to the results for 100 or 1000 iterations).
It seems to exhibit the same behaviour for any raid setup, but the specific one I was using for most of my tests was Simcraft v332.4. I was using it straight from the download, using the priest T10 264 BiS item set and turning on scaling for spirit and spellpower. All other settings were unchanged from the default.
I even tried deleting all the simcraft files and reinstalling it, just in case simcraft was clever enough to realise it had already done this particular simulation and therefore was simply reloading the results rather than recalculating them. But still got the exact same results.
I'm no programming expert but it kind of looks as if the simulation is using the same RND seed for every run (hence leading to the same results). All I could find on the forums was the following:

Originally Posted by dedmonwakeen
|
Have you considered using seed-based RNG to allow for repeatable results? I deal a lot with simulations for my work, and was surprised that it uses non-repeatable RNG. It seems like that would allow for faster gear/stat comparisons, as you wouldn't need to do as many iterations for each gear set to get consistent mean values.
|
I guess the short answer question to your first question is "yes". Simply set the parameter "seed" to a numerical value.
It is very very very tricky to add determinism into the RNG without creating edge conditions that result in discontinuities in the scale factor generation. One of our developers (Nenad) invested a considerable amount of time creating so-called "smooth" RNG packages. (See sc_rng.cpp for details.) He successfully reduced the number of iterations required for convergence by two orders of magnitude. This can be experimented with by setting smooth_rng=1.
However....... There were certain stats for certain classes (Shadow Priests and Haste, for example) that did not behave well with smooth_rng=1. Discontinuities began to appear in the DPS/Stat curve that resulted in step-functions in scale factor generation. I still believe that this functionality is vital to gear exploration....... but it needs more investment of my time. Until that happens, I have to turn it off by default.
|
However, this seems to indicate that the default setting shouldnt be using a particular seed for all runs. I did try messing around with the setting "seed=<number>" but it didn't seem to make any difference.
While playing around with 1 iteration simulations I noticed something else a little odd as well. The priest T10 264 BiS gear ends up with hit% of 10.98%. i.e. about a 1 in 5000 chance of missing a spell.
Now, with 1 iteration (c. 300 casts) this isn't likely to result in a miss. Yet when I turned on the heroic_presence buff the calculated dps actually went down! Fair enough I thought - maybe the buff is doing something which messes with the RNG seed and causes a different result to be calculated (which could well happen to be lower).
However the confusing bit happened when I tried the same test with the priest T10 277 BiS gear which is actually hit capped (has 11.02% hit on the gear). This time the results with heroic presence were exactly the same as without it.
In case you are curious, the only reason I was looking at this at all was during a discussion about scaling factors on the shadowpriest.com forums. Someone seemed to be suggesting that the scaling factors were reliable to 0.01 but I'm not convinced that even 10k iterations is enough for this. I thought the best way to test it would be to try exactly the same simulation a few times and see how variable the scaling factors were. Thats when I noticed they were identical every time and wondered what was happening.