Machinator

This thread is a product of the Execution/Mongoose thread to determine the proc rates of enchants and items and develop tools to do so. Currently we pretty much have assumed numbers, but the data doesnt always match and it would be good to nail these down.

Data Collection:
I think it is best if we set some standards for testing so we can combine the data for more accurate results. We want both fast and slow tests so we can tell the difference between PPM and % procs. I would suggest 1.4 and 2.6 speed, and maybe a 3.6 speed 2h, as they are most common. To start no passive haste or haste procs should be used as their effect on PPM enchants is in question.

Proc rate determination:
Im not sure what the best ways of doing this are yet without posting a whole bunch of stuff on statistics. Percent based procs are easy to understand. Proc Per Minute is the average procs you would expect per minute regardless of weapon speed. I have seen evidence, but not sure if it is conclusive, that for normal attacks(not special) the chance of a proc is adjusted by the current speed of the weapon under haste rather than the base speed.

Utilities:
It would be great if we could get a program to parse combat logs and output information like hits, procs, multiple procs, uptime, ect.

"Information is ammunition."

 

• Aldriana

I'll work on assembling a program to do the parsing; I have most of the code in place, it's just a matter of grafting an appropriate UI onto it.

In terms of proc rate determination: for those of you that wish to take data, the general rules of thumb are as follows:
1) unequip all items that proc haste effects - Thundering Skyfire Diamond, Mongoose, Dragonspine Trophy, etc. - with the (possible) exception of the proc you're testing.
2) only equip a single weapon; while two-weapon testing is sometime useful, it is only circumstantially possible; in general, testing with one weapon is safer.
3) If you have haste rating items equipped, record how much and report this value with the data.

For information on the interaction between haste and PPM effects, I would refer you to the following thread: Proc Per Minute Mechanics & Haste
The information found there, while not 100% conclusive, is reasonably convincing that PPM effects on noninstant attacks is based on current weapon speed, and on instant attacks it is based on base weapon speed.

In terms of weapon speed: many rogue will find either 2.6 or 2.7 to be convenient to test, and either 1.4 or 1.5 as well; relatively fewer will find both to be convenient. Warriors/shaman, I can't speak for. However, we don't need to standardize across all tests; so long as data for any particular proc is collected for the same speeds, we can change between procs. So, for instance, Mongoose testing has started at 1.4 and 2.7; but we could still do, say, 2.6 and 1.5 for Executioner.

Additionally, some procs were pinned down in some detail in the Mechanics Testing Thread, which I can't find anymore but do remember a few results for:

1) Rod of the Sun King is ~4.5% proc rate, hence hypothesized as 1 PPM. Would need more testing to determine if it's actually 4.5% or if it's 1 PPM, as they perform differently with haste.
2) Dragonspine Trophy is 1 PPM with a 20 second cooldown
3) Tsunami Talisman is 10% on crits with a 45 second hidden cooldown.
4) Band of the Eternal Champion is 1 PPM with a 60 second hidden cooldown.
5) Madness of the Betrayer is 1 PPM with no internal cooldown.
6) Warp Spring Coil is 25% on specials, with a 30 second internal cooldown.

Stuff that I don't remember seeing detailed tests on, and/or forget what the answer was, that are of general interest to endgame melee DPS:

1) Mongoose
2) Executioner
3) Siphon
4) Blacksmithing Maces
5) Deathmantle 4/5 Set Bonus (was narrowed down but I don't remember seeing a conclusive proof)

There are probably others, but those are the ones that immediately spring to mind.

 

• koaschten

Might be of interest to give a short walktrough on what mobs to test on and how to reach them.

 

hannigaholic

For a rogue (and anybody else who can get there quickly with either stealth pots or invisibility pots if possible) I think the best mobs to test on are the Gordok Spirits in Dire Maul North. Just stealth to the King's room and unload for hours on end behind a Spirit. They don't turn around to face you so you're attacking from behind, as you would in a bossfight, and they never drop below 100% so you can keep going forever.

 

Machinator

The gordok spirits are probably best. Level shouldnt matter since we want all attacks to land anyway. I think blasted lands servants are probably as close as you can get to them, you will parry, but I dont think it will affect the data.

If you can solo a DM north boss then you can get to the spirits behind the king. Invis on its own wont help as you need to open the inner door somehow and there are mobs to kill. I did it in dps gear with a couple pots and a death to kromcrush's adds after I killed him. It can probably be improved upon for the better prepared people. Now that I think about it ogre suits might work too if you can get them.

The hard part is not going AFK while you test. I think you log after 15m of inactivity, so you need to press a button while you watch TV or something. I cant think of any way around that that would be legal. I load up the PTR and Live at the same time and run AV and test.


Also, so we can get it out of the way, does anyone have a whole lot of haste gear? I dont know how much you need to be statistically significant but somehow I would like to conclusively test if haste changes the proc rate of a PPM enchant or not.

"Information is ammunition."

 

• koaschten

Guess i can dig up crafted shoulders / bracers, 2 BT rings, Grips of Damnation in my bank, get shallow-grave trousers for badges and take a blade of infamy to Dire Maul. Should be 194 haste rating / 12.3% passive haste. I'll see if i can manage to get on the EU PTR over the holidays, last weekend when i tried i couldn't even get the EU PVE PTR to show up in the realm list, even after completly wiping and re-installing the wowtest folder.

 

Xoya

I can guarantee that Syphon of the Nathrezim has no internal cooldown, at least none worth mentioning. I don't remember if I have seen it refresh itself but I know for a fact I've had it proc immediately after the buff ran out.

 

◊ Malan

I could have sworn that the conclusion of that thread was that haste did not have an effect on PPM?

Vyktianity already has over 75 billion followers. The first verse in his book "Gift of the Wild" is "In the beginning Vyk cast lifebloom and then maul."

 

• Astrylian

One thing to add... Be careful about tests on those Gordok spirits, or really anything besides 70/71/72/73s. If you come up with a very different proc rate than expected, that could be why. At least in one case (Idol of Terror), it appears that the proc rate scales with target level. Most procs don't seem to work that way, but just thought I'd mention it, in case anyone's looking for an explination of why their proc rates seem high/low on the lowbie mobs.

 

• Aldriana

When people say things like "haste does not affect PPM", what they mean is that haste effects don't increase the number of observed procs; that is, absent other feedback mechanisms, if a rogue DPSs for an hour with no haste equipped he'll observe some number of procs; if he then equips 150 haste rating and DPSs for another hour, he will (statistically speaking) observe the same number of procs.

However, what this means for the proc rate on any given attack is that it's based on the current weapon speed rather than the base weapon speed; hence, the proc rate per attack changes with haste, whereas the proc rate per unit time does not.

I could be mistaken, but this is my understanding of what was determined.

 

• Latito

Went to DM, used a Searing Sunblade in my OH, no MH weapon. No haste so 1.3 speed attacks - 1.27 while mongoose'd. Only other proc was Hyjal Exalted ring, recheck proc rate on that if you want. After the sunblade broke I poped on Merc. Glad. Quickblade.. gear was the same otherwise. Since this was an offhand sword with swordspec, I was getting MH attacks occasionally - yes, I punched him a few times from SS procs. If you look at the dmg amounts, its always 185 w/o either proc up so its safe to say that its a MH punch, thus not able to get Mongoose. Just filter those attacks out, minor annoyance, sorry :P Arena partner logged on ~20% of the way into the Quickblade test, hence a much smaller data sample.

Oh.. this was all autoattack, no specials.

Also.. I saved the Grips of Damnation from sharding this week so I can put together 105 haste (belt, ring, gloves) if something needs testing with static haste. Not quite Daemona's 194.. but its something.


Free file hosting by Savefile.com
Free file hosting by Savefile.com

 

Borodin

Just wondering if it might be worth waiting for the 2.4 (?) changes to the UI and specifically the Combat Log which might make things a lot easier to analyse.

If it's 3.0 that those changes arrive then obviously that's a long way off and I shall shuttup.

 

◊ Malan

No the log changes are scheduled for 2.4.

Vyktianity already has over 75 billion followers. The first verse in his book "Gift of the Wild" is "In the beginning Vyk cast lifebloom and then maul."

 

Rezarel

When I took data from the Blasted Lands mobs, I wouldn't afk out until I left combat (when I died, or when someone else broke the crystal and killed my mob). I'm not sure if the Dire Maul mobs would actually put you in combat though.

 

Machinator

I copied a rogue to the PTR to test in DM. The ogres do put you in combat and you dont seem to DC, just go afk if you dont do anything. Im not sure how the combat log reacts to disconnects though. I am gonna let one run overnight and see what I get.

The reason for finding someone to stack haste is to run one test without haste, and then again with lots of haste to confirm if you actually get more procs in the same time period. So if you can do 2 tests like that we can possibly settle that. I would assume Slice and Dice, Flurry, and hasted swings from parries work the same way, but thats much harder to keep up for an entire test.

Test Results:
I dug out my comp science class stuff and wrote a crappy program to look through each line and look for procs and determine how long they lasted. I assumed that any proc longer than 15s was 2 procs and longer than 30s was 3 procs. I learned that because of how the combat log is affected by lag you need to give some extra time for procs. The first time I ran through I had as much double procs as singles because the time between a gain and fade was something like 15.177 seconds. So I counted singles as less than 15.250s and double as less than 30.500s. Hopefully I remember to upload the log later.

Results before weapon broke:
Hits: 5135 (2.6 speed weapon, no haste)
Time: 222.517 minutes
0-15s: 132 gains
15-30s: 53 gains
30+s: 9 gains
Total Procs: 265
PPM: ~1.19 PPM

Since this is longer than my other tests I would assume more accurate. I will keep running these while I am at work/sleeping. Anyone good with stats that can give a number how many hits or minutes it would take to be say 99% or 99.99% accurate?

EDIT: My test was for Executioner.

"Information is ammunition."

 

Machinator

Test 2 lasted twice as long before weapon broke, maybe I forgot to repair the first time.
Hits: 10056
Time: 435.76m
Procs: 467
PPM: ~1.07

Total for both tests: ~1.11 PPM

"Information is ammunition."

 

• Aldriana

As promised, I have written a simple command-line utility to compute proc rate data from combat logs, which, for the moment, I have named ProcCounter. You will need python to make it work, and you may find it helpful to have a working understanding of how command line tools typically work. If there is significant demand for it I might be persuaded to write a GUI for it, but for the moment, this should be enough to get started.

The tool makes the following assumptions about the input data:

1) The proc you are investigating is buff proc - that is, it leaves specific "gain" and "fade" messages in the combat log. This program will *not* work on instantaneous procs like combat potency or sword specialization.
2) The combat log comes from the tester. So if you bring a healer along to heal you while testing, you must use *your* combat log, not there's. The program works by looking for "You gain", "You hit", etc. type messages - not "Aldriana gains", "Aldriana hits", etc.
3) On a similar note: this only works on english-language combat logs. I don't know if it's even possible to log in different languages, but assuming for the moment it is, this program will not work on those logs.

Also, it assumes that your data is reasonably "clean", defined as follows:
1) You are only attacking with one weapon.
2) You are only autoattacking (no specials)
3) You have no Haste or attack-speed modifying effects active, other than the one being tested. So if you're testing Mongoose proc rate, you can't have your Dragonspine Trophy equipped, you should be specced out of flurry, you're not somehow getting SnD up, and so on.
4) Your data set is reasonably large. It will give some answers on smaller data sets, but to get reasonable accuracy you will need at least 1000 attacks or so, and more is better.

How to use ProcCounter
First, you will need to have Python installed. If you don't, you will need to go to python.org, download python, and install it.

Second, you will want to have ProcCounter in the same directory as the combat log(s) you wish to parse.

Third, open a command line prompt and navigate to that directory.

At this point, if you're familiar with command-line tools, you can probably just type 'python procs.py -h' and get enough information to get started. For those that would like a little more direction:

The general structure of a command for ProcCounter is as follows: python procs.py <options> [files]

There are two options that must be specified in each command line. First, you must specify the base speed of the weapon you are using. This is done with the command line option --speed (or -s for short), followed by the weapon speed. Hence, if you were testing with, for instance, Talon of Azshara, you would enter the option "-s 2.7" or "--speed 2.7"

You will also need to specify which buff you're looking for. There are two ways to do this. The first is with the option --buff (-b for short), followed by the exact name of the buff generated by the proc in quotes. For instance, if you are testing the proc rate of Band of the Eternal Champion, you would type in -b "Band of the Eternal Champion". If you misspell or abbreviate the buff name, it will not work. If you fail to capitalize, it will not work. For instance, none of the following commands will work:

* -b Band of the Eternal Champion
* -b "Eternal Champion"
* --buff "Band"
* --buff "band of the eternal champion"

Also note that you are entering the buff name; so, for instance, if you were testing Mongoose, you would want to type "Lightning Speed", not "Mongoose".

As a convenience for some of the more common procs to test, I have also added special command line options "--mongoose", "--executioner", and "--dragonspine" that you can use in place of the --buff option. The option --executioner is exactly the same as the option -b "Executioner".

Additionally, there are two more options available. The first, "--haste", allows you to enter the amount of haste rating you have equipped for the test. So, if you have a Band of Devastation on and no other haste equipped, you would enter --haste 31

Also, there is a --crits option (-c for short), which indicates that the proc can only occur on critical strikes, useful for testing, for instance, Tsunami Talisman.

After entering all appropriate options, enter one or more filenames to be parsed. You can put as many different filenames as you want. Note that all files to be parsed will be checked for the same buff, and assume the same weapon speed. So you can't mix data sets taken for two different buffs or two different speeds and expect to get correct answers.

So, for instance, if you had a test of Mongoose enchant on Talon of Azshara stored in a file called "mongoose.txt", you would enter the command line:

python procs.py --speed 2.7 --mongoose mongoose.txt

If you had 3 files of data, taken with Blade of Savagery with 31 haste rating equipped, you would use

python procs.py -s 1.4 -h 31 --mongoose file1.txt file2.txt file3.txt

If you were, for instance, testing the proc rate on Tsunami Talisman with Malchazeen instead, you would enter

python procs.py -s 1.8 --buff "Fury of the Crashing Waves" --crits file1.txt file2.txt file3.txt

When you enter an improper command line, you will get an error message which will hopefully lend some insight into what you did wrong.

If you enter a correct command, you will get a brief summary of information about the buff, including internal cooldown, uptime periods, eligible attacks, and some basic statistical analysis.

I haven't fully tested every combination of options, so there may be bugs; if so, please let me know and I will look into fixing them.

Edit: Fixed bug described by Machinator in post 18.

Attached Files

procs.rar (2.1 KB, 13 views)

 

Machinator

First, thanks for making that tool, now to nitpick.

I tried using this on one of my executioner tests(2.6 speed, autoattack, no haste).
Your program counts 194 uptime periods, which is what I count. But it says 3994 eligible attacks, I'm not sure what that means but there were 5135 hits in the file. Is this just something for haste procs?

I also had this come up at the end:


After running it through all the logs Ive been collecting I get
There is no output of time or total attacks which makes it hard to translate those numbers to anything PPM.

It also seems like a pretty big range for something like 38 hours of logs. Any idea how many attacks you need to narrow that down to around +- .1%?

"Information is ammunition."

 

• Latito

Just a note - unless I missed that option in the Python installation, you will need to manually add the python.exe location to your system Path variable.

To do this (Windows 2K / XP):
Right Click My Computer
Properties
Advanced tab
Environment Variables button
Under System Variables, find "Path"
If you found it
-select it and hit Edit
-add a semi-colon to the end of your existing variable value and then put in the python path (default was C:\Python25)
If you can't find Path
-hit New and enter the variable name as "Path"
-enter the variable value as the python path (default was C:\Python25).

You should end up with a variable name that might look like this:
%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;c:\cygwin\usr\bin;c:\cy gwin\bin;C:\Program Files\Java\jdk1.5.0_07\bin;C:\Program Files\ant\bin;C:\Python25

Note - you likely won't have the cygwin, java or ant stuff in there if you are reading this.

If you are using a different OS (Vista, Mac OSX, etc).. good luck!


Oh, and if you use the option -h (or --help) you will get a pretty handy printout of all the options Ald described above.

 

• Latito

Machinator - the error your describing is due to a typo in line 34, it should be "crits = False", not "crit = False". I PM'd Ald about it already, for now you can just manually change the procs.py file. Once you change that it will print out the PPM properly.

About the "smaller range"... get more data In the DPS Simulator I'm currently writing.. without a lot of proc-type stuff in there I still get variances of +/- 0.5 dps in a ~500 dps scenerio over 200+ hours of simulation. Thats just from basically weapon range, crit chance and a few talent stuff specific to rogue yellow-attack cycles.


Results from the data I posted several posts up:

 

• Aldriana

This has to do with my preferred method of measuring proc rates for refreshing procs. The problem with counting all attacks is that you don't really know how many procs actually occurred due to refreshing procs. My solution to this is based on the realization that the proc rate is also equal to the number of attacks outside of the buff uptime divided by the number of uptime periods; basically, you throw out all procs and attacks that occur while the buff is already active. So for procs that have no cooldown, "eligible" means "did not happen while the buff is already active". For procs with a cooldown, it means attacks that happened while not on cooldown.

Wow, that'll teach me to make last-minute edits before uploading. I'll update the link above with a fixed version in a few minutes.

This is related to error message you got. It's supposed to do PPM calculations for you, I just made a mistake in that section of code.

The issue with error ranges it that to narrow the answer by a factor of 2, you need 4 times as many attacks. So it takes a *lot* of attacks to get really