WoW Simulator - What do you want? - Page 4

dedmonwakeen · 01/13/09, 3:32 PM

Originally Posted by Ullas

Send me a PM if you're interested in learning about this stuff, and how you can add it to simulationcraft. A lot of the work involved in using this data is finding out what is what. Nothing is labeled, so you have to figure out what each column of numbers means via induction, and I would have no problem sharing that info with you.

Thanks! I must admit I'm starting to get a bit burnt-out coding. I don't have sufficient time to both develop AND play..... and I'd like to get back to playing soon..... Heh.

But rest assured: I'll be keeping a close eye on this thread!

Good luck!

Ullas · 01/16/09, 1:53 PM

Alright, so I got the front end for setting up priorities working.

Each priority may have parameters, like "Keep health above: X%", X being the percentage of course. Also, each priority has a list of actions that pertain to it. In the example of keeping your health above X%, you have a list of Actions that will increase your health. The priority takes care of figuring out which one is best, based on a whole bunch of things.

You will note that as long as "Keep Debuffs Up: Blood Plague, Frost Fever" is above the "Damage: Do as much damage as possible", you shouldn't end up casting blood strike or obliterate without BOTH diseases being up. The code also knows that Obliterate is a better option than blood strike if there is a GCD + 1.5 seconds (hardcoded for now) left on either disease. I'll have to consult my fellow DKs to make sure this logic is sound.

I know we're familiar with figuring out an optimal rotation and sticking with it, but my hope is to base this software on nothing more than data from the game and general principles of each class. If the rotations we have accepted as "optimal" are indeed that, they should be what the program figures out anyway.

More to come next week!

kurokaze · 02/02/09, 4:18 AM

Wow. I've been working on a similar project, but it's much less far along, just in the basic planning and UI/data structure creation stages. You definitely look like you have something going here. I'm very interested in giving your project a look-through and probably helping out here instead of developing my own independent version. Unfortunately, I have nil experience with Java, but it shouldn't be too hard to pick up. Let me run through the general featureset that my project is/was intended to have and see if it meshes with your design goals:

- Multiple player, multiple mob simulator with eventual support for 2D positioning
- All actions, auras, properties etc. described as combinations of basic parts that can be broken down into a small set of simple actions or auras/properties
- Easy comparison of different game rule-sets (e.g. comparing patch 3.0.2 to patch 3.0.8)
- User editable (XML, in my case) item/spell/ability/etc. databases
- Three operation modes:
---Simulation with n iterations
---Realtime simulation with 1 iteration (with speed control) - mainly for fun/interest
---'spreadsheet mode' that provides 'formulation' with a non-random, deterministic answer that doesn't require thousands of iterations for near-accuracy. This necessitates some simplification; most notably, 2D positioning would make no sense and should/could not be used in this mode.

Now, this last thing could seem a bit difficult to pull off tied so closely to a simulator, but it's not too far out there. The basic methodology is that in 'spreadsheet mode', all auras and other bits of game state have a probability tied to them rather than being binary on or off. Health and such are averaged, which could lead to some fudginess, but this mode is intended to be used in simple situations (target dummy-like) where that kind of thing isn't a big deal. A simple example might be that the gamestate at one moment is that Killing Machine has a .5 probability to be up and Blood Plague has a .8 probability to be up, you have no RP, and you have all runes up and none death; the simulation would add in a Plague Strike with .2 probability, an Icy Touch with .4 probability, and a Blood Strike with .4 probability. Obviously, after the first few attacks things would get a bit more complicated, but it doesn't really become prohibitively hard to deal with. This mode may be unnecessary if the simulator is fast enough that it can provide a reasonably non-random result at Rawr-like speeds, but I haven't seen one that can.

- Priority-based rotations much like you described earlier. However, the last post's methodology ('do as much damage as possible') isn't going to be smart enough, nor will it be able to answer the question of "how much worse is X idea someone came up with than 'optimal'?" People have had trouble with Deathgraf locking them into nonsensical rotations, I've heard (I can't get the thing to load, myself), and duplicating that flaw is short-sighted. Furthermore, it doesn't quite fit the more complex rotations. Let's examine a 0/20/51 priority system, since that's what I know best:

PS if no BP > KM IT > UB if no UB > IT if no FF > BS if non-Death Blood rune > DC > IT > PS

Now, in trying to duplicate this with the simplified generic paradigms from your last post, we run into a few stumbling blocks. First, if we try to simply 'keep dots up', what priorities do we use within that? You can't just hard-code it without seriously damaging the user's ability to test rotations: for example, I'm not personally sure whether PS if no BP should come before or after KM IT, and I'd like to see the numbers for both. And what if I'm 10/20/41 so I don't have Rage of Rivendare? Same rotation in essence, but BP becomes much lower priority. The situation is murky; UB is the strongest DoT, BP carries a 10% damage increase, and FF has a large amount of instant damage that comes with it. Sure, you can figure out the right order of the three with some complex logic, but even so, it's going to be nearly impossible to take in the final factor: that ITs should be 'saved' for KM procs when possible, so have artificially lower priority when the KM buff is inactive and artificially higher priority when it is active.

Then the next stumbling block. We've covered the 'keep dots up' section. What about the 'do as much damage as possible' section? Well, KM IT is clearly attempting to do as much damage as possible, but the next thing on the list is BS, which is the second-least damaging ability used. You could hard-code some logic to give Death Rune-generating abilities some ghost damage so that they count as 'doing as much damage as possible', but that's a kludge, a band-aid that will keep you hard-coding in more and more exceptions for each similar situation. You either need a much smarter (prohibitively so) rotation generation system that creates the rotation using the logic that derived it, or you need the flexibility for the user to enter in what they want to test out. And really, while generating a programattically 'good' rotation could be a handy tool, it's much more important that the tool be flexible enough to meet users' needs. When comparing two rotations - for example, the Frost 2H Antiseptic and 6xIT rotations - it's not enough to be able to say that WoWsim's rotation generator generated 6xIT. You'd want to at least confirm that WoWsim gives higher actual DPS numbers when fed 6xIT than when fed Antiseptic. You probably would want to see how much the difference was and check a damage output breakdown to see where the inflection points are - after all, Antiseptic scales much better with weapon damage and ArPen, while 6xIT may give better returns from raw AP.

I should note that my primary goal is to develop an accurate, flexible, and extensible DK DPS simulator that supports custom priority systems because there's really no tool out there that does so. Upon beginning design I noticed that the flexible methods and data structures I was using could easily be extended to a much larger-scale project and the goals expanded from there, but my main objective and motivation is to have a tool similar to Rawr but with significantly more accuracy and complexity (for an example of how Rawr fails, at least Rawr.DPSDK, it doesn't even have simple rotation support beyond 'N skill1 and N skill2 in 20 seconds', so it would always miss things like if X happens over 12 seconds after a Plague Strike then it has no chance of getting the 5% Desecration buff.)

Ullas · 02/03/09, 10:30 AM

kurokaze,

Regarding your design goals

Originally Posted by kurokaze

- Multiple player, multiple mob simulator with eventual support for 2D positioning

This is an eventual goal of mine, but obviously it's more important to get simple things working before we attempt to tackle the multi player / multi mob problem. I'm keeping WoWSim as modular as possible, so 2D positioning support would probably just plug right in to the simulation module. Right now I have:

MODULES:
Netbeans - Takes care of a huge amount of boilerplate code especially relating to user interface.
WoWSim - Base (Requires SwingX, some other third party libraries) - Has some commonly used design patterns, base class, object manager class (i.e., a folder or group of objects), support for an object tree (see a screenshot on my google code page), support for localization, and some boilerplate code for dealing with modules.
WoWSim - Data (Requires WoWSim - Base, BerkeleyDB) - Handles retrieval of data from the World of Warcraft game client, represents the database of information found there as a tree, and allows the existence of multiple "Game Models" (spells, talents, etc...). Includes a WoWHead interface, which runs in the background finding items and adding them to a cache.
WoWSim - Common (Requires WoWSim - Data, WoWSim - Base) - Has the objects like "Character" and "Party" and "Raid"
WoWSim - Armory (Requires WoWSim - Common, WoWSim - Data) - Capabilities to pull data from armory.worldofwarcraft.com
WoWSim - Simulation (Requires WoWSim - Common, WoWSim - Data) - The multi-threaded solver. - NOT YET IN SVN REPOSITORY
WoWSim - Optimization (Requires WoWSim - Simulation) - Optimization framework for running many tests sequentially.
WoWSim - Results (Requires WoWSim - Simulation) - Postprocessing tools for analyzing data.

Originally Posted by kurokaze

- All actions, auras, properties etc. described as combinations of basic parts that can be broken down into a small set of simple actions or auras/properties
- Easy comparison of different game rule-sets (e.g. comparing patch 3.0.2 to patch 3.0.8)

This is what has been slowing me down so much, but I'm making progress. What I'm doing is converting this game data (pulled directly from the client) into a database file. When I get all of this working, I'm going to start with the earliest WoW version I have (1.04 maybe?) and copy the relevant MPQ file (locale-enUS.MPQ) from each version as I patch it up to 3.08. I would then make the database files available for download. From the user's point of view, getting the most current game data is as simple as picking where you want to store your database, and pointing the software to your WoW installation directory.

Originally Posted by kurokaze

- User editable (XML, in my case) item/spell/ability/etc. databases

These can be XML or CSV or whatever. I'm using BerkelyDB (an Oracle serverless open source database product). I originally tried storing all of this data textually, but there's a whole lot of it, and it ended up slowing things down a while lot (i.e., the "Spells" database has 38,003 entries, each with 220+ fields).

Originally Posted by kurokaze

- Three operation modes:
---Simulation with n iterations
---Realtime simulation with 1 iteration (with speed control) - mainly for fun/interest
---'spreadsheet mode' that provides 'formulation' with a non-random, deterministic answer that doesn't require thousands of iterations for near-accuracy. This necessitates some simplification; most notably, 2D positioning would make no sense and should/could not be used in this mode.

Now, this last thing could seem a bit difficult to pull off tied so closely to a simulator, but it's not too far out there. The basic methodology is that in 'spreadsheet mode', all auras and other bits of game state have a probability tied to them rather than being binary on or off. Health and such are averaged, which could lead to some fudginess, but this mode is intended to be used in simple situations (target dummy-like) where that kind of thing isn't a big deal. A simple example might be that the gamestate at one moment is that Killing Machine has a .5 probability to be up and Blood Plague has a .8 probability to be up, you have no RP, and you have all runes up and none death; the simulation would add in a Plague Strike with .2 probability, an Icy Touch with .4 probability, and a Blood Strike with .4 probability. Obviously, after the first few attacks things would get a bit more complicated, but it doesn't really become prohibitively hard to deal with. This mode may be unnecessary if the simulator is fast enough that it can provide a reasonably non-random result at Rawr-like speeds, but I haven't seen one that can.

I don't see value of a real time simulation with a speed slider. Spreadsheet mode could be more easily done using regression instead of simplification (i.e., run 200 simulations, and you can generate your spreadsheet using the data). I'm not to worried about speed issues, since it's essentially just a sequence of timed events, with each character being on its own thread. I'm very much trying to stay away with simplifying models, because subtle but relevant things often fall through the cracks. For example, in your above simplification, if a player has an "on-use" trinket and they only want it up when killing machine procs, you'd have to fudge the trinket's effect too in order to get the same result as you'd see in the game. If the trinket is not one easily modeled by a caster aura state (i.e., a stacking buff or something), things get really complicated really fast -- in fact more so than if we had just avoided simplifications to begin with! I know we're all used to spreadsheets, but it's really not a necessary component of an analysis tool like this.

Using regression for finding relationships between various quantities is favorable because only minimal effects are minimized, as opposed to things you anticipate to be minimal or things you can't simplify well.

Originally Posted by kurokaze

- Priority-based rotations much like you described earlier. However, the last post's methodology ('do as much damage as possible') isn't going to be smart enough, nor will it be able to answer the question of "how much worse is X idea someone came up with than 'optimal'?" People have had trouble with Deathgraf locking them into nonsensical rotations, I've heard (I can't get the thing to load, myself), and duplicating that flaw is short-sighted. Furthermore, it doesn't quite fit the more complex rotations. Let's examine a 0/20/51 priority system, since that's what I know best:

PS if no BP > KM IT > UB if no UB > IT if no FF > BS if non-Death Blood rune > DC > IT > PS

I don't accept any rotations, specs, etc... determined to be "optimal" on these forums to be true. I think it would be much slicker if the software spit them out confirming that they are in fact the best way to go, without adding them to the very short list of assumptions.

Originally Posted by kurokaze

Now, in trying to duplicate this with the simplified generic paradigms from your last post, we run into a few stumbling blocks. First, if we try to simply 'keep dots up', what priorities do we use within that? You can't just hard-code it without seriously damaging the user's ability to test rotations: for example, I'm not personally sure whether PS if no BP should come before or after KM IT, and I'd like to see the numbers for both. And what if I'm 10/20/41 so I don't have Rage of Rivendare? Same rotation in essence, but BP becomes much lower priority. The situation is murky; UB is the strongest DoT, BP carries a 10% damage increase, and FF has a large amount of instant damage that comes with it. Sure, you can figure out the right order of the three with some complex logic, but even so, it's going to be nearly impossible to take in the final factor: that ITs should be 'saved' for KM procs when possible, so have artificially lower priority when the KM buff is inactive and artificially higher priority when it is active.

I'm going to support forcing the character to stick to a certain rotation, but only as a stopgap for my priority system. The whole point is to mimic the user's decision making process, and some people think in terms of priorities, while others think in terms of a rotation.

In either case, you could easily optimize by changing the order of priorities (or attacks) between tests.

Finally, thanks to Java's reflection API, you will be able to write some custom java code if my models don't fit what you're looking for. This part of the simulator is completely compartmentalized, so as long as you provide some way to determine which action to take next (given your character's state, enemy's state, party's state, etc...), things will work properly. If you can come up with another model in addition to priorities and rotations that is of general interest (i.e., not specific to DW death knights), I will consider adding it with a nice UI and everything.

Originally Posted by kurokaze

Then the next stumbling block. We've covered the 'keep dots up' section. What about the 'do as much damage as possible' section? Well, KM IT is clearly attempting to do as much damage as possible, but the next thing on the list is BS, which is the second-least damaging ability used. You could hard-code some logic to give Death Rune-generating abilities some ghost damage so that they count as 'doing as much damage as possible', but that's a kludge, a band-aid that will keep you hard-coding in more and more exceptions for each similar situation. You either need a much smarter (prohibitively so) rotation generation system that creates the rotation using the logic that derived it, or you need the flexibility for the user to enter in what they want to test out. And really, while generating a programattically 'good' rotation could be a handy tool, it's much more important that the tool be flexible enough to meet users' needs. When comparing two rotations - for example, the Frost 2H Antiseptic and 6xIT rotations - it's not enough to be able to say that WoWsim's rotation generator generated 6xIT. You'd want to at least confirm that WoWsim gives higher actual DPS numbers when fed 6xIT than when fed Antiseptic. You probably would want to see how much the difference was and check a damage output breakdown to see where the inflection points are - after all, Antiseptic scales much better with weapon damage and ArPen, while 6xIT may give better returns from raw AP.

You'll be able to customize this via weights in the priority system, so ghost damage is not necessary. Things like damage done, cost of the spell, etc... all influence the "weight" of the spell (quantitatively, how "good" it is at that moment), and the user may add a multiplicative fudge factor.

Ullas · 02/06/09, 11:26 AM

A few new updates:

1) I can now grab almost all data available to the WoW Client. The "/World of Warcraft/Data/<LOCALE>/locale-<LOCALE>.mpq" file takes a while to parse, but once it's done for a given patch, it's done. You can share these already-parsed databases with others, and I intend to make them available for download. However, if I decide to stop supporting this software in the future, anyone can make their own with two clicks, providing you know where your WoW installation directory is.

1b) Unfortunately, the data files have no column headers, so I/we have to figure out what each piece of data means. This may change from patch to patch, so I have externalized the (Column, Label, Data Type) information for each table -- meaning it's an XML file that you can update as needed. These can also be easily shared, if I ever do decide to stop providing them myself. I will probably stop working on making these files better once the obvious/necessary pieces are figured out, but they can be continually improved as people discover what various numbers mean. I call these XML files "DBC Descriptors", and they take the following form:

SpellDispelType.xml:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
<properties>
<entry key="Id">0~~~class java.lang.Integer</entry>
<entry key="Name">1~~~class java.lang.String</entry>
</properties>

The line

<entry key="Id">0~~~class java.lang.Integer</entry>

Says that in the table "SpellDispelType", column 0 is named "Id", and holds data of type java.lang.Integer (the alternatives being java.lang.String or java.lang.Float), and column 1 is named "Name", and holds data of type java.lang.String.

If a column in the table has no information in this descriptor (everything except column 0 and 1in this case), it's treated as "unknown". I haven't decided what this means yet, as far as retrieving and using the data.

2) I have finished the API for using online data sources. So far I have only fully implemented the WoWHead data source, but I have designed it to be easily extended for other sites like Armory. An example of the power of a tool like this is the WoWHead crawling tool, which runs in the background when activated, grabbing data for items that it doesn't have cached yet.

Attached is a screenshot showing these new features in action. Note that there would be many more tables available in the database, but I have to write about 5 lines of code for each of them before WoWSim will load them (basically, making an object to hold the data, tell it where to look for the data in the MPQ archive, what's the database's key, value, etc...)

Also it seems like I'm using a lot of memory, but if you were to click on that graph (the one showing memory consumption), some cached data is dumped (sorted versions of all the tables, for one).

How you can help: If you're interested in dissecting World of Warcraft, and helping my project in the process, let me know and I'll give you the information you need to work on one of these tables. Here's a good example of an interesting one:

SpellShapeshiftForms Table:

[0]: 1, 1, Cat Form, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 760, 1, 1534, 1000, 892, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[1]: 2, 2, Tree of Life Form, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 209, 4, 0, 0, 864, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[2]: 3, 0, Travel Form, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 216, 1, 0, 0, 918, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[3]: 4, 0, Aquatic Form, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 216, 1, 0, 0, 2428, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[4]: 5, 3, Bear Form, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 728, 1, 496, 2500, 2281, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[5]: 6, 0, Ambient, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 0, 0, 0, 0, 328, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[6]: 7, 0, Ghoul, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 3544, 6, 0, 0, 25527, 0, 0, 0, 6603, 47468, 47481, 47482, 47480, 0, 47484, 47496
[7]: 8, 3, Dire Bear Form, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 728, 1, 496, 2500, 2281, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[8]: 9, 0, Steve's Ghoul, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16711934, 128, -1, 0, 0, 24994, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[9]: 10, 0, Tharon'ja Skeleton, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16711934, 10, 6, 0, 0, 9784, 0, 0, 0, 50799, 49613, 49609, 49617, 0, 0, 0, 0
[10]: 11, 0, REUSE, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16711918, 1118, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[11]: 12, 0, REUSE, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16711918, 0, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[12]: 13, 0, REUSE, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16711918, 0, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[13]: 14, 0, Creature - Bear, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 0, 1, 0, 0, 2281, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[14]: 15, 0, Creature - Cat, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 0, 1, 0, 0, 892, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[15]: 16, 0, Ghost Wolf, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 216, 1, 0, 0, 4613, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[16]: 17, 1, Battle Stance, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 7, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[17]: 18, 2, Defensive Stance, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[18]: 19, 3, Berserker Stance, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[19]: 20, 0, Test, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 0, -1, 0, 0, 0, 0, 0, 0, 6603, 20577, 0, 0, 0, 0, 0, 0
[20]: 21, 0, Zombie, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 3094, 6, 0, 0, 26942, 0, 0, 0, 43949, 51230, 56560, 0, 56528, 0, 0, 0
[21]: 22, 0, Demon Form, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 3288, 3, 0, 0, 25277, 0, 0, 0, 54767, 50587, 50581, 54785, 50589, 50577, 0, 0
[22]: 23, 0, , 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712188, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[23]: 24, 0, , 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712188, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[24]: 25, 0, Undead, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 204, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[25]: 26, 0, , 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712188, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[26]: 27, 0, Flight Form, Epic, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 136, 1, 0, 0, 21243, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[27]: 28, 1, Shadowform, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 9, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[28]: 29, 0, Flight Form, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 136, 1, 0, 0, 20857, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[29]: 30, 1, Stealth, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[30]: 31, 4, Moonkin Form, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 705, -1, 0, 0, 15374, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
[31]: 32, 0, Spirit of Redemption, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16712190, 256, -1, 0, 0, 16031, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0

If you look for the entry "2500" in the fourth row (Bear form), and the same column of the top row (Cat form), you'll see "1000"
So,
Bear Form: 2500
Cat Form: 1000
Dire Bear Form: 2500
This is most likely attack speed while in the form, in milliseconds, if the form's attack speed overrides the attack speed of your weapon.

Some of the larger numbers are actually bitmasks, meaning it's the some of several powers of two (i.e., 2^3 + 2^7 + 2^16), where each power of two means something. There's a "Can/Can't be cast in combat" flag in the spell table, for example. These are some of the harder ones to figure out.

greesla · 02/06/09, 4:17 PM

Great project - I've recently been playing with simulationcraft, so I appreciate any effort making a user-friendly UI for a simulation application ;-)

I'm a java developer by profession with 6+ years of Swing experience. The project I'm currently involved with at work is a symbolic simulation engine for engineering - I'm involved in the rendering engine for 3d simulation results for mechanical/multi-body systems. I've been playing Warcraft since vanilla WoW, though I didn't get any raid experience until TBC. I've tanked, healed and dps-ed, but for now I'm only playing my shadow priest for raiding. I'm new to theorycrafting and have limited knowledge of game mechanics (still trying to soak up all the masses of info here at EJ), but I'd be glad to help in any way that I can. I'm also working on a small personal project to create a tool for evaluating SP raiding gear (a modest best-raid-gear-available tool specifically for Shadow Priests).

Ullas · 02/23/09, 12:18 AM

Two things:

1) I'm having some difficulty figuring out how to read the itemcache.wdb file. It used to be similar to the game database files (spells, instances, icons, etc...), where essentially you have a table with every row having the same number of values. As of WoW client 3.02, the rows in itemcache.wdb don't seem to have a set number of rows.

Is there some mod that I could use to make an easily parsable file containing:

1) Information pertaining to all the items my character has seen
2) The contents of my bank
3) Any other information relevant to the character, which is not available in armory (i.e., info on quests, currency, guild notes, etc..)

2) What types of searches would people be interested in using armory for? For example, I know a common one might be get all characters in guild X on realm Y so you can use them in simulations. Another might be get all items for X slot that drop in instances A, B or C. Would people want to do more complicated things, like finding characters that resemble their own gear, spec, etc...? Do people find the "find an upgrade" for an item useful?

InternetFett · 02/23/09, 3:41 PM

Originally Posted by Ullas

Would people want to do more complicated things, like finding characters that resemble their own gear, spec, etc...? Do people find the "find an upgrade" for an item useful?

I don't think something like searching players would be possible, considering the number of armory requests involved.
How to get XML with PHP from armory

As far as 'find an upgrade' goes, I think that would be a very good feature, although I think using customizable stat weights (like the ones built into Rawr or on WoWHead) would be more useful than pulling from the Armory.

Ullas · 02/23/09, 5:26 PM

Originally Posted by InternetFett

I don't think something like searching players would be possible, considering the number of armory requests involved.
How to get XML with PHP from armory

There's "easy" and there's "possible". It is already looking like I'm going to have to have a database on wow-sim.com for information regarding proc rates, some of the more unique items (i.e., what's the equivalent dps of a particular summoned guardian like a peasant of barov), and so on. Storing a very abbreviated version of each character would not be too difficult.

Character Name
Character Realm
Talents (the string like 0000000000000001422352342342300000000000000 created in the official WoW talent calculator)
Equipped Items (ids only)

This isn't a whole lot of data to store, especially considering I just need to have a good sample for each talent/spec/gear level combination. The armory XML already has a "last modified" timestamp, so that makes discarding old data fairly easy. As for getting data, I can just pick one "active" character on each realm, and then get his guildies, get their arena team mates, get their guildies, their team mates, etc... It's not perfect but it's still nearly as useful as if I could "advanced search" characters (some similar tool to the advanced item search).

One thing that's missing is a way to differentiate geared from not-geared players. Maybe I could store the average iLevel of all equipped items?

As for the armory "find an upgrade" feature, I was tempted to make a UI for it since I get it for free (I already have a parser for most/all of the armory xml pages), but it's a poor substitute for making an upgrade suggestion based on the differential performance increase per increase in each stat (where the differential is found by running a simulation, increasing the stat by some small amount, running it again).

InternetFett · 02/24/09, 11:13 PM

Originally Posted by Ullas

Storing a very abbreviated version of each character would not be too difficult.

Well, that certainly changes things.

Originally Posted by Ullas

One thing that's missing is a way to differentiate geared from not-geared players. Maybe I could store the average iLevel of all equipped items?

That sounds like as good a measure as any, although you may also want to take resilience into account as well. I wouldn't want to be looking at a pvp-geared character regardless of how similar they are otherwise. Also, in regards to talents, what about a feature that reports something like "X% of similarly geared players have Y talent that you do not"?

Originally Posted by Ullas

As for the armory "find an upgrade" feature, I was tempted to make a UI for it since I get it for free

The armory's feature seems to work pretty well. The problem I have with it though is simply that it reports an arbitrary 'relevance' rather than an actual measure of dps gained. I would say all or nothing with this feature, either run the simulation or leave it out.

Redbeard · 02/27/09, 3:20 AM

Armory data fetching is fine but doesn't solve the problem of people switching their gear for Pvp/Solo/Farm/Whatever purpose. With a 25-man group this problem escalates even more. What would be required for simulations purpose would be an addon to go in combination with the simulator. It would need to inspect and store the whole raids gear setups and talent distributions mid-raid. Perhaps even gear and talent switches mid-raid?

When dual-specs enter the picture I fully expect Blizzard will enable viewing both specs via Armory but then there's the problem of which spec to use in simulation. If those specs are PVE/PVE, selecting based on Armory data becomes even more problematic.

InternetFett · 03/01/09, 6:02 PM

I definitely agree with Redbeard that Armory data is going to be a pain to use. It's too bad there's not a website like CTProfiles around anymore. I like the idea of using an addon to pull from in-game. Maybe even just pulling data from specific characters with problematic armory data to compliment what the sim pulls.

Ullas · 03/02/09, 11:07 AM

Originally Posted by Redbeard

Armory data fetching is fine but doesn't solve the problem of people switching their gear for Pvp/Solo/Farm/Whatever purpose. With a 25-man group this problem escalates even more. What would be required for simulations purpose would be an addon to go in combination with the simulator. It would need to inspect and store the whole raids gear setups and talent distributions mid-raid. Perhaps even gear and talent switches mid-raid?

When dual-specs enter the picture I fully expect Blizzard will enable viewing both specs via Armory but then there's the problem of which spec to use in simulation. If those specs are PVE/PVE, selecting based on Armory data becomes even more problematic.

I'm aware of these issues, and am addressing them in the following way:

There's not much I can do to get the 'real' raid gear for all characters in a raid. What I have now is a collection of gear sets owned by each character, a pool of other other items owned by the character, and a pool of items attainable by the character (things he has access to via rep, raiding, arena points, arena rating, etc..., but has not yet obtained.).

Certain game data can be "Updated", as long as it has a "Data source" associated with the object's type. For example, armory is a data source for characters (basic info, reputations and talents can each be updated independently of updating the whole character), guilds, arena teams and items. Allakhazam and Wowhead are data sources for items. You would associate a particular object with a data source, in effect telling WoWSim where to put the data. For example, if you have 7 specs you're experimenting with, exactly one of them would be associated with Armory. When you "Update" your character, that one spec may change, but nothing else will. I have pretty much finalized the API for registering a new data source with WoWSim (details here). If someone were to write a new data source for a gear set, their code would just receive the CharacterGearSet object that needs to be updated, and they would be free to appropriately change things. One possibility would be detecting how different a spec is (maybe this means X% or more talent points have changed), and asking the user whether the "armory talent spec" should be updated, or a new spec should be created. This would, in theory, allow a PvP build to not override a PvE build.

The reason I'm not sweating this armory issue too much, is because it's extremely easy to set up talents and gear sets, especially when you've been playing for a while. When you look at the alternatives like Simulationcraft, even just pulling data from armory, in addition to a typical talent tree creation panel, is a huge step forward. My goal is to make things about as easy as manually setting your character up in CTProfiles was (minus the annoying wait every time you added an item, changed a gem, etc...) and leave it at that.

Also, I hope you guys realize you can tweak whatever data you get from Armory. If you logged out in a white wedding dress, you still have the ability to replace it with something a little more practical. Even if your data is only 70% right, I call that a success, because it's a better starting point than being naked with 0/0/0.

I believe blizzard will have its own version of ItemRack in patch 3.1. If this data ever makes it to Armory, that'll make the dual-spec thing quite a bit easier.

If you guys have a better idea, I'm all ears, but I would rather get everything working at an acceptable level before investing too much time refining a particular thing to perfection.

Ullas · 03/03/09, 12:28 PM

I have a 10 second favor to ask:

I'm building an XML feed that provides base stats for any Class+Race+Level combo.

Could you please add data to the set by entering your character or guild's name, along with your realm here:
http://feeds.wow-sim.com/baseStats.php

Note: I have hardcoded the U.S. armory URL in my php code, and thus only U.S. guilds will be properly read.

Depending on how many people do this at once, my web server may earn its self a 12 hour IP ban from armory. If you get an error, this is probably why, so don't worry about it.

Thanks!

Ullas · 03/04/09, 11:17 AM

I'm having some problems with making my stat calculations agree with armory.

Base Health:
I am trying to establish whether the following is true:

Mh: Health multiplier (i.e., Frost Presence, etc..)
Ah: Added health (i.e., [Battlemaster's Audacity] )

Ms: Stamina multiplier (i.e., Improved Blessing of Kings, Heart of the Wild, etc... )
As: Added stamina (i.e., Prayer of Fortitude, Items with stamina on them, etc... )

TotalHealth = Mh * (BaseHealth + HealthFromStamina + Ah)

HealthFromStamina = 10*(TotalStamina - 20) + 20

TotalStamina = Ms * (BaseStamina + As)

BaseHealth and BaseStamina should be numbers that are the same for all players of the same class, race and level. However, looking at two characters of the same class, race and level (Paladin, Human, 80)

Excerpt from The World of Warcraft Armory - Strydaris of Azjol-Nerub

<characterBars>
   <health effective="29869"/>
</characterBars>
<baseStats>
   <stamina base="220" effective="2284" health="22660" petBonus="-1"/>
</baseStats>

Excerpt from The World of Warcraft Armory - Troggen of Azjol-Nerub

<characterBars>
   <health effective="16314"/>
</characterBars>
<baseStats>
   <stamina base="143" effective="956" health="9380" petBonus="-1"/>
</baseStats>

(1) Why is Strydaris' "base stamina" 220, while Troggen's is only 143?

Even setting this aside for a moment..
Strydaris' Base Health: 29869 - 22660 = 7209
Troggen's Base Health: 16314 - 9380 = 6934

Strydaris has the talents Sacred Duty and Combat Expertise, so depending on whether these add or multiply, this would result in Ms = 1.14 or Ms = 1.1448, but we have supposedly already subtracted all health from stamina. I can't explain why Strydaris' base health is 3.966% (275) higher than Troggen's.

Anyone have any ideas as to what's going on here?

EDIT:
I never realized that mining now adds a passive stamina buff. Taking into account that Strydaris is a 450 miner (+50 stamina), his base stamina is
(143 + 50) * 1.14 = 220.

EDIT:
And I neglected to account for Strydaris' +275 health to chest enchant. Everything adds up now. This model for health works.