Armory Scraping - Page 2

• malthrin · 06/02/10, 12:32 PM

It's not the same message as the query throttle. You also run into the same error while browsing the web interface, so I'd wager it's a problem on their end.

edit: looks like it's mostly fixed this afternoon

Stack · 06/05/10, 10:30 PM

I was able to get the main summary page http://www.wowarmory.com/auctionhouse/index.xml#summary into my program as XML but has anyone found the JSON equivalent link, it would be a lot simpler if I could route all the responses from the armory thru JSON.

EDIT: Found it http://www.wowarmory.com/auctionhouse/status.json

 {"auctionStatus":{"factionId":0,"mailInfo":{"current":2,"max":50,"total":2,"fullPercent":4},"activeHighBids":0,"activeOutBids":0,"auctionsActive":0,"auctionsCancelled":2,"auctionsExpired":0,"auctionsSold":0,"bidsCancelled":0,"bidsWon":0,"goldEarned":0},"command":{"f":0,"cn":"charactor","r":"realm"},"pra":{"status":"active"}}

Another http://www.wowarmory.com/auctionhouse/cancel.json

{"success":true,"command":{"auc":901314668,"f":0,"cn":"charactor","r":"realm"},"pra":{"status":"active"}}

Another important one http://www.wowarmory.com/vault/chara...ct-submit.json

Takes a CN=charactor&r=realm as post data which returns.

{"success":true}

♦ Slackie · 06/06/10, 2:38 AM

Thanks for finding status.json, I hadn't seen that before. Does anyone know of a way to find current transactions remaining without actually posting a new auction or bidding on something? That would be useful.

Here are all the various URLs I know about, just so everyone is on the same page (I didn't add status.json from above yet)

root: "http://www.wowarmory.com/auctionhouse/"
status: "http://www.wowarmory.com/login-status.xml"

# Miscellaneous
money: "http://www.wowarmory.com/auctionhouse/money.json?%s"
alerts: "http://www.wowarmory.com/vault/calendar/alerts-user.json?%s"
inventory: "http://www.wowarmory.com/auctionhouse/inventory.json?%s"

# Calendar
calendar: "http://www.wowarmory.com/vault/calendar/month-user.json?%s"
calendar_world: "http://www.wowarmory.com/vault/calendar/month-world.json?%s"
calendar_details: "http://www.wowarmory.com/vault/calendar/detail.json?%s"

# Mailbox
mailbox: "http://www.wowarmory.com/auctionhouse/mail.json?%s"
mailbox_move: "http://www.wowarmory.com/auctionhouse/takeMail.json?%s"

# Auctionhouse related
auction_post: "http://www.wowarmory.com/auctionhouse/create.json?%s"
auction_browse: "http://www.wowarmory.com/auctionhouse/search.json?pageSize=50&rhtml=false&%s"
auction_active: "http://www.wowarmory.com/auctionhouse/auctions/index.xml?rhtml=false&%s"
auction_cancel: "http://www.wowarmory.com/auctionhouse/cancel.json?%s"
auction_deposit: "http://www.wowarmory.com/auctionhouse/deposit.json?%s"
auction_buy_get: "http://www.wowarmory.com/auctionhouse/bid.json?%s"
auction_buy_post: "http://www.wowarmory.com/auctionhouse/bid.json"

Stack · 06/06/10, 8:59 PM

I've been working with the takeMail.json all day. Based on what I've watched while using the wowamory.com site, ive concluded that you can only claim money remotely and can only see items in your mail box that you could possibly relist.

what is odd is that wowarmory doesn't supply a auc id when it does its call -9223372036854775808 is the value I get when I omit it in my post data, including it doesn't get you mail that is not just money however.

sample rejection

{
  "error": {
    "code": 112,
    "error": true,
    "message": "Mail not found"
  },
  "command": {
    "auc": -9223372036854775808,
    "mailIds": "2119124985",
    "f": 0,
    "cn": "Nullberri",
    "r": "Bonechewer"
  }
}

sample success

{
	"mailIds":[2120517468],
	"money":950000,
	"command":
		{
			"auc":-9223372036854775808,
			"mailIds":"2120517468",
			"f":0,
			"cn":"Nullberri",
			"r":"Bonechewer"
		},
	"pra":{"status":"active"}
}

Psy7th · 06/14/10, 8:27 AM

For what it's worth: -9223372036854775808 is really -(2^63) or signed long 0x8000 0000 0000 0000.

• malthrin · 07/25/10, 9:02 AM

Has anyone been getting logged out periodically since this week's maintenance?

♦ Slackie · 07/25/10, 9:59 AM

Yes. I had to modify my library to raise an error when I get the "You must log in." message so that scripts know they need to authenticate again.

• malthrin · 07/26/10, 2:42 PM

Yeah, I did the same. Kind of a pain for anyone trying to do this on an authenticator account, though.

dozens · 10/15/10, 11:10 AM

looks like I am little late to the party

The auction browsing is based off your primary toon, does anyone know the url to select a different primary toon?

Rinji · 11/22/10, 4:37 PM

To Login (w/o Authenticator):

Post to URL: https://us.battle.net/login/en/login.xml
Arguments:

accountName: <account name>
password: <password>

Afterwards, I do a get request to http://www.wowarmory.com/auctionhouse/index.xml, which should build the correct cookies. To validate a successful login, check the cookies for (perl regular expression):

/auction_sk=([\w-]*?);/

Note: I do a match in the cookies for the 'auction_sk' code, as it appears to be passing this param 'sk' around during POSTs, but currently it does not seem to be needed for a successful response.

To List all Characters:

Get URL request: http://www.wowarmory.com/vault/character-select.xml

XML response should be easily queried by xpath (ex.):

/page/characters/character/@name

Note: The first name listed (I believe) is the default character.

To Switch the Default Character:

Originally Posted by dozens

does anyone know the url to select a different primary toon?

Post to URL: http://www.wowarmory.com/vault/character-select-submit.json
Arguments:

cn: <character name>
r: <server>

Successful response:

{"success":true}

♦ Slackie · 12/05/10, 11:05 PM

Lost my last post in the ether, but it looks like Blizzard has moved over to the new Armory in prep for Cataclysm.

They have switched to using REST calls for using the Remote AH. Here are the new base search pages:

https://us.battle.net/wow/en/vault/character/auction/horde/
https://us.battle.net/wow/en/vault/character/auction/alliance/
https://us.battle.net/wow/en/vault/character/auction/neutral/

Browsing has also changed:

https://us.battle.net/wow/en/vault/character/auction/horde/browse?key=val&key2=val2 (same options at the end as before)

The old method of setting rhtml=n to get JSON/XML does not seem to produce JSON/XML anymore. This should probably be what people work on figuring out first.

I have not looked into the other methods for posting/canceling auctions yet, but they should be trivial to discover.

immense · 12/06/10, 12:49 AM

While trying to rework my scripts I found that the Id key no longer works to find items by their ID, for example:

https://us.battle.net/wow/en/vault/c...browse?id=4306 will NOT give you results for silk cloth

I'll keep digging and post updates to this thread.

immense · 12/06/10, 4:01 AM

You can search the new interface by item ID with the following string:

https://us.battle.net/wow/en/vault/c...se?itemId=4306

This would only return the results for Silk Cloth rather than getting results for items such as a bolt of silk cloth when using the text search.

If anyone has figured out what file has replaced the bid.json script please feel free to share =)

♦ Slackie · 12/06/10, 4:07 AM

This should get you started:

browse: GET https://us.battle.net/wow/en/vault/character/auction/horde/
bid_auction: POST https://us.battle.net/wow/en/vault/character/auction/horde/bid
cancel_auction: POST https://us.battle.net/wow/en/vault/character/auction/horde/cancel
create_auction: POST https://us.battle.net/wow/en/vault/character/auction/horde/createAuction
deposit (ticket request): POST https://us.battle.net/wow/en/vault/character/auction/horde/deposit
money: POST https://us.battle.net/wow/en/vault/character/auction/horde/money
my_auctions: GET https://us.battle.net/wow/en/vault/character/auction/horde/auctions
my_bids: GET https://us.battle.net/wow/en/vault/character/auction/horde/bids

Keep in mind that the "horde" portion of the URL is dynamic and can be "horde", "alliance", or "neutral".

So far I've found that the bid, cancel, create, deposit (ticket request) and money requests all return native JSON.

I still haven't found a way to get the browse, "my auctions", or "my bids" requests to return anything but HTML.

Also, upon looking at this I got the idea that since all of the POST requests returned JSON that maybe any POST request would, but sending POST instead of GET to the browse URL still results in HTML.

♦ Slackie · 12/06/10, 4:54 AM

From a PM:

In regards to https://us.battle.net/wow/en/vault/c...n/alliance/bid, what variables are you posting to bid / buyout an auction?

Is it still the Auction id and Money (in copper)?

Thanks

There is an additional key required now (other than the auction id (auc) and amount to bid (money)) that is "xtoken". The value of this key is taken from the cookie "xstoken" that is set during login.

These new "xtoken" and "xstoken" keys are used for several of the operations. There seems to be some confusion with them, for example in the POST request for bidding on an auction, the "xtoken" value is taken from the cookie "xstoken" (note the "s").

In the cancel operation, the "xtoken" value comes from the "xtoken" cookie, same with the create operation.

In the deposit operation when creating a new auction, there is also a new key "sk", which is derived from the "xstoken" cookie.