How to get XML with PHP from armory

zork · 07/05/07, 10:03 AM

The following is a bit special and probably for web-developers only.

It took me a while until I figured out how to get the XML data from the armory-item-search with PHP / AJAX without having it rendered into HTML.

Here is the problem:
When viewing the source code from the armory with your browser you will see all the XML data, but when you are trying to get it with a PHP script you will have it rendered to HTML which you would have to rip apart again with "preg_replace" and stuff.

What must be done is to let your PHP script tell the armory it is a browser.

Here is the original armory site I want XML data from:
http://surl.se/bsrf (sorry the url has some special characters so I had to use surl.se)

Here is my example:
http://dm.next-gen.org/files/search_xml_example.php

As you see in the source its all XML.

Here is the PHP source code that does the job:
http://dm.next-gen.org/files/search_xml_example.html

The important line is:

$send .= "User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)\r\n";

With this line you will tell the armory that you are using Internet Explorer 6 and Windows XP.

If you comment it out you will get this result:
http://dm.next-gen.org/files/search_html_example.php

This can have its use if the HTML is only a little snippet (like item-tooltips) but not if it is a complete page.

Another important part is the language. You can control it the same way you control the XML output. Just add one line and tell the armory the language you want to have:

For German

$send .= "Accept-Language: de-de\r\n";

For English

$send .= "Accept-Language: en-en\r\n";

For France

$send .= "Accept-Language: fr-fr\r\n";

Example:
http://dm.next-gen.org/files/search_xml_example_fr.php
Source:
http://dm.next-gen.org/files/search_xml_example_fr.html

Ok, what can be done with this?

Well you can do a lot of stuff. Since you now have all the XML to play with. You can have list boxes that show all items that will match specific criterias and stuff like that.

• Goggles · 07/05/07, 10:14 AM

Thanks for this. I had originally intended to do some playing with data from the Armory but PHP (and web development in general) is not something I have much experience with so I never got sufficiently motivated to do it. I think this will give me the impetus to have another go as it means I don't need to spend ages working out how to get the data and concentrate on the manipulation of it.

zork · 07/05/07, 10:38 AM

Just to give you some quick example what can be done:

http://dm.next-gen.org/files/item_to...980&lang=fr-fr
http://dm.next-gen.org/files/item_to...980&lang=de-de
http://dm.next-gen.org/files/item_to...980&lang=en-en

Source:
http://dm.next-gen.org/files/item_tooltip_xml.html

So you could have tooltips for every item id you want in multiple languages with just one little script that you could call with AJAX in real time.

If you want to lookup for items per "name", no problem. I want every item with "onslaught" in it. (beware of browser language!)
http://armory.worldofwarcraft.com/se...uery=Onslaught

Ok now with the php script:
http://dm.next-gen.org/files/item_qu...en&q=onslaught
http://dm.next-gen.org/files/item_qu...de&q=onslaught
http://dm.next-gen.org/files/item_qu...fr&q=onslaught

Just have the query string you are searching for behind the "q=".

Source:
http://dm.next-gen.org/files/item_query_xml.html

Thats it.

sp00n · 07/05/07, 12:56 PM

I admit I haven't done any XML parsing yet, but did you take a look at http://www.php.net/manual/en/ref.xml.php?

You should be able to read XML files without having to convert them.

BTW, those links don't work for me:
http://dm.next-gen.org/files/item_qu...en&q=onslaught

Pomperipossa · 07/05/07, 2:24 PM

Originally Posted by sp00n

BTW, those links don't work for me:
http://dm.next-gen.org/files/item_qu...en&q=onslaught

Like in you can't connect or you just get a blank page? If it's the latter you need to look at the source of the page: View->Source code or something like that.

Asmik · 07/05/07, 3:32 PM

I'm getting the following error, but it's probably because I don't know what I'm doing.

Renew · 07/05/07, 4:26 PM

http://phparmory.sourceforge.net/ or http://sourceforge.net/projects/phparmory/ is a parse I found around the net that puts everything into php arrays. It's pretty neat and easy to add in your own code (if you know a bit of PHP). I added in some code to get Arena Teams for use on my guilds under dev new layout and it works nice.

D1g1talS0ul · 07/05/07, 5:35 PM

I use cURL to pull down the xml and simplexml to parse it for my guilds roster page. Requires PHP5, which most webhosts haven't switched to (including our host). I run the script below on my desktop that has PHP5, it parses the info I need and puts it into a mysql database at our host. Then on our hosts server with PHP4, roster.php parses the data from the mysql database.


			
#!/usr/bin/php

<?php



class roster_gen {



 const BROWSER="Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.2) Gecko/20070319 Firefox/2.0.0.3";



 public $query;

 public $server;

 public $guild;

 public $guildie;

 public $page;



public function __construct ( $query, $server, $guild, $guildie, $page ) {

    $this->query = $query;

    $this->server = $server;

    $this->guild = $guild;

    $this->guildie = $guildie;

    $this->page = $page;

 } // end of __construct()



public function pull_xml() {



    if( $this->query === 'roster' ){

        $url = 'http://armory.worldofwarcraft.com/guild-info.xml?r=' . $this->server . '&n=' . $this->guild . '&p=' . $this->page;

        

      }elseif( $this->query === 'character' ){

        $url = 'http://armory.worldofwarcraft.com/character-sheet.xml?r=' . $this->server . '&n=' . $this->guildie;

        

        }



    $ch = curl_init();

    curl_setopt ($ch, CURLOPT_URL, $url);

    curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);

    curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, 15);

    curl_setopt ($ch, CURLOPT_USERAGENT,  roster_gen::BROWSER);

    

    $url_string = curl_exec($ch);

    return simplexml_load_string($url_string);

    

    curl_close($ch);

 } // end of pull_xml()



} // end class



// query, realm, guild, guildie, page

// query can be roster or character

$roster = new roster_gen (roster, cenarius, insurrection, NULL, 1);



$xml_roster = $roster->pull_xml();

$battlegroup = $xml_roster->guildInfo->guild['battleGroup'];

$num_guildies = $xml_roster->guildInfo->guild->members['memberCount'];



$roster->query=character;

$name = 'someguildie';  //change to character you'd like to lookup

$roster->guildie=$name;

$xml_char = $roster->pull_xml();



$tree1 = $xml_char->characterInfo->characterTab->talentSpec['treeOne'];

$tree2 = $xml_char->characterInfo->characterTab->talentSpec['treeTwo'];

$tree3 = $xml_char->characterInfo->characterTab->talentSpec['treeThree'];

            

$lastLOGIN =  $xml_char->characterInfo->character['lastModified'];

            

$title =  $xml_char->characterInfo->character['title'];



echo "Guild:$roster->guild Realm:$roster->server Battlegroup:$battlegroup\n";

echo "Name:$name($title), Talents:$tree1/$tree2/$tree3 Last Login: $lastLOGIN\n";

?>

If you do a var_dump($xml_char) or var_dump($xml_roster) you'll see all the options you can pull from the armory.

Hope this is helpful.

Ragnor · 07/05/07, 9:44 PM

I was mucking around testing with Firefox, IE7 and Safari for Windows today and I happened to notice Safari is awesome for using the armory (mostly). Firefox and IE7 will choke/lock up on the large xmlhttprequests and not respond until they have finished loading/processing the page. Safari however will let you carry on no problems, opening new tabs etc while the armory is loading in the background tab.

It's a pain Safari doesn't render text white in input boxes though

sp00n · 07/06/07, 4:09 AM

Originally Posted by Pomperipossa

Like in you can't connect or you just get a blank page? If it's the latter you need to look at the source of the page: View->Source code or something like that.

Huh, yeah.
Weird, normally Firefox displays the XML data and doesn't just put out a blank page.

zork · 07/06/07, 4:44 AM

Originally Posted by Asmik

I'm getting the following error, but it's probably because i don't know what I'm doing.

The XML data is not for viewing in your browser

, its for your php scripts.
So it doesn't matter that you do not see nothing.

Just look into the source to see the XML.

The Armory XML is very very fast, what makes the application so slow is all the AJAX crap. It just eats the browser alive >_<.

@renew
Thanks for the link I will take a closer look at this one. I am now at the point where I have to transfer the XML data into PHP arrays.

Melnor · 07/06/07, 10:34 AM

Thanks for the insight zork. I'll begin the c# asp.net port shortly. There's probably already one out there but I'm bored.

• christide · 07/06/07, 2:01 PM

Originally Posted by Melnor

Thanks for the insight zork. I'll begin the c# asp.net port shortly. There's probably already one out there but I'm bored.

static HttpWebRequest RequestBuilder(string requestUri, string currentPage, string referer, string address)
	{
		Cookie cookieOne = new Cookie("currentPage", currentPage);
		Cookie cookieTwo = new Cookie("cookies", "true");
		Cookie cookieThree = new Cookie("cookieMenu", "all");

		cookieOne.Domain = address;
		cookieTwo.Domain = address;
		cookieThree.Domain = address;

		CookieContainer thisContainer = new CookieContainer();

		thisContainer.Add(cookieThree);
		thisContainer.Add(cookieTwo);
		thisContainer.Add(cookieOne);

		HttpWebRequest request = (HttpWebRequest)WebRequest.Create(requestUri);

		request.Accept = "*/*";
		request.Headers.Add(HttpRequestHeader.AcceptLanguage, "en-us");
		request.Headers.Add("US-CPU", "x86");
		request.UserAgent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506; .NET CLR 1.1.4322)";
		request.Referer = referer;
		request.CookieContainer = thisContainer;

		return request;
	}

The ugly hacked up function I wrote a while back to form a request object with the required parameters to get the xml back.

I would call it like:

HttpWebRequest request = RequestBuilder("http://" + address + "/guild-info.xml?r=" + serverName + "&p=1&n=" + guildName + "&lang=", @"guild-info.xml%3Fr%3D" + serverName + "%26n%3D" + guildName + "%26p%3D1", @"http://" + address + "/#guild-info.xml?r=" + serverName + "&n=" + guildName + "&p=1", address);

Used for http://www.thepalsforlife.com/ftp/ArmoryParse/. If you're offended by the word faggot, that link will most likely be extremely offensive.

EDIT: I attached the source to my armory parse. It's pretty ugly and completely devoid of comments since it's something I hacked up in an afternoon and never really bothered with again. Hopefully someone can get some value out of the source however.

zork · 07/10/07, 6:26 AM

I just want to give you an update on this to show what amazing stuff can be done.

I built our own item search based on the Armory XML, take a look

All data is generated in real time, no database is used.

p.s. I couldn't use the phparmory XML parser, sadly it had a bug with XML attributes. But i found another one that worked properly and nearly the same.

Coeus · 07/10/07, 12:56 PM

Originally Posted by zork

p.s. I couldn't use the phparmory XML parser, sadly it had a bug with XML attributes. But i found another one that worked properly and nearly the same.

What did you end up using? I was starting to use phpArmory, but if it's bugged, I should either look at fixing it or just change to something else.

Kallisti · 07/10/07, 2:14 PM

I can just recommend antiarcs armory scripts as basis.

http://wow.tachyonsix.com/armory/
http://svn.digitalsentience.com/svn/armory/

Especially:
http://svn.digitalsentience.com/svn/armory/_armory.php

You can remove some code (which is only used for some special functions of his scripts) and edit it to fit your needs, but I found it to be perfect to build some websites quickly.

I'm using some lines of it in the script which generates our memberlist (http://pernoctem.de/members.php) twice a day automatically via cron jobs.

zork · 07/11/07, 5:58 AM

@Coeus
phpArmory bugged for me so I switched to another parser.

@Kallisti
I don't know if the scripts from antiarc require php5.0 which some web-server don't have. (simplexml is 5.0 only)

Hung_war3 · 09/24/07, 4:42 AM

Originally Posted by D1g1talS0ul

I use cURL to pull down the xml and simplexml to parse it for my guilds roster page. Requires PHP5, which most webhosts haven't switched to (including our host). I run the script below on my desktop that has PHP5, it parses the info I need and puts it into a mysql database at