Elitist Jerks - View Single Post

• spawnstah · 01/23/08, 7:13 PM

My aim with this was not really to make life easier for Blizzard and reduce the load on the Armory.
The problem as I see it, is that lets say 10 sites are running 10 crawlers each downloading the data. Since the armory and the millions of characters are quite a lot of data, it takes these individual sites like 2-3 maybe even 4-5 weeks to get all the data they need.
Wowjutsu needs to have data that is no older than a few days to be counted as accurate and up to date.
Other sites working with class balance, realm population and so on might be okay with data that is 1 month old.
But if those 10x10 crawlers could all collect the info to a central database that they all could use, the data could in theory always be max 2-3 days old.

I can also see how instead of having those 10x10 crawlers running by the different sites, we could have hundreds of players running 1 crawler each and help with the updates.

Kalroth I understand your point, and yes Blizzard do have a central database which is useful if you use it like you do. But sites that need to access every character and guildpage to gather information kind of need to download all of those small .xml files, which takes time. Having a central database that you can access and get instant answers from, do queries to or download the database localy for faster queries.

I'm not sure how "open" the database would be, but I would like to leave it as open as possible so anyone who would be inerested could query it. But I guess it depends on the bandwidth used. Maybe give access to the database for a small fee, 10-50 usd per month/year whatever just to have some kind of control and make sure it does not go out of hands.
Or like wowwebstats does, give priority queries to the ones who pays. I dont know and I'm not sure its important at this point. =)

01/23/08, 7:13 PM	#7 (permalink)
• spawnstah Von Kaiser Stitch Tauren Warrior <Horizons> Genjuros (EU)	My aim with this was not really to make life easier for Blizzard and reduce the load on the Armory. The problem as I see it, is that lets say 10 sites are running 10 crawlers each downloading the data. Since the armory and the millions of characters are quite a lot of data, it takes these individual sites like 2-3 maybe even 4-5 weeks to get all the data they need. Wowjutsu needs to have data that is no older than a few days to be counted as accurate and up to date. Other sites working with class balance, realm population and so on might be okay with data that is 1 month old. But if those 10x10 crawlers could all collect the info to a central database that they all could use, the data could in theory always be max 2-3 days old. I can also see how instead of having those 10x10 crawlers running by the different sites, we could have hundreds of players running 1 crawler each and help with the updates. Kalroth I understand your point, and yes Blizzard do have a central database which is useful if you use it like you do. But sites that need to access every character and guildpage to gather information kind of need to download all of those small .xml files, which takes time. Having a central database that you can access and get instant answers from, do queries to or download the database localy for faster queries. I'm not sure how "open" the database would be, but I would like to leave it as open as possible so anyone who would be inerested could query it. But I guess it depends on the bandwidth used. Maybe give access to the database for a small fee, 10-50 usd per month/year whatever just to have some kind of control and make sure it does not go out of hands. Or like wowwebstats does, give priority queries to the ones who pays. I dont know and I'm not sure its important at this point. =)