[Theories and Math] The New Arena Rating System - Page 4

pqueiro · 01/29/09, 9:20 AM

Clarifications by Slorkuz(Blue Poster, EU Forums) (<- that's a link in case it's hard to see)

Originally Posted by Slorkuz

Due to there being some confusion regarding the ratings, we wanted to clarify a little:

Team Rating
Team Rating is the primary measure of a team's skill during the course of the team's existence. Titles and arena points are based on the team rating.

Personal Rating
The intention of this rating is only to make sure that a player has played enough games with their team to deserve the benefits of the team's standing and rating. When a player's personal rating gets close enough (within a certain range) of the team rating, it is considered likely that the player deserves the benefits of the team.

Matchmaking Rating (Hidden Rating)
The purpose of this rating is to ensure quality match-ups for a team regardless of players changing teams or resetting their team rating. Additionally, the theoretical mathematical foundations built into the matchmaking system allows for this rating to adapt and change much faster than team rating and personal rating; resulting in making the system match teams more correctly.

We should note that in our measurements thus far, this rating has been quite accurate in predicting the outcomes of future matches.

--------

That being said, we feel it is important to note that currently the personal rating of a player converges towards their matchmaking (hidden) rating rather than the team rating. We realize this is a source for much confusion and frustration with the players and will be changing the way it works a little: Instead of having the personal rating of each player converge towards their matchmaking (hidden) rating, it will instead be converging towards the team rating. We believe this change will help alleviate a lot of the current concerns and frustrations players are reporting.

Something that we wanted to take a minute to clarify is the fact that the matchmaking (hidden) rating is solely based on the history of wins/losses for your team(s). It does not factor in your gear, damage or healing done or any of the other speculations and rumours flowing about these forums.

Another clarification to an issue which is causing some confusion is the fact that a team's members can (and often does) have different personal ratings from one another as well as differing from the team rating. The reason for this is the following: A team's rating currently converges towards the average of that team's matchmaking (hidden) ratings while a player’s personal rating currently converges towards his individual matchmaking (hidden) rating. As mentioned above; this converging of personal rating will be changed to move towards the team's rating for a more consistent and less confusing result.

In regards to the question as to why team members who have played every game together exclusively and not with other players are getting different individual matchmaking (hidden) ratings, the reason is the following: When calculating the matchmaking (hidden) rating, we included match history from prior to the implementation of the new matchmaking system. In other words, for players in teams who have played with other people during this current season prior to 3.0.8, the data from these games are also currently included in the calculation which determines their current matchmaking (hidden) rating.

Lastly, we also want to dispel the rumours that there is one unified matchmaking (hidden) rating for all brackets making a player’s 3v3 performance count towards his matchmaking in 2v2 teams. Simply put, there is a separate matchmaking (hidden) rating for each bracket that is pertinent solely to its respective bracket.

We hope this helps alleviate some of the confusion.

One thing: if your personal rating converges on your team rating and your team rating converges on the average of your hidden ratings, well... it'll still converge to the hidden rating. That change is pointless.

Tzeni · 01/29/09, 10:14 AM

Originally Posted by pqueiro

Clarifications by Slorkuz(Blue Poster, EU Forums) (<- that's a link in case it's hard to see)

One thing: if your personal rating converges on your team rating and your team rating converges on the average of your hidden ratings, well... it'll still converge to the hidden rating. That change is pointless.

It's not pointless; it fixes a lot of issues people have with "My team rating is 1500, but I'm at 1475 and my teammate is at 1525, so I feel like I'm getting screwed here"

dysent · 01/29/09, 11:44 AM

It's not pointless; it fixes a lot of issues people have with "My team rating is 1500, but I'm at 1475 and my teammate is at 1525, so I feel like I'm getting screwed here"

/Signed. I was going to have to play to 2050 or 2100 to buy weapons because my hidden rating was artificially low from "fun" teams at the start of the season. This change is a big help for anyone who had a similar problem to stay on even footing.

It's really good that they've clarified the factors involved - seems the system is relatively straightforward.
Also good to know that only season 5 games contribute to hidden rating.

A team's rating currently converges towards the average of that team's matchmaking (hidden) ratings

I assume this means Team Hidden Rating a straight average and isn't weighted based on number of games teamates have played relative to each other or anything else... has anyone had any experience that suggested otherwise?

• zeidrich · 01/29/09, 1:53 PM

Originally Posted by dysent

I assume this means Team Hidden Rating a straight average and isn't weighted based on number of games teamates have played relative to each other or anything else... has anyone had any experience that suggested otherwise?

It's weighted based on the number of games teammates have played.

Take an existing 2v2 team. If your hypothesis was correct, the team rating would be (R1+R2)/2. Add a third player with a different rating. The team rating would now be (R1+R2+R3)/3. Since your team rating doesn't immediately change when you add a new player, it can't be a straight average.

It just tends towards the average over time, based on the number of games you play with those members.

pqueiro · 01/29/09, 2:23 PM

Originally Posted by Tzeni

It's not pointless; it fixes a lot of issues people have with "My team rating is 1500, but I'm at 1475 and my teammate is at 1525, so I feel like I'm getting screwed here"

Yes - that is my case - but it still doesn't address the issue of team mates with the same games over the entire season having different hidden ratings. When I considered it pointless it was simply due to the fact that the problem with this system is that you're handed an arbitrary classification of "skill" by Blizzard without any explanation or means of verification, and you're stuck with that. The fact that now your whole team gets dragged down to whatever arbitrary factor Blizzard bestowed upon you is a purely cosmetic operation. You're still stuck with an arbitrary "skill" level that has - as far as I can see - nothing to do with your actual skill.

And to prove this statement I offer you the post I made earlier. My rating before the new system was 1615. My Hidden Rating post-system seems to be around 1450, because that's where my personal rating tends to over 50 games. Yes, I played one week at 1440; then I played four straight weeks at 1500-1550-1600-1615. Is that first one, that one single week, the only week they're taking into account? If so, how the hell is pushing my team's rating down towards it going to change the fact that my Hidden Rating was - seemingly - set arbitrarily and totally disregards my actual performance in arena?

Bnol · 01/29/09, 2:42 PM

The problem is that this system was released in the middle of a season that had (and still does have) huge class imbalances. Thus, those playing inferior classes/comps are being hurt even more, considering that their ghost rating will be trending downward, even after later buffs and nerfs. Essentially those who have the advantage now, will be able to keep the advantage longer, and those that are disadvantaged are kept down longer.

This would not be a huge problem were it not for rating requirements on gear. This compounds the problem of rich staying rich, and the poor staying poor. Yes, granted those that are more skilled will be able to overcome it, but with teams of equal skill, the better gear will generally win (outside of RNG), and the system is supposedly trying to match you with people of equal skill.

This system also discourages people who are new to the arena. They will likely be trended downward because of early losses, and have to struggle to get back to 1500 and stay above. Previously they could try out the arena, and not have to worry too much about losing while learning by being able to reset back to 1500. Yes you could do the practice arenas, but people do not want to wait in long queues for games that are not usually indicative of how the rated games would play out. It is like playing poker without real money. The game is completely different.

They really should have fixed imbalances and released the system next season, with a reset.

Tzeni · 01/29/09, 3:18 PM

Originally Posted by pqueiro

Yes - that is my case - but it still doesn't address the issue of team mates with the same games over the entire season having different hidden ratings. When I considered it pointless it was simply due to the fact that the problem with this system is that you're handed an arbitrary classification of "skill" by Blizzard without any explanation or means of verification, and you're stuck with that. The fact that now your whole team gets dragged down to whatever arbitrary factor Blizzard bestowed upon you is a purely cosmetic operation. You're still stuck with an arbitrary "skill" level that has - as far as I can see - nothing to do with your actual skill.

And to prove this statement I offer you the post I made earlier. My rating before the new system was 1615. My Hidden Rating post-system seems to be around 1450, because that's where my personal rating tends to over 50 games. Yes, I played one week at 1440; then I played four straight weeks at 1500-1550-1600-1615. Is that first one, that one single week, the only week they're taking into account? If so, how the hell is pushing my team's rating down towards it going to change the fact that my Hidden Rating was - seemingly - set arbitrarily and totally disregards my actual performance in arena?

The hidden rating should not be considered arbitrary. If we are to trust Blizzard, then it seems that their internal data shows that matchups made using hidden rating do a very good job at predicting the outcome of matches. (And so, it should be considered a superior method in terms of matchmaking and rewarding points). To back that point up, your first 50 games went 24-26 while the next 10 you did with a new team went 6-4, both being very close to a 50% win rate. A good matchmaking system that lines you up with appropriate competition should always keep you around a 50% win rate-- and in this case it even carried over when you made a new team.

Your other point, the post that you made earlier, is what concerns me. Even if you played 100 games at 1440 pre-patch, it should not take more than 50 games for you to start closing the distance between your Hidden Rating with that of your partner. Either his should decrease to meet yours, or yours should increase to meet his (or both). However, since the games are matched based on Team average hidden rating, and you had a 50% win rate, it should be noted that the average of your individual hidden rating produced a desirable set of match-ups.

This system also discourages people who are new to the arena. They will likely be trended downward because of early losses, and have to struggle to get back to 1500 and stay above. Previously they could try out the arena, and not have to worry too much about losing while learning by being able to reset back to 1500. Yes you could do the practice arenas, but people do not want to wait in long queues for games that are not usually indicative of how the rated games would play out. It is like playing poker without real money. The game is completely different.

Not necessarily. This system was constructed specifically so that new people would find equal matchups and not get stomped by Gladiator team re-rolls. If it works as it should, your hidden rating should gradually improve as your record improves and you continue to climb the skill ladder. However, as mentioned earlier, we have (at least from 1 person, so limited sample here) evidence that it is currently very difficult to change your Hidden Rating. Or at least, given a 2v2 team, we see a case in which the "team personal rating" (supposedly an average of the individuals queuing) is pretty accurate, but even over a large sample of games, one partner can be way ahead of the other.

pqueiro · 01/29/09, 8:28 PM

Originally Posted by Tzeni

The hidden rating should not be considered arbitrary. If we are to trust Blizzard, then it seems that their internal data shows that matchups made using hidden rating do a very good job at predicting the outcome of matches. (And so, it should be considered a superior method in terms of matchmaking and rewarding points). To back that point up, your first 50 games went 24-26 while the next 10 you did with a new team went 6-4, both being very close to a 50% win rate. A good matchmaking system that lines you up with appropriate competition should always keep you around a 50% win rate-- and in this case it even carried over when you made a new team.

How are we to trust Blizzard when they won't reveal their methods? How can we trust a system that is implicitly secretive? Besides, I never questioned the matchmaking system; I questioned the outcome in terms of rewards for each player. The system itself is dubious given the differences in ratings between matchups (and yes, I know we kept to a close 50%, but we spent most of our time facing Twilight Vanquishers, Flawless Victors and teams that won 2-6 points when they beat us), and even then, how do you explain that a 24-26 rating puts my team mate at 1625? Is his hidden rating so much higher than mine? If so, why? We played all the games together, why should his be higher than mine? Which leads us to...

Originally Posted by Tzeni

Your other point, the post that you made earlier, is what concerns me. Even if you played 100 games at 1440 pre-patch, it should not take more than 50 games for you to start closing the distance between your Hidden Rating with that of your partner. Either his should decrease to meet yours, or yours should increase to meet his (or both). However, since the games are matched based on Team average hidden rating, and you had a 50% win rate, it should be noted that the average of your individual hidden rating produced a desirable set of match-ups.

That is what I question; the arbitrariness of the Hidden Rating in the sense that the only vaguely connected piece of data I have were 10 games more than a month before the new system came out. The way I see it, it cannot possibly have taken into account the remainder of my Season 5 history, for if it had, it would've given me a much higher Hidden Rating. At this point there are two possibilities; either I got saddled with a rating I got more than a month ago and then Blizzard ignored the following entire month of arena or Blizzard just assigned me a rating arbitrarily. Both possibilities are, imho, unacceptable.

As for the ~50% success rate, towards the tail end of the first run I was just pushing for games for more data. It's not exactly a feather in my cap but we didn't really care about the outcome.

On this series, the 6-4 record, we're up to 40-32 and at the moment we have:

Team Rating = 1672
Mate's Rating = 1715
My Rating = 1608

I was going to post the results at 50, but we got carried away and meh, we're 28 games away from the 100 the blues poster in my thread asked for, so we're making a run for it tomorrow. I've started to get a feel for when the system changes its estimate of my HR (can't spot it for my mate yet though) and when the system is about to clobber me with a big fat hammer, so this set is a lot more carefully played as well. Still, I have a gap of 64 points between me and the team rating and 107 between me and my team mate, which is ridiculous.

Originally Posted by Tzeni

Not necessarily. This system was constructed specifically so that new people would find equal matchups and not get stomped by Gladiator team re-rolls. If it works as it should, your hidden rating should gradually improve as your record improves and you continue to climb the skill ladder. However, as mentioned earlier, we have (at least from 1 person, so limited sample here) evidence that it is currently very difficult to change your Hidden Rating. Or at least, given a 2v2 team, we see a case in which the "team personal rating" (supposedly an average of the individuals queuing) is pretty accurate, but even over a large sample of games, one partner can be way ahead of the other.

A few tips I've gathered from my experiments:

- If you lose more than twice, call a break. The system will clobber you (well one of you at any rate) mercilessly after the third loss.
- If you want to climb the ranks, expect to maintain at least a 2:1 win:loss ratio.
- Losing streaks cost you far more than winning streaks benefit you.
- If you want to know when the system changes its estimate of your hidden rating, keep a simple log of the difference between your team's rating changes and your own. It'll be apparent after 20-30 games.

In the meantime, you know what I'd love to have? A Recount-like addon that tracked this sort of data in arena. It'd really help people keep a hold on their Hidden Rating imho. If only I had the time to put in that kind of effort =/

sulliwan · 01/30/09, 5:00 AM

I see absolutely no reason why team rating and personal rating should change separate from eachother. After all, only things that PR matters for are achievements and items available to you, if you play 100% games in a team then there is no reason why your PR shouldn't be the same as team rating unless there's an objective system to take into account your individual contribution to the games(which is between impossible and completely ridiculous to achieve).

Currently it seems that when you create a team it creates matchups based on your average hidden rating(well, a combination of your hidden ratings and the amount of uncertainty the system has about your ratings). However, what happens when you play some games? Well, if you get around 50/50 win/loss ratio with evenly matched players, the system simply reduces the uncertainty of your hidden ratings and neither hidden rating actually changes. Because it simply thinks that it was correct about the ratings.

Let's say you have a player with 1300 hidden rating and a player with 1700 hidden rating playing together, for simplicity's sake, let's say the uncertainty about these ratings is quite low so the players have already played quite a lot of games. They get matched for 100 games in a row against 2 players with 1500 hidden rating and go 50/50 with them. The system now can simply assume that it was right about the player ratings and it isn't necessary to change any of them. So after 100 games, currently, the player with 1300 hidden rating should have around 1300 personal rating and the one with 1700 should have around 1700 PR while the team rating is still exactly 1500.

Even if you win or lose the majority of your games, your personal ratings will still keep the distance between themselves. This is completely fine for matchmaking, however making PR converge towards the matchmaking rating was quite a large mistake and I hope the personal ratings are retroactively recalculated when they change the interaction.

Disclaimer: this is all just guessing and assumes you don't swap teams and always play with the same teammate(s). With teamswapping and player rotation within the team the hidden rating would probably act a bit more interesting. Also, this is guessing that the hidden rating is basically TrueSkill or something very similar to it(rating is composed of skill value and how certain the system is about that).

pqueiro · 01/30/09, 8:30 AM

You know, there's one thing that's been bothering me for a while now...

... for God knows how long Blizzard have been adamantly refusing to balance PVP around 1v1, and yet this new system - which is NOT balanced around individual performance - distributes rewards based on an individual stat.

Taking sulliwan's example a little bit up the scale, if your Hidden Rating is, say, 1500, your team mate's is 1900, your Team Rating should hover around 1700. Let's suppose you do a month or two of Arena like this. At the end of it, your 1900-rated team mate will have all sorts of goodies both Hateful and Deadly, thus raising his potential rating even further. You, on the other hand, despite being an integral part of the team's 1700 rating, will get zilch for your efforts, because Blizzard - while refusing to balance PVP around 1v1 skill - is doling out rewards based on an individual "Hidden Rating" that no one really knows how to calculate.

But I can hear the complaints now; "the 1900 dude is boosting the 1500 dude! Serves the 1500 dude right!". That would be an acceptable argument if we knew how the Hidden Ratings were chosen; thus far my data indicates either complete arbitrariness or utter systemic failure, meaning Blizzard is either deciding player's Hidden Ratings simply of its own accord or its system is so bad it should be scrapped anyway.

As a sort of simpler, less painful example, I offer my own team; I'm rated 1608. My team mate 1715. The team is at 1672. Three more points and he can buy the Hateful chest; three more points and I'll be 64 points away from the same reward. We've played every single game together from 1500. Brilliant.

Tzeni · 01/30/09, 1:21 PM

Originally Posted by pqueiro

As a sort of simpler, less painful example, I offer my own team; I'm rated 1608. My team mate 1715. The team is at 1672. Three more points and he can buy the Hateful chest; three more points and I'll be 64 points away from the same reward. We've played every single game together from 1500. Brilliant.

And after the change, your PR will be 1672, your teammate will be at 1672, and your team will be at 1672. Which is how it should be.

I think the poster above had the right idea: in your first 50 games, based on your performance against certain teams, the game thought it was right about each of your hidden ratings and kept it the same. You were losing against teams you probably should lose against (according to your rating), and you were winning against teams you should win against. And thus you ended up at 24-26, and stayed where you were.

In the next batch of games, you admittedly stepped up your game and stopped more or less pushing games to produce more data. Now both of your ratings have gone up, but since your teammate has had a better history WITHOUT you than you do without HIM, he continues to have a higher Hidden Rating.

To illustrate:

Player X does the following:
- Plays 10 games week 1, goes 0-10, finishes at 1400.
- Patch hits, then Plays 30 games in week 2 with Player Y (new team), goes 30-0, finishes at 1900.

Player Y:
- Plays 10 games week 1, goes 10-0, finishes at 1600.
- Patch hits, then plays 30 games in week 2 with Player X (new team), goes 30-0, finishes at 1900.

Now Blizzard uses Guassian Desity Filtering, a much more complex mathematically rigorous approach, to figure out how good each player is. But to save ourselves the headache, lets use simple algebra to try and figure it out our own way:

Player X average rating= (1400*10+1900*30)/40=1775
Player Y average rating= (1600*10+1900*30)/40=1825

So as far as the game is concerned, you are still 50 points worse than your teammate despite sharing the same team rating, because he did better without you than you did without him. Also, with this method the gap will close so long as these two players continue to play each other. If these two players layed 50 games at a 1900 level, their rating difference will be only ~33 points. 70 games would close it to a 25 point gap. No matter what, Player Y's hidden rating will always be ahead of Player X, based on the information that your history provides, but the more games you play together, the closer your ratings will converge. I'm not sure that GDF works like this (in fact, I'm almost certain it doesn't), but there are some similarities in terms of the discrepancy between teammate's ratings.

However, blizzard has recognized that basing your PR off this hidden rating is flawed because your PR should reflect how much you contributed towards your team rating, not how good you are. HR (hidden rating) should only be used to calculate what teams you should face, not what rewards you receive. So this is getting fixed. It's a good fix. And as long as HR continues to give teams good matchups, none of this should matter.

gia · 01/30/09, 1:26 PM

Copy/paste since it's quite important for this discussion.
Source: World of Warcraft - English (NA) Forums -> Rating system changes last night

Originally Posted by Kalgan

We made some adjustments to the rating system last night that are now in effect.

1. Personal Ratings will now always trend toward Team Ratings (instead of the internal matchmaking rating). This should eliminate the odd behavior players noticed of seeing their PR change in ways that seemed bizzare. It also eliminates the possibility that you can play every game with your team, yet be ineligible to get items when the team qualifies for them (or ineligible to receive full points).

2. You will now gain or lose around 12 points when winning or losing to teams of roughly equivalent matchmaking rating to your own team (previously this number was 8).

3. The rating scale has been extended to a range of 240 to about 2800 to more accurately reflect what was possible under the old system. This means that most players over 1500 will experience some slightly easier than expected rating gain as their team adjusts upward to the new scale.

4. Players with less than 1500 rating will still get the same number of points as players at 1500.

Soylent13 · 02/03/09, 3:25 PM

I hope this helps someone figure out how things work.

I am on a 2vs2 team. There's only been 2 people on this team ever: Me and my brother.
We have not played on anyone elses 2vs2 team nor have we played on any 3vs3, 5vs5 ours or anyone elses.

Our gear is not exactly the same.

Our Team rating is identical.

Our Personal ratings are identical.

Ofcourse our hidden ghost rating is unknown.. but I'm gonna say it's identical as well.

Since our gear is different we can safely say gear does not effect any or the 3 ratings whatsoever.

poof312 · 02/13/09, 4:15 AM

Blue post saying PR is not gear nor spec nor comp dependent: World of Warcraft - English (NA) Forums -> New Arena Matchmaking System Overview

rockygi · 03/04/09, 1:00 PM

Right i will add my data to the pool..

I started 2v2 arena with a druid.. We created a team to practice on so that we could get used to eachothers playstyle.. Went well so we decided to create a proper team to get a high rating..

We played 27 games..

Won 20 & lost 7...

That is a 74% win ratio...

Our team rating : 1480..

Im not sure how that worked out but every match we lost we were losing 24 rating and each one we won we got like 5-7..

Anyone have any explanation to this?