BCS Series: Review of Colley ratings

For those that have read the first five installments of my BCS Ratings review, you’ll notice one major theme: nobody publishes their full methodology for how they calculate their ratings. Many of them are a “black box” where the inputs go into, some magic happens, and the output comes out. Well, the final review is of the Colley Matrix rating system and he publishes his entire methodology. Finally!

The Colley ratings only use wins and losses to begin with, so they are a perfect fit for the BCS as no adjustments needed to be made. The first step is to adjust each team’s winning percentage. Colley does this by adding 1 to the numerator and 2 to the denominator. Winning percentage is calculated as “wins / games”, thus, by adding 1 to the top and 2 to the bottom of the equation, Colley is giving each team 1 win and 1 loss.

Colley explains his choice by stating that:

All teams at the beginning of the season, when no games have been played, have an equal rating
of 1/2. After winning one game, a team has a 2/3 rating, while a losing team has a 1/3 rating, i.e.,
“twice as good,” much more sensible than 100% and 0%, or “infinitely better.”

Why is it more sensible? Essentially what Colley is doing is starting with a prior (think of it as an educated guess) that each team is a .500 team. By choosing “1/2”, he is giving this prior a weight of 2 games. That is, after a team has played 2 games, our best guess is equal parts their current record and our initial guess of .500. After 6 games, it is 3/4 the team’s record and 1/4 the initial guess. But why a weight of 2? Colley provides no explanation except that it is “much more sensible”.

The final adjustment for Colley’s method is to account for schedule strength. He does this by adjusting each team’s record based on the ratings of their opponents. By iterating this process over and over, the opponent adjustments are continually changed until they converge to a stable point. However, what Colley does is turn this iterative process into a matrix that can be solved without iteration, and he shows how the two methods are equivalent.

The Colley Matrix is a solid rating system, and thankfully it describes its entire methodology. The main question mark is why the prior of 1/2 is used (should it be 2/4 or 3/6 or something else?), but the effect on the rankings should be minimal since each team is given the same adjustment. This may effect the absolute ratings but should not have a large effect on the ordering of the teams.

This entry was posted in BCS Series, College Football, Football, review, team evaluation and tagged , , , . Bookmark the permalink.

4 Responses to BCS Series: Review of Colley ratings

  1. Adam H says:

    He actually does explain why his prior is given a weight of 2 games. It is derived from an equation that uses integrals so I won’t be able to give it here, but it’s in colley’s method pdf.

  2. Monte says:

    Adam/1, I could be wrong, but I think his explanation would work with any fraction that reduces to 1/2. Why not 2/4 or 0.5/1?

  3. Adam H says:

    Well, I’m looking at what he calls equation 6. He has an integral divided by another integral equaling (1+nw)/(2+nw+nl), not (2+nw)/(4+nw+nl) or (3+nw)/(6+nw+nl).

    I could easily be misunderstanding the math.

  4. Monte says:

    Ah ok, I see what you’re saying. I’m not caught up on my integrals so I’m not quite following his whole explanation there, so you’re probably right. I was coming at it from the simple Bayesian case of what actually will be implemented, which is adding 2 games worth of .500 record to a team’s win-loss record.

    But like I said, I don’t think this actually matters as far as the ordinal ranking of teams is concerned. Thanks for pointing that out.

Leave a Reply

Your email address will not be published.