BCS Series: Review of Colley ratings

For those that have read the first five installments of my BCS Ratings review, you’ll notice one major theme: nobody publishes their full methodology for how they calculate their ratings. Many of them are a “black box” where the inputs go into, some magic happens, and the output comes out. Well, the final review is of the Colley Matrix rating system and he publishes his entire methodology. Finally!

The Colley ratings only use wins and losses to begin with, so they are a perfect fit for the BCS as no adjustments needed to be made. The first step is to adjust each team’s winning percentage. Colley does this by adding 1 to the numerator and 2 to the denominator. Winning percentage is calculated as “wins / games”, thus, by adding 1 to the top and 2 to the bottom of the equation, Colley is giving each team 1 win and 1 loss.

Colley explains his choice by stating that:

All teams at the beginning of the season, when no games have been played, have an equal rating
of 1/2. After winning one game, a team has a 2/3 rating, while a losing team has a 1/3 rating, i.e.,
“twice as good,” much more sensible than 100% and 0%, or “infinitely better.”

Why is it more sensible? Essentially what Colley is doing is starting with a prior (think of it as an educated guess) that each team is a .500 team. By choosing “1/2”, he is giving this prior a weight of 2 games. That is, after a team has played 2 games, our best guess is equal parts their current record and our initial guess of .500. After 6 games, it is 3/4 the team’s record and 1/4 the initial guess. But why a weight of 2? Colley provides no explanation except that it is “much more sensible”.

The final adjustment for Colley’s method is to account for schedule strength. He does this by adjusting each team’s record based on the ratings of their opponents. By iterating this process over and over, the opponent adjustments are continually changed until they converge to a stable point. However, what Colley does is turn this iterative process into a matrix that can be solved without iteration, and he shows how the two methods are equivalent.

The Colley Matrix is a solid rating system, and thankfully it describes its entire methodology. The main question mark is why the prior of 1/2 is used (should it be 2/4 or 3/6 or something else?), but the effect on the rankings should be minimal since each team is given the same adjustment. This may effect the absolute ratings but should not have a large effect on the ordering of the teams.