BCS Series: Review of Sagarin ratings

Jeff Sagarin produces some of the most respected ratings, not just for college football but for the NBA, NFL, college basketball, and others. His ratings, found here, include both a Predictor and an Elo Chess rating. The Predictor rating includes margin of victory and is intended to, well you guessed it, predict future games. In other words, it is a measure of the quality of a team. We are concerned with his other rating, the Elo Chess, which is the one used by the BCS. This rating considers only wins and losses, and in Sagarin’s words “makes it very “politically correct”.”

Sagarin does not divulge the details of his system, but the name suggests that it is based on the Elo rating system, most known for its use in rating chess players. Brian Burke took a look at Sagarin Elo ratings in the NFL. Go here for his explanation. Essentially what the Elo system does is calculate the probability of team A beating team B based on their current ratings and then adjusts their ratings based on the outcome of the game. The more likely that team A is to beat team B, the less credit they’ll receive for a win and the more they’ll be hurt by a loss: Oklahoma gets more credit for beating Texas than they would for beating North Texas.

For those familiar with my Achievement Ratings, this is similar to what I propose. As I’ll get to, one of the main differences in my approach is that I use the best predictive ratings to project the odds of one team beating another, while the Elo system uses itself, which is a non-predictive rating.

Overall, the Sagarin ratings are a simple yet sound way to rate teams for the BCS. They do not use any other information besides wins and losses from games from the current season. Schedule strength is incorporated, though it is unclear to what extent. Obviously the strength of the opponent is taken into account, but it is unclear whether the location of the game is a factor. In addition, Brian Burke conjectured that Sagarin includes some recency weighting, which would have a significant effect the ratings. I generally prefer to not use recency, but it is certainly an acceptable choice and more a matter of preference as long as it is done in a reasonable way. Despite being a bit of a black box, Sagarin ratings are a good system and a worthy addition to the BCS collection.

Next up: Richard Billingsley’s ratings.