BCS Series: Review of Anderson & Hester ratings

In the third installment of my review of the BCS computer rankings, I will take a look at the ratings of Anderson and Hester. For starters, they have a great tagline on their website: “showing which teams have accomplished the most”. For those of you that have been following, you know my stance on how teams should be judged for inclusion to the BCS title game and this fits perfectly.

Anderson and Hester don’t give many details about their system, but they do highlight four ways in which they believe their ratings to be distinct. Let’s take them one by one.

1. Unlike the polls, these rankings do not reward teams for running up scores.  Teams are rewarded for beating quality opponents, which is the object of the game.  Posting large margins of victory, which is not the object of the game, is not considered.

This follows from their tagline as the way to measure what teams have accomplished is through wins and losses, “which is the object of the game.” This is exactly right.

2. Unlike the polls, these rankings do not prejudge teams.  These rankings first appear after the season’s fifth week, and each team’s ranking reflects its actual accomplishments — on the field, to date — not its perceived potential.

If you caught my review of the Billingsley ratings I harped on this point quite a bit. If you are trying to reward teams for their season, you cannot use anything but their performance in that season.

3. These rankings compute the most accurate strength of schedule ratings.  Each team’s opponents and opponents’ opponents are judged not only by their won-lost records but also, uniquely, by their conferences’ strength (see #4).

This is where we run into a problem. Why do you need to use the conference strength to rate teams? According to this site, “A&H also weighs strength of schedule according to conference strength, which is also measured by the conference’s opponents and opponents’ opponents.” Teams should be judged by the teams they actually play. Their conference should have no bearing on their rating outside of the fact that they play many teams within their conference. However, the rating system should simply look at the games each team played, and adjust accordingly. For instance, what if an SEC team plays Alabama but avoids Vanderbilt, while another plays Vandy but avoids the Crimson Tide? Do they both also get the same conference adjustment? That is clearly not right. The team that played Alabama clearly faced a tougher schedule. Simply look at each individual team and use only the opponents on their schedule to determine the strength of schedule they faced.

The previous web site I linked to also mentions that “as far as strength of schedule, A&H takes into account both a given team’s opponents and the team’s opponents’ opponents.” If this is true, this is an acceptable way to look at strength of schedule. Looking at wins and losses is definitely biased, as you will find a large spread in talent between teams with the same record based on the schedules they played. Ideally, you would use something like an iterative process to look at opponents’ opponents’ opponents (the third level deep), the fourth level, and so on. You do this until you reach convergence for the strength of schedule (meaning that going another level deeper only changes your SOS rating less than some previously determined small amount). However, in practice, going just two levels deep is likely sufficient (though this is worthy of testing).

4. These rankings provide the most accurate conference ratings.  Each conference is rated according to its non-conference won-lost record and the difficulty of its non-conference schedule.

This is a claim the authors make, and it may or may not be true. In any case, the BCS rankings are concerned with teams and not conferences so I will not comment on it here.

Summary

The main framework of the Anderson & Hester rating system is a good one. They incorporate wins and losses with a strength of schedule based on two levels of opponents’ records, while focusing solely on the current season. The one odd part of the system is the inclusion of conference strength in the ratings, which is unnecessary once a team’s schedule has already been considered. Overall, this seems like an adequate but flawed system. Learning about exactly how the conference strength is factored in is key to determining the quality of the system.

Category: BCS Series, College Football, Football, review, team evaluation | Tags: , , 5 comments »

5 Responses to “BCS Series: Review of Anderson & Hester ratings”

  1. Austin Link

    Hey Monte, cool work you’re doing here, I did a very similar thing last year (http://austinsportsstats.blogspot.com/2010/11/bcs-week-bcs-computers.html). Loved the Billingsley article, great way to break down a well intentioned but out of date system.
    I’ve always been a little confused about A&H and their conference strength, but think I’ve found what could be the answer. Because of conferences, it turns out that even going to opponents-opponents wins does not create a very accurate strength of schedule, and teams with light schedules will still be roughly equal to other teams. I think the conference rating is a fudge factor that tries to create the differentiation you’d expect between BCS and mid-major teams. As a result Big conference teams with weak schedules (Wisconsin) do well, but small conference teams with strong schedules (Boise St. so far) do worse than they should.
    Thanks for the detailed articles, and keep up the good work!
    -Austin

  2. Monte

    Very cool Austin, and thanks for the information on A&H’s conference strength. I still think it’s a flaw and that instead of including conference strength, you need to just do a better job of schedule strength (sounds like your article agrees).

  3. get download

    Good write-up, I’m normal visitor of one’s website, maintain up the excellent operate, and It’s going to be a regular visitor for a lengthy time.

  4. get blackhat

    Hello, you used to write excellent, but the last several posts have been kinda boring… I miss your super writings. Past several posts are just a little bit out of track! come on!

  5. get download

    Hey, you used to write fantastic, but the last few posts have been kinda boring… I miss your super writings. Past several posts are just a little bit out of track! come on!


Leave a Reply



Back to top