Category: College Football


College Football Playoff 2016 – Achievement Rankings

December 3rd, 2016 — 11:30pm

Tomorrow, we’ll find out who the committee thinks should be the final four teams in the College Football Playoff. As I’ve laid out before, I’m not a fan of the “committee” approach, which is given vague and unspecific guidelines on how to select the four teams. That leaves the criteria open to interpretation and invites inconsistent and biased decision-making. Are we selecting the “best” or “most deserving” teams? What does “best” or “most deserving” actually mean? Does the “eye test” matter? Do big wins outweigh bad losses? Does head-to-head matter, and if so, for how much? Same for conference championships?

There are so many questions to answer, and many are open for debate. However, leaving that debate for after the games have been played and up to each committee member to answer for themselves is a travesty. The answers to those things need to be decided ahead of time so that every team knows the parameters by which they’ll be judged, and they need to be applied consistently and objectively to remove as much subjectivity and bias (whether conscious or subconscious) as possible.

Achievement Rankings

My solution is the Achievement Rankings, which I’ve applied to the NCAA basketball tournament for years and carried over to football as well.

The base of my system is simple–how many wins would a baseline team be expected to win against your schedule, including your opponents and the location of the games. For my baseline, I use the #10 team in the country.

Things can be added in as seen fit, and the biggest one is the value of Conference Championships. This year, the committee’s weight on that will play a huge role, as 11-1 Ohio State did not win the Big Ten conference, while 2-loss teams Penn State (Big Ten) and Oklahoma (Big 12) both did. How much should those titles be worth? The answer to that question could determine who goes to the playoff and who doesn’t.

Another big question this year is the value of head-to-head wins, as Penn State beat Ohio State this year, but Ohio State beat Oklahoma, and Michigan beat Penn State but lost to Ohio State. I am not a proponent of giving additional bonuses for head-to-head wins, which deserves a longer post, but the main reasons involve difficulties applying them in a logically consistent way (for instance, in 3-way ties) and the dubious idea that they only matter if you just so happen to be close in the ranking to a team you beat.

This year, I’ve decided to include a conference championship bonus of +1 achievement points (I gave non-Power 5 champions +0.5 points), but eschew any head-to-head adjustments. The choice of +1 for conference titles was rather arbitrary–the main thing is that this number should be arrived at ahead of time before the season and applied consistently. As it stands now, each committee member gets to decide whether to apply a bonus and how much it should count for–and they may not even apply it consistently!

2016 CFB Playoff – Achievement Rankings

So who does my system say should go this year?

Alabama is a stone-cold lock as the #1 seed. Both Clemson and Washington grade out very well with just 1 loss and a Power 5 conference championship for each team.

That leaves just the fourth and final spot up for grabs. The contenders–Penn State, Ohio State, and Michigan from the Big Ten, Oklahoma from the Big 12, and Western Michigan as the undefeated non-Power 5 MAC champion. My system narrows it down to two of those 5 teams–Penn State and Ohio State. The Buckeyes have a tougher schedule and one fewer loss–they went 11-1 against a schedule that the #10 team in the country would be expected to win 9.13 games against, meaning they won 1.87 games more than expected. Without a conference title, that’s their score. Penn State went 11-2 against a schedule that the #10 team would win 10.05 games on average if they played the same slate of games, giving PSU just a 0.95 score. However, they are the Big Ten champions so get a +1 bonus, putting them at +1.95 and just ahead of the Buckeyes. As you can see, the conference championship bonus amount matters greatly this year.

cfb-playoff-2016-achievement-rankings

There’s one other thing to point out–Penn State benefits from playing 13 games to Ohio State’s 12. My system uses an aggregate score, but if you instead used a per-game average, Ohio State would edge ahead of Penn State, with a +.156/game score compared to +.150/game for the Nittany Lions.

Conclusion

Ultimately, we’ll be at the mercy of the committee members and their application of individual rankings of this year’s teams. But with so many potential factors and razor thin margins, we really need a system that lays out the details of how teams will be graded–objective, consistent, and transparent. Of course, teams can always feel safe if they go undefeated and win their conference, like Alabama was able to do this season.

 

Comment » | CFB Achievement Rankings, CFB Playoff, College Football

College Football Playoff – Final Achievement Rankings

December 7th, 2014 — 11:09am

We’re less than an hour from the CFB Playoff selection, so I figured I’d release a final version of my system. For an explanation of the system and caveats with it, see this post.

We can set the baseline team wherever we want, so I’ll show 2 different sets of rankings–vs the #10 team and vs the #25 team.

Achievement Rankings – #10 team baseline

Rk Team ExpW ExpL W L Score Score/G
1 Alabama 9.00 4.00 12 1 3.00 0.231
2 Florida St 11.49 1.51 13 0 1.51 0.116
3 Mississippi St 8.81 3.19 10 2 1.19 0.099
4 Baylor 10.04 1.96 11 1 0.96 0.080
5 TCU 10.08 1.92 11 1 0.92 0.077
6 Oregon 11.34 1.66 12 1 0.66 0.051
7 Auburn 7.65 4.35 8 4 0.35 0.029
8 Mississippi 8.66 3.34 9 3 0.34 0.028
9 Ohio St 11.78 1.22 12 1 0.22 0.017
10 Missouri 10.09 2.91 10 3 -0.09 -0.007

Achievement Rankings – #25 team baseline

Rk Team ExpW ExpL W L Score Score/G
1 Alabama 6.62 6.38 12 1 5.38 0.414
2 Florida St 9.04 3.96 13 0 3.96 0.305
3 Mississippi St 7.16 4.84 10 2 2.84 0.236
4 Oregon 9.06 3.94 12 1 2.94 0.226
5 Auburn 5.30 6.70 8 4 2.70 0.225
6 TCU 8.44 3.56 11 1 2.56 0.214
7 Baylor 8.47 3.53 11 1 2.53 0.211
8 Mississippi 6.52 5.48 9 3 2.48 0.206
9 Missouri 7.92 5.09 10 3 2.09 0.160
10 Ohio St 9.92 3.08 12 1 2.08 0.160

Obviously Mississippi State is going to be the big surprise here, but Sagarin’s ratings which I’m using, are astronomically high on the SEC West (see here). This could be too high or it could be right (I think we’d all agree the SEC West was an extremely strong division), but in the end it makes their schedule over a loss tougher than the other top contenders. Adjusting the SEC downward a bit, would give you either Baylor and TCU (no Oregon) in the #10-team version or Oregon and TCU in the #25-team version. The issue with Oregon is that the high baseline we set combined with Sagarin’s view of the Pac-12–lots of good but few great teams–makes Oregon’s schedule look relatively easy for a top 10 or even top 25 team.

I think there are good reasons for moving Mississippi State down, but this is a good reminder that teams shouldn’t be automatically excluded simply because they have more losses than another team. An extremely tough schedule can be enough to account for the extra loss.

Also, remember that conference championships, head-to-head, margin of victory, and the “eye test” are not included here, but those are things the committee could consider. Most of those things would not work in Mississippi State’s favor.

It will be interesting to see what the committee does today, both in selection and seeding. And then we all get to enjoy college football’s first playoff on the field, which promises to be exciting no matter who is selected.

 

Comment » | CFB Achievement Rankings, CFB Playoff, College Football, Football, team evaluation

College Football Playoff – Achievement Rankings

November 16th, 2014 — 1:46pm

Decided to dust off the ol’ blog for the new College Football Playoff system. I’m simply going to apply my Achievement S-Curve [see here] to college football to see who should be selected for the inaugural CFB Playoff this year.

Quick summary of what the Achievement Rankings are doing: determine the most deserving teams based on on-field accomplishments. From each team’s perspective all that matters is whether they won or lost, and how difficult the game was (opponent strength, home/away, etc.). For opponent strength, however, we are free to use a more predictive rating, or true strength. Simple example: the Arizona Cardinals at 8-1 against a tough schedule would come out on top of a most deserving ranking in the NFL and if they season ended today, they certainly deserve the #1 seed. However, the Broncos or Packers or maybe even Seahawks may be the best team and would come out on top of a predictive system (say, Inpredictable‘s, which has DEN #1, GB #2, NE #3, SEA #4…and ARZ #14).

There are two choices to make to utilize my system: you need a rating system to determine the difficulty of the game (mostly the strength of each opponent, but also home-field advantage, etc.) and you need to choose a baseline team to compare against. I’m going to use Sagarin‘s Predictor rating to determine opponent strength and for now, I’ll use the equivalent of the #10 team in the country as our baseline team since we are trying to determine the top 4 teams. So for each team I’ll find what record we’d expect the #10 team in the nation (think Ohio State or Oklahoma) to have had they played their schedule, and we’ll use Sagarin to determine that.

Let’s take Florida State as an example. Here’s their schedule with how often both the #10 team and the #25 team would be expected to win each game:

Loc Opponent #10 Team #25 Team
N- Oklahoma St 98% 83%
vs Citadel 100% 100%
vs Clemson 73% 58%
@ NC State 94% 79%
vs Wake Forest 100% 100%
@ Syracuse 93% 77%
vs Notre Dame 73% 58%
@ Louisville 58% 43%
vs Virginia 98% 82%
@ Miami FL 60% 45%

On average, we’d expect the #10 team to win about 8.5 games against this schedule, and the #25 team would win 7.25 on average. Since Florida State is 10-0, their score is +1.5 or +2.75, depending on the baseline team you choose.

Achievement Rankings

Here are the rankings with a #10 team as the baseline:

Rk Team ExpW ExpL W L Score Score/G
1 Alabama 6.49 3.51 9 1 2.51 0.251
2 Mississippi St 7.43 2.57 9 1 1.57 0.157
3 Florida St 8.46 1.54 10 0 1.54 0.154
4 Oregon 7.71 2.29 9 1 1.29 0.129
5 TCU 7.93 2.07 9 1 1.07 0.107
6 Baylor 7.14 1.86 8 1 0.86 0.095
7 Auburn 6.19 3.81 7 3 0.81 0.081
8 Mississippi 7.30 2.70 8 2 0.70 0.070
9 Georgia 7.32 2.68 8 2 0.68 0.068
10 UCLA 7.45 2.55 8 2 0.55 0.055
11 Ohio St 8.57 1.43 9 1 0.43 0.043
12 Arizona St 7.91 2.09 8 2 0.09 0.009

And if we lower the baseline to the #25 team, we get:

Rk Team ExpW ExpL W L Score Score/G
1 Alabama 5.28 4.72 9 1 3.72 0.372
2 Florida St 7.25 2.75 10 0 2.75 0.275
3 Oregon 6.49 3.51 9 1 2.51 0.251
4 Mississippi St 6.52 3.48 9 1 2.48 0.248
5 Auburn 4.82 5.18 7 3 2.18 0.218
6 UCLA 5.93 4.07 8 2 2.07 0.207
7 TCU 6.98 3.02 9 1 2.02 0.202
8 Georgia 6.11 3.89 8 2 1.89 0.189
9 Baylor 6.38 2.62 8 1 1.62 0.180
10 Mississippi 6.23 3.77 8 2 1.77 0.177
11 Ohio St 7.49 2.51 9 1 1.51 0.151
12 Arizona St 6.69 3.31 8 2 1.31 0.131

I prefer the higher baseline, which gives more credit to top wins and less to middle tier wins. In either system, the top 4 are Alabama, Florida State, Oregon and Mississippi State, albeit in different orders. TCU, Baylor, and Auburn are next in line in the #10 version. UCLA climbs up to #6 in the #25 version, as they have a bunch of middle-tier wins (Virginia, Memphis, Texas, Washington, etc.). It’s up to you what type of resume you want to reward.

A couple quick notes:

  • College football teams play so few “connecting” games (i.e. non-conference games) that team strength ratings (like Sagarin) are very sensitive to these and conferences as a whole can move up or down the ratings based on just a handful of out of conference matchups. The SEC is certainly very strong this year, but it’s possible they as a conference are overrated due to this phenomenon.
  • This system does not incorporate certain things that others may thing they should. Some of these I’m okay with: conference championships, for instance, or recent performance. Others, I am strongly against: head-to-head and especially the dreaded “eye” test.

In any case, I’m excited to see how the whole thing plays out. College Football finally has a Playoff.

Comment » | CFB Achievement Rankings, CFB Playoff, College Football

BCS Series: Review of Colley ratings

December 3rd, 2011 — 2:59pm

For those that have read the first five installments of my BCS Ratings review, you’ll notice one major theme: nobody publishes their full methodology for how they calculate their ratings. Many of them are a “black box” where the inputs go into, some magic happens, and the output comes out. Well, the final review is of the Colley Matrix rating system and he publishes his entire methodology. Finally! Continue reading »

4 comments » | BCS Series, College Football, Football, review, team evaluation

BCS Series: Review of Massey ratings

December 3rd, 2011 — 2:22pm

The Massey ratings have been around since December of 1995, according to his site. The explanation he lists is actually for his rankings that include scoring margin, and not those that are used in the BCS (which can’t use score margin).

However, perhaps we can derive some understanding of Massey’s BCS ratings if they are calculated similarly to his other ratings. Continue reading »

3 comments » | BCS Series, College Football, Football, review, team evaluation

BCS Series: Review of Wolfe ratings

October 26th, 2011 — 10:54pm

Continuing with my review of BCS computer rating systems, the 4th of the 6 systems in my series is Dr. Peter Wolfe’s ratings.

On his site, Wolfe only gives a brief explanation:

We rate all varsity teams of four year colleges that can be connected by mutual opponents, taking note of game locations….The method we use is called a maximum likelihood estimate.  In it, each team i is assigned a rating value πi that is used in predicting the expected result between it and its opponent j, with the likelihood of i beating j given by:

 π/ (πi + πj)

The probability P of all the results happening as they actually did is simply the product of multiplying together all the individual probabilities derived from each game.  The rating values are chosen in such a way that the number P is as large as possible.

First thing to note is that Wolfe rates all teams from FBS through Division III and even NAIA. He includes all games between any two varsity teams at any level. Other systems, like Sagarin, only rate Division I teams. Some only rate the FBS teams. I am not sure any one method is more “right” than the others, but it is odd that the BCS allows different systems to rate different sets of teams. Continue reading »

5 comments » | BCS Series, College Football, Football, review, team evaluation

BCS Series: Review of Anderson & Hester ratings

October 20th, 2011 — 10:13pm

In the third installment of my review of the BCS computer rankings, I will take a look at the ratings of Anderson and Hester. For starters, they have a great tagline on their website: “showing which teams have accomplished the most”. For those of you that have been following, you know my stance on how teams should be judged for inclusion to the BCS title game and this fits perfectly.

Anderson and Hester don’t give many details about their system, but they do highlight four ways in which they believe their ratings to be distinct. Let’s take them one by one. Continue reading »

5 comments » | BCS Series, College Football, Football, review, team evaluation

BCS Series: Review of Billingsley ratings

October 19th, 2011 — 10:52pm

Next up in my review of the computer ranking systems in the BCS is Richard Billingsley. He gives a much more detailed explanation of his ratings on his website: read them here. I will pull out pertinent parts of his explanation and comment. Let’s start with his summary.

 I guess in a sense, my rankings are not only about who the “best team” is, but also about who is the “most deserving” team.

This is a decent start. As I have touched on before, I believe that postseason play–whether it be the BCS, NCAA Tournament, or NFL playoffs–should be a reward for the most deserving as opposed to the “best” teams. However, people get into trouble when they try to satisfy both types of ratings: predictive and descriptive. By straddling the line, ratings suffer from trying to do too many things. Focusing on answering just one question will provide the best, purest, most useful answer. Continue reading »

6 comments » | BCS Series, College Football, Football, review, team evaluation

BCS Series: Review of Sagarin ratings

October 18th, 2011 — 10:34pm

Jeff Sagarin produces some of the most respected ratings, not just for college football but for the NBA, NFL, college basketball, and others. His ratings, found here, include both a Predictor and an Elo Chess rating. The Predictor rating includes margin of victory and is intended to, well you guessed it, predict future games. In other words, it is a measure of the quality of a team. We are concerned with his other rating, the Elo Chess, which is the one used by the BCS. This rating considers only wins and losses, and in Sagarin’s words “makes it very “politically correct”.” Continue reading »

1 comment » | BCS Series, College Football, Football, review, team evaluation

BCS Series: Introduction

October 18th, 2011 — 9:25pm

In the lead up to March Madness, I wrote about determining which teams are the “most deserving” as opposed to which teams were the “best”. I eventually created what I called the Achievement S-Curve (in college basketball, the S-Curve refers to ranking and seeding teams for the tournament), essentially a rating of teams based on what they accomplished on the court.

With the initial BCS rankings released this week, I’d like to do something similar for college football. However, before revealing my rankings, I’ll first go through and discuss each of the six computer rankings in use by the BCS. I’ll point out what they do well and critique what they don’t. Following that, I’ll unveil my own Achievement Rankings. In addition, I’ll look at some other interesting aspects of the BCS system along the way: What’s the best way to make the title game? Who are this year’s best contenders? And, of course, would a playoff system be a better alternative to crowning a national champion?

If you have anything you’d be interested in seeing, post in the comments and I’ll see if I can add it in to the list. First up: a review of Jeff Sagarin’s rankings.

Comment » | BCS Series, College Football, team evaluation

Back to top