Tomorrow, we’ll find out who the committee thinks should be the final four teams in the College Football Playoff. As I’ve laid out before, I’m not a fan of the “committee” approach, which is given vague and unspecific guidelines on how to select the four teams. That leaves the criteria open to interpretation and invites inconsistent and biased decision-making. Are we selecting the “best” or “most deserving” teams? What does “best” or “most deserving” actually mean? Does the “eye test” matter? Do big wins outweigh bad losses? Does head-to-head matter, and if so, for how much? Same for conference championships?
There are so many questions to answer, and many are open for debate. However, leaving that debate for after the games have been played and up to each committee member to answer for themselves is a travesty. The answers to those things need to be decided ahead of time so that every team knows the parameters by which they’ll be judged, and they need to be applied consistently and objectively to remove as much subjectivity and bias (whether conscious or subconscious) as possible.
Achievement Rankings
My solution is the Achievement Rankings, which I’ve applied to the NCAA basketball tournament for years and carried over to football as well.
The base of my system is simple–how many wins would a baseline team be expected to win against your schedule, including your opponents and the location of the games. For my baseline, I use the #10 team in the country.
Things can be added in as seen fit, and the biggest one is the value of Conference Championships. This year, the committee’s weight on that will play a huge role, as 11-1 Ohio State did not win the Big Ten conference, while 2-loss teams Penn State (Big Ten) and Oklahoma (Big 12) both did. How much should those titles be worth? The answer to that question could determine who goes to the playoff and who doesn’t.
Another big question this year is the value of head-to-head wins, as Penn State beat Ohio State this year, but Ohio State beat Oklahoma, and Michigan beat Penn State but lost to Ohio State. I am not a proponent of giving additional bonuses for head-to-head wins, which deserves a longer post, but the main reasons involve difficulties applying them in a logically consistent way (for instance, in 3-way ties) and the dubious idea that they only matter if you just so happen to be close in the ranking to a team you beat.
This year, I’ve decided to include a conference championship bonus of +1 achievement points (I gave non-Power 5 champions +0.5 points), but eschew any head-to-head adjustments. The choice of +1 for conference titles was rather arbitrary–the main thing is that this number should be arrived at ahead of time before the season and applied consistently. As it stands now, each committee member gets to decide whether to apply a bonus and how much it should count for–and they may not even apply it consistently!
2016 CFB Playoff – Achievement Rankings
So who does my system say should go this year?
Alabama is a stone-cold lock as the #1 seed. Both Clemson and Washington grade out very well with just 1 loss and a Power 5 conference championship for each team.
That leaves just the fourth and final spot up for grabs. The contenders–Penn State, Ohio State, and Michigan from the Big Ten, Oklahoma from the Big 12, and Western Michigan as the undefeated non-Power 5 MAC champion. My system narrows it down to two of those 5 teams–Penn State and Ohio State. The Buckeyes have a tougher schedule and one fewer loss–they went 11-1 against a schedule that the #10 team in the country would be expected to win 9.13 games against, meaning they won 1.87 games more than expected. Without a conference title, that’s their score. Penn State went 11-2 against a schedule that the #10 team would win 10.05 games on average if they played the same slate of games, giving PSU just a 0.95 score. However, they are the Big Ten champions so get a +1 bonus, putting them at +1.95 and just ahead of the Buckeyes. As you can see, the conference championship bonus amount matters greatly this year.
There’s one other thing to point out–Penn State benefits from playing 13 games to Ohio State’s 12. My system uses an aggregate score, but if you instead used a per-game average, Ohio State would edge ahead of Penn State, with a +.156/game score compared to +.150/game for the Nittany Lions.
Conclusion
Ultimately, we’ll be at the mercy of the committee members and their application of individual rankings of this year’s teams. But with so many potential factors and razor thin margins, we really need a system that lays out the details of how teams will be graded–objective, consistent, and transparent. Of course, teams can always feel safe if they go undefeated and win their conference, like Alabama was able to do this season.