Category: team evaluation

Grading the Selection Committee’s In-Season Preview

February 11th, 2017 — 11:03pm

Today, the NCAA Selection Committee put out their first ever in-season preview, releasing the current top 16 if the season were to end today. Let’s see how they did.

First, here is their s-curve alongside my own Achievement Rankings, ESPN’s Strength of Record rankings, and ESPN’s current bracketology seeds.

Continue reading »

Comment » | College Basketball, descriptive, March Madness, review, team evaluation

Conference Tournament Predictions 2016 – Final Results

March 14th, 2016 — 12:28am

Today is Selection Sunday, representing the start of the NCAA Tournament, but it also marks the end of the conference tournament season.

Earlier, I recapped how my projection system fared in 2015 with conference tournament predictions against Ken Pomeroy and Team Rankings predictions. Unfortunately for myself, the results were the same as when I tracked these in 2013–a 3rd place finish behind 2nd place KenPom and 1st place Team Rankings.

This year, things were finally different and my projections scored a resounding victory over the other two competitors, while Team Rankings edged out KenPom for 2nd place honors. The full results are in a google spreadsheet here. My projections had a strong showing, “winning” over half of the 31 conference tournaments–in 16 conferences I had the lowest cumulative score by conference, with 9 2nd place finishes and 6 3rd places. Team Rankings actually had more last place finishes than KenPom (13-12) but had twice as many 1st places (10-5) which was enough to secure the 2nd place overall finish for TR.

Comment » | College Basketball, Conference Tournament predictions, predictive, team evaluation

Conference Tournament Predictions 2015 – Final Results

March 3rd, 2016 — 11:52pm

Three years ago, I compiled predictions for the conference tournaments from three sources–my own, Ken Pomeroy, and Team Rankings. When the dust settled, Team Rankings had narrowly edged out KenPom for the title as I lagged behind a distant third.

I didn’t get around to it in 2014 (though perhaps I can find time to go back and gather predictions from that season), but last year I did track things. Unfortunately, I’m just now getting around to posting it. The results were the same, though this time, Team Rankings won comfortably over KenPom and my own predictions. I’ve posted the full spreadsheet on Google docs, which you can find here. I discuss the scoring system in this post. Since we are posting advancement odds, we don’t have predictions for each individual matchup. Instead, predictions are essentially a rolled up version of all possible matchups. To score them, I use the log of each team’s predictions to get exactly to the round they did. For instance, my predictions for Montana in the Big Sky tournament were 81%/61%/43%, meaning an 81% chance of winning the 1st round and advancing to the semifinals, 61% of reaching the final, and 43% of winning the title. Another way of looking at it is that Montana had a 19% chance to lose in the 1st round (that’s 100% minus the 81% chance to win in the 1st round), a 20% chance of winning one game and then losing in the semis, an 18% chance of winning twice and losing in the final, and, of course, the 43% chance to win it all. Those are the probabilities that are scored.

This year is under way. If I get around to it, I may post the predictions for each of the three systems, but either way, I’ll be back in a couple weeks with the final results. Good luck to Ken Pomeroy and Team Rankings; I hope to be able to at least climb out of the cellar this year.

Comment » | College Basketball, Conference Tournament predictions, March Madness, predictive, review, team evaluation

College Football Playoff – Final Achievement Rankings

December 7th, 2014 — 11:09am

We’re less than an hour from the CFB Playoff selection, so I figured I’d release a final version of my system. For an explanation of the system and caveats with it, see this post.

We can set the baseline team wherever we want, so I’ll show 2 different sets of rankings–vs the #10 team and vs the #25 team.

Achievement Rankings – #10 team baseline

Rk Team ExpW ExpL W L Score Score/G
1 Alabama 9.00 4.00 12 1 3.00 0.231
2 Florida St 11.49 1.51 13 0 1.51 0.116
3 Mississippi St 8.81 3.19 10 2 1.19 0.099
4 Baylor 10.04 1.96 11 1 0.96 0.080
5 TCU 10.08 1.92 11 1 0.92 0.077
6 Oregon 11.34 1.66 12 1 0.66 0.051
7 Auburn 7.65 4.35 8 4 0.35 0.029
8 Mississippi 8.66 3.34 9 3 0.34 0.028
9 Ohio St 11.78 1.22 12 1 0.22 0.017
10 Missouri 10.09 2.91 10 3 -0.09 -0.007

Achievement Rankings – #25 team baseline

Rk Team ExpW ExpL W L Score Score/G
1 Alabama 6.62 6.38 12 1 5.38 0.414
2 Florida St 9.04 3.96 13 0 3.96 0.305
3 Mississippi St 7.16 4.84 10 2 2.84 0.236
4 Oregon 9.06 3.94 12 1 2.94 0.226
5 Auburn 5.30 6.70 8 4 2.70 0.225
6 TCU 8.44 3.56 11 1 2.56 0.214
7 Baylor 8.47 3.53 11 1 2.53 0.211
8 Mississippi 6.52 5.48 9 3 2.48 0.206
9 Missouri 7.92 5.09 10 3 2.09 0.160
10 Ohio St 9.92 3.08 12 1 2.08 0.160

Obviously Mississippi State is going to be the big surprise here, but Sagarin’s ratings which I’m using, are astronomically high on the SEC West (see here). This could be too high or it could be right (I think we’d all agree the SEC West was an extremely strong division), but in the end it makes their schedule over a loss tougher than the other top contenders. Adjusting the SEC downward a bit, would give you either Baylor and TCU (no Oregon) in the #10-team version or Oregon and TCU in the #25-team version. The issue with Oregon is that the high baseline we set combined with Sagarin’s view of the Pac-12–lots of good but few great teams–makes Oregon’s schedule look relatively easy for a top 10 or even top 25 team.

I think there are good reasons for moving Mississippi State down, but this is a good reminder that teams shouldn’t be automatically excluded simply because they have more losses than another team. An extremely tough schedule can be enough to account for the extra loss.

Also, remember that conference championships, head-to-head, margin of victory, and the “eye test” are not included here, but those are things the committee could consider. Most of those things would not work in Mississippi State’s favor.

It will be interesting to see what the committee does today, both in selection and seeding. And then we all get to enjoy college football’s first playoff on the field, which promises to be exciting no matter who is selected.


Comment » | CFB Achievement Rankings, CFB Playoff, College Football, Football, team evaluation

NCAA Tournament Predictions – 2013

March 21st, 2013 — 2:29pm

With the tournament under way, I wanted to post my NCAA Tournament predictions. Things didn’t go so well for me with my Conference Tournament predictions, so hopefully the big dance will provide some sort of redemption.

I really hate the traditional bracket with normal scoring rules, as the best bracket ends up just being pretty much chalk and, well, what’s the fun in that? However, I’m guessing most people want to see my “bracket” so I’ll provide it. It’s really unexciting: only two double-digit seeds are favored by my system in the first round–11-seeds St. Mary’s and Minnesota–and there are only a couple more mild upsets along the way.

2013 March Madness Bracket

There’s a lot of information in predictive systems like mine, but this bracket shows virtually none of it. A better way to display all of the information is with advancement odds, like I did for conference tournaments. Here is the likelihood of each team advancing to each round of the tournament.

RgSdTeamRtgRkRd of 32Sweet 16Elite 8Final 4Champ GameChamp
42Ohio State96.1691.668.446.824.012.85.8
13Michigan State93.61475.244.722.
32Miami (FL)92.31990.957.235.310.23.51.2
14Saint Louis92.22077.045.413.
111Saint Mary's (CA)92.61670.036.917.
18Colorado State92.81555.514.
43New Mexico91.12482.038.514.
25Virginia Commonwealth91.32272.
15Oklahoma State90.32956.628.
47Notre Dame90.62659.
27San Diego State90.62765.331.
35Nevada-Las Vegas90.03069.328.
28North Carolina87.63655.
38North Carolina State87.13856.
49Wichita State89.93128.
410Iowa State87.03940.910.
44Kansas State85.64357.514.
413La Salle81.55942.
113New Mexico State78.17323.
213South Dakota State68.51007.
216Western Kentucky51.81556.
115Albany (NY)52.31536.
215Florida Gulf Coast48.51697.
214Northwestern State56.11403.
316James Madison48.31711.
116North Carolina A&T21.82730.

The table is fully searchable, sortable, and filterable. I added in the region and seed so you can sort and look at best/worst teams by seed and region.

For now, it’s time to finally enjoy the games.


Comment » | College Basketball, March Madness, predictive, team evaluation

Winners and Losers from Selection Sunday

March 19th, 2013 — 12:20am

Despite what many television analysts might say, seeding does have an enormous impact on a team’s chances to advance in the tournament. Every seed line you move up increases your chances of going further in the tournament. But the seeds don’t always play out that way, and so when the bracket is released we can see exactly what matchups each team will face on their path through the tournament.


Indiana has a clear path to the Final Four

The Hoosiers head the easiest of the four regions. Their 2nd round opponent will be the easiest of the 8/9 matchups (NC State or Temple). In the Sweet 16, Syracuse could provide a stiff test but each other region has a 4 or 5 seed as good or better than the Orange. And the bottom half of Indiana’s bracket is by far the easiest of any region: Miami is the worst 2-seed, Marquette is the worst 3-seed (along with New Mexico) and none of the other teams provide much of a threat. Nobody is ever a shoo-in for the Final Four, there’s too many games against too many good teams, but Indiana definitely increased their odds on Sunday with the path they were dealt.

Also benefiting from this easy bracket is 6-seed Butler, who has a relatively easy path to the Elite 8. Could they shock the world…again…and make it to the Final Four? Continue reading »

Comment » | College Basketball, March Madness, predictive, team evaluation

The Achievement S-Curve – 2013 Final

March 18th, 2013 — 9:51pm

Selection Sunday 2013 is in the books. Time to release the final Achievement S-Curve of 2013 and see how it compares to the actual bracket.

The 2013 Achievement S-Curve (click twice to embiggen):

Achievement S-Curve 130318 Continue reading »

2 comments » | College Basketball, descriptive, March Madness, predictive, review, team evaluation

Conference Tournament Predictions – Final Results

March 18th, 2013 — 5:13pm

Last time, I laid out the method by which I would grade the conference tournament predictions.

The tournaments are over so it’s time to present the results…and it’s not pretty for my predictions.

TeamRankings: -317.53

KenPom: -317.87

Predict The Madness (me): -321.26

Yeah, the latter half of the conference tournaments did not go so well for my system. It’s not the biggest sample, but around 300 games gives us some indication. Perhaps next year I’ll grade all regular season predictions.

And now, it’s time for the real tournament. Enjoy the madness.

Comment » | College Basketball, Conference Tournament predictions, predictive, team evaluation

Conference Tournament Predictions – Update 3/14/2013

March 14th, 2013 — 1:17am

As I laid out in my introductory post, I am laying out my conference tournament predictions in order to compare them to other predictions out there. The two that I know of are Ken Pomery’s Log5 predictions and TeamRankings predictions.

In that first post, I proposed a sum of squared errors measure to score each system. After talking with multiple people much smarter and more well-versed in this area than me, I settled on using a logarithmic scoring rule. One way to grade each system’s predictions would be to apply the log to the “winning” probability of each game (for instance if the winning team was given a 75% chance to win, the score for that game would be the log(.75); if the other team were to win, the score would instead be log(.25)). However, each set of predictions simply gives the probability of each team advancing to each round, so we don’t have individual game probabilities. As a replacement, I decided to grade each team based on the predicted odds that they would go exactly as far as they did. Say Team A won their 1st round and quarterfinal games but lost in the semifinals. If the prediction said they had a 75% to make it to the semifinals and a 50% chance to win in the semifinals, then the chance that they win the quarters but lose the semis is 75% – 50% = 25%. Thus, the score for Team A is log(.25). This double counts games, but it double counts every game, so there shouldn’t be a bias. Continue reading »

5 comments » | College Basketball, Conference Tournament predictions, predictive, review, simulation, team evaluation

ACC, Big Ten, A-10, Big West, Great West, and Big Sky Conference Tournament Predictions – 2013

March 12th, 2013 — 11:43pm

Click here to check out all the conference tournament predictions.

In the final installment of my conference tournament predictions, we look at the remaining 6 conferences. They range from the Big  Ten–one of the best conferences of recent memory–to the Great West, at 5-team conference which doesn’t even receive an automatic bid to the NCAA Tournament.

In the ACC, it’s Duke out in front…again. NC State was a disappointment, while Miami was a surprise regular season champion. However, with Ryan Kelly back, the Hurricanes are back to doing what the rest of the ACC is used to doing: trying to catch Duke.

1Miami (FL)75.948.121.8
3North Carolina71.923.211.8
5North Carolina State83.545.222.57.9
8Boston College60.316.14.50.8
9Georgia Tech39.
6Florida State44.411.11.50.4
10Wake Forest29.
12Virginia Tech16.

The Big Ten was an absolute gauntlet this season. My predictive rankings have 6 teams from the Big Ten in the top 12. That’s right, half of my top 12 are from one conference. Throw in Iowa (27) and Illinois (32) and two-thirds of the conference are 2nd-round-caliber teams. Indiana is the class of this conference, which speaks to just how good the Hoosiers are this year. Unfortunately for them and Michigan and Wisconsin, the three best teams in the conference reside in the same half of the bracket. That gives 2-seed Ohio State a (relatively) easier path to the final and the most likely team to win the tournament should Indiana falter. Whatever happens, it should be an entertaining weekend.

2Ohio State86.054.722.6
3Michigan State65.029.38.2
12Penn State5.

The Atlantic-10 looks like a two-team tournament, with Saint Louis and VCU on a collision path toward the final. Three teams are lurking–Temple, Butler, and La Salle–hoping to crash the party. Those 5 teams may all find themselves in the big dance should things fall into place this week.

1Saint Louis80.955.533.2
2Virginia Commonwealth76.054.832.5
4La Salle46.215.06.3
10Saint Joseph's45.
11George Washington55.319.14.91.0

Perhaps the most balanced conference tournament of all this year is the Big West. The favorite, Pacific, isn’t even a 1-in-4 chance to win it all, while UCSB is the biggest longshot but still has a 4% chance to pull it off.

3Cal Poly63.434.218.5
1Long Beach State58.036.317.8
8Cal State Fullerton42.014.86.3
7California-Santa Barbara30.311.74.1

I’m not even going to say anything about the Great West, but here are the predictions.

3Chicago State63.831.3
5Utah Valley69.334.418.1
2Texas-Pan American36.214.5
4Houston Baptist30.79.53.8

It feels like every year, but once again Montana and Weber State are the overwhelming favorite to meet for the Big Sky championship. My predictive rankings actually rank Weber State as a better team than Montana (#126 versus #188), but the conference gives the #1 seed every advantage possible. The whole tournament is on Montana’s home floor, only the top 7 teams qualify meaning the #1 seed receives a bye to the semifinals and only has to win two games, and the semifinals are re-seeded so should an upset occur in the first round Montana will be the beneficiary. All of that puts the Grizzlies right at 50/50 with Weber State close by at 41%.

2Weber State86.672.740.8
5Northern Colorado51.711.63.0
3North Dakota64.913.52.8
4Montana State48.310.82.8
7Northern Arizona13.41.40.4
6Southern Utah35.12.80.2

That does it for this year’s conference tournament predictions. I’ll report back after the conclusion of all the tournaments with how my predictions fared against KenPom and TeamRankings (and maybe I’ll report some intermediate results some time this week).

Comment » | College Basketball, Conference Tournament predictions, predictive, simulation, team evaluation

Back to top