Evaluating QBs: The Truth Behind QB Records

*UPDATE: I’ve temporarily changed the blog theme so that the tables in this post will be sortable and searchable.*

With the tedious boring stuff out of the way (if you missed the boring parts, here is boring part 1 and part 2), it’s time for the payoff. I’ll post some results and comment on some of the more interesting findings.

First, the caveats, the fine print. All games from 2000-2012 are included, regular season is assumed unless otherwise noted. From last post, we defined the “QB of record” for each game; that is instead of the starting QB we’ll use the QB who had the most dropbacks for his team in each game (dropbacks = pass attempts + sacks). Again from the previous posts, we defined different phases of the game, which we’ll measure by Expected Points Added (EPA)–despite having my own expected points model, I decided to borrow Brian Burke’s more well-known EP model for this series. Those phases are defense, special teams and offense; most of the time here we’ll be dividing offense into two parts: QB EPA, which are plays where the QB is the passer or rusher, and Non-QB EPA which is all other offensive plays. While part 1 showed that QBs have control over QB EPA but little to no influence over Non-QB EPA, Defensive EPA, or Special Teams EPA that should not be confused with QBs having all control over QB EPA. While that is heavily influenced by the quarterback, receivers, lineman, running backs, the opposing defense, etc. all have some impact as well on these plays.

With the disclaimers out of the way, let’s dive right in.

The Best QBs of the 2000s

While I will save most of the Peyton Manning/Tom Brady analysis for the final post of this series, there’s no way around mentioning them when looking at the best quarterbacks of this millennium. To start, we’ll look at xWin% (reminder from last post, xWin% is a quarterback’s expected winning percentage based on how well he played in that game measured by QB EPA; in other words, how often should we expect the QB’s team to win based on his performance?) for all games from 2000-2013. We’ll set the cutoff at 50 games, or about 3 full seasons of play.

QuarterbackxWin%ActualsWin%
Peyton Manning0.6860.7270.483
Aaron Rodgers0.6800.6670.536
Tom Brady0.6400.7800.564
Rich Gannon0.6230.6430.494
Drew Brees0.6150.5890.476
Tony Romo0.6030.5980.478
Matt Ryan0.6020.7110.513
Philip Rivers0.5880.6160.534
Matt Schaub0.5860.5560.470
Jeff Garcia0.5720.5240.479

Peyton Manning comes out on top, followed closely by the newest member of the uber-elite QB class, Aaron Rodgers. Tom Brady comes in 3rd, but with a sizable gap between him and our top two. A couple important things: Manning’s career started in 1998 so his rookie and 2nd seasons are not included here. While he had a strong sophomore campaign his rookie year had some growing pains and would certainly drag him down in these rankings. Brady meanwhile, has his worst years from the beginning of his career included here. Rodgers, on the other hand, has the opposite bias: he is still in his prime and hasn’t had to deal with the potential downslope of his career nor sustain his excellence for as long (he also sat behind Brett Favre which both saved him from poor early seasons and hindered his ability to start racking up stats). Regression to the mean is a nasty thing that is nearly impossible to avoid, and it is certainly more impressive for Brady and Manning to sustain excellence over twice as many games. Looking at total expected wins shows Manning (128.3 in 187 games) and Brady (110.7 in 173 games) far ahead of the pack. Brees (103.3 in 168 games) is third with a large gap between those 3 and 4th place Brett Favre.

Those top 3 QBs (Manning, Brady, Rodgers) each own one of the 3 elite QB seasons of past decade-plus. Brady’s 2007 campaign ranks 3rd with and expected win % of 81.4%; Manning’s 2004 is 2nd at 82.2% and Rodgers’ 2011 is the best of the bunch at 82.5%. No other quarterback-season since 2000 tops 77%.

Following the big 3 we get the tail end of Rich Gannon’s career: his best years came in OAK to finish his career from 1999 on, factoring in his earlier years would drop him down the list. Brees has at times been the 3rd amigo with Manning and Brady but he hasn’t had quite the sustained excellence of his peers. Romo, Ryan, Rivers, Schaub, and Garcia round out the top 10. Some might be surprised to see Schaub so high after what some deemed a disappointing season. Still, he ranked in the top 10 in EPA, EPA/Play, and WPA this season, and has done so each of the past 4 years. On the other hand, Roethlisberger just misses out on the top 10 checking in at #11. While his actual .685 win percentage is impressive, he is the beneficiary of the best support of any QB in the top 15. An average QB would be expected to nearly 60% of his games given the support Big Ben got in his career. Schaub’s 47.0% support win %, meanwhile, is the worst of the top 10.

What’s Worse than Bad Play? Promise

Let’s take a peak at the other side of this list, the bottom 7. It takes some special circumstances to rack up 50+ starts with well-below average performances but our magnificent seven were able to do so while posting xWin%’s below 45%. The only thing worse than being bad is having promise that you might get better.

QuarterbackxWin%ActualsWin%
Joey Harrington0.3970.3660.499
Mark Sanchez0.4030.5250.624
Alex Smith0.4180.5140.576
David Carr0.4270.2950.393
Kyle Orton0.4460.4850.518
Vinny Testaverde0.4470.4920.497
Ryan Fitzpatrick0.4490.3770.436

There’s some cautionary tales in that list. Let’s start with Fitzpatrick. After years of poor play–he never posted an xWin% higher than 50%–Fitzpatrick had a strong start to his 2011 season with an xWin% of 67.5% through the first 6 games. The Bills were 4-2 including a huge win over the Patriots. Buffalo got excited, slapped a big check in front of him ($24 million in guaranteed money), and then saw him post a dismal 36.6% xWin% the rest of the season. He followed that up with yet another below-average season (48% xWin% in 2012) and now the Bills are likely looking to start all over.

Kyle Orton is an interesting case. He was a terrible QB in Chicago, with 31 games and an xWin% south of 40%. However, he received tremendous support–his sWin% (the winning percentage we’d expect for a QB given how well the defense, special teams, and running game played) was over 60% each season with the Bears lifting “his” record to a respectable 19-12. Traded to Denver, Orton actually played alright with xWin%’s around 50%, but received terrible support from the Broncos including an unthinkable 25.6% sWin% in 2010. Even the Mannings and Bradys of the world would severely struggle to get a team with that support up to even .500.

David Carr and Joey Harrington are typical cases of top picks getting tons of chances but Alex Smith is one who seems to have actually gotten “it”. But is that true? Here is a chart of his xWin%, sWin%, and actual Win% for each season of his career.

SeasonxWin%ActualsWin%
20050.2440.2860.471
20060.4270.4380.446
20070.2450.3330.523
20090.4240.4550.633
20100.4340.3000.501
20110.4830.8130.703
20120.5230.7500.733

While Smith has improved from the terrible start to his career (two seasons with an xWin% below 25!), he is still just a mediocre QB. What has changed, however, is the support he’s gotten. He’s posted a 19-5 record the past two seasons but most of the credit goes to the best defense over the past two years as SF has put up sWin%’s over 70% for Alex Smith since 2011. While Smith may be able to sustain his average play, it is likely that he will be a disappointment once separated from his dominating defense. Buyer beware.

Oh Mark Sanchez. His signing cost one guy his job, and it likely should have been avoided. Sanchez actually is a double warning: (1) beware of QBs with gaudy records fueled by their support and (2) do not overreact to small sample sizes. In the regular season, Sanchez has been a horrendous QB posting xWin%’s of 40%, 46%, 42%, and finally 33% this year. However, the Jets have had strong defenses, solid running games, and above average special teams units helping to mask Sanchez’s poor play. Despite xWin%’s in the low 40s, Sanchez managed to win at least half his games in each of his first 3 seasons. In the playoffs, Sanchez has actually played rather well in leading his teams to the AFC Championship game two straight years. His xWin% was 56% in 2009 and 58% in 2010 while his support was just average (sWin% of 48% and 51% respectively). However, 46 games of regular season play are a much better indicator of his talent than 6 playoff games and the Jets will pay (literally) for not acknowledging that.

Clutch or Lucky?

While QBs are judged on their regular season record, the ultimate criteria people use are “rings”. We’ll see just how misleading some quarterbacks postseason resumes are.

Here are the quarterbacks (min. 4 playoff games) who received the best support in their playoff careers.

QuarterbackxWin%ActualsWin%
Trent Dilfer0.4531.0000.876
Rex Grossman0.2620.5000.731
Brad Johnson0.6010.7500.702
Joe Flacco0.4890.6670.679
Rich Gannon0.5050.5710.607

No surprise, Trent Dilfer leads the list thanks to Baltimore’s dominating defense during the 2000 playoffs. The Ravens defense averaged–averaged–(-22.63) EPA during their Super Bowl run (remember, negative EPA is good for the defense). That means in every game they held the offense over 3 TDs below what a normal offense would score, and in the playoffs you’re playing against better offenses than normal. Dilfer for his part was ok–a 45% xWin%–and that was good enough, but nobody should mistake him for a playoff stud.

While most people acknowledge that Dilfer (and to a lesser extent the next two QBs on the list–Brad Johnson and Rex Grossman) were the beneficiaries of incredible support, the other Ravens QB on the list is garnering all the accolades. That’s right, Mr. Elite himself: Joe Flacco. While Flacco has actually played incredibly well this postseason with an xWin% of 77%, he has been below average in each of his previous 4 playoff visits with xWin%’s of 39%, 31%, 41%, and 49%. However, the Ravens supported him incredibly well with sWin%’s of 82%, 61%, 79%, and 52%…that is the reason Flacco and the Ravens were able to post a 5-4 record prior to this season. This year, the support has continued (62%) but Flacco is finally holding up his end of the bargain and the Ravens have benefited with a trip to the Super Bowl.

However, remember the cautionary tale of Mark Sanchez earlier. While Flacco has been better in the regular season, he has been worse in the playoffs. In both cases, his win-loss record is a mirage sustained by the incredible defense, special teams, and running game Baltimore gives him. Thus far in his career Flacco’s record is a stellar 54-25, a 68.4% winning percentage. But a look at his sWin% of 64.6% shows that he has only been able to slightly outperform what any average QB could do given the rest of the Ravens team. Flacco’s own xWin% bears that out, as he’s posted only a slightly above average 52.6%. So given 79 regular season games and 9 more playoff starts prior to this postseason we have overwhelming evidence that Joe Flacco is an average quarterback buoyed by a strong defense and running game. Would you pay Manning/Brady/Brees money for that resume based on 3 outstanding postseason games?

Rising to the Occasion

Here are the top four postseason QBs (min. 4 playoff games) of this century, the only 4 to post xWin%’s of 60% or higher (you’ll see why I cut it off at 4 in the next post…hint hint).

QuarterbackxWin%ActualsWin%
Aaron Rodgers0.6920.6250.399
Drew Brees0.6740.5560.379
Brad Johnson0.6010.7500.702
Kurt Warner0.6000.6000.440

Rodgers and Brees have been outstanding and while they both have a ring to show for it, they also both have had some postseasons cut short due to lack of support. In fact, they are 2 of the 6 quarterbacks with sWin% numbers under 40% in the playoffs. Matt Ryan tops the list; while he hasn’t played great in his 5 playoff games (44.3% xWin%), he has really suffered from terrible defense and special teams (Matt Bryant’s long game-winner notwithstanding) as the Falcons have posted an sWin% of 17.4%. Remember that next time somebody tells you Matt Ryan is a choker…Superman himself would have trouble winning games with that kind of defense.

Final Thoughts

A couple last tidbits that some might be curious about. Who’s been the worst playoff quarterback? It’s not even close, Rex Grossman’s 26% xWin% takes the cake. Amazingly, Chicago’s defense was able to eke out a couple playoff wins in spite of Grossman’s play. Honorable mention to one other QB who didn’t meet the 4-game cutoff, Andy Dalton, who has an xWin% just over 20% in his two playoff starts.

Lastly, what about Tim Tebow? In his first 3-game stint in 2010, Tebow actually played well with a strong 67% xWin%. However, the Broncos support hurt (sWin% of 21%) Tebow’s record and he finished 1-2. The magical 2011 season was a different story. This time, Tebow was a poor 35% xWin% but the rest of the team picked up the slack with a 55% sWin%. Still, the team’s actual record of 7-4 (64% win %) actually outperformed the sWin% despite Tebow’s poor xWin%. I guess there is such thing as a Tebow miracle after all.

Finally, I am obligated to mention Colin Kaepernick since it is Super Bowl week. In his 7 regular season games this year he was very good with an xWin% of 62% and he also benefited from the 49ers defense, special teams and running game, with a 61% sWin%. With a minimum of 5 games, his xWin% would place him 7th in the set.

Here is a list of regular season performance for all quarterbacks with at least 5 games since 2000. “xWin% diff” is the difference between actual and xWin%, higher numbers mean the QB actually won more than expected based on his performance and usually this implies a higher sWin% (i.e. the QB was lucky). “sWin% diff” is the difference between actual and sWin%, higher numbers mean the QB actually won more than expected based on the support he received and usually implies a higher xWin% (i.e. the better the QB).

*UPDATE: The table below is now fully sortable and searchable. You can sort by any column or type part of a player’s name into the search box to find a specific quarterback.*

QuarterbackGamesActualxWin%xWin% diffsWin%sWin% diff
Peyton Manning1870.7270.6860.0410.4830.244
Aaron Rodgers780.6670.680-0.0130.5360.131
Robert Griffin III150.6000.650-0.0500.4230.177
Tom Brady1730.7800.6400.1410.5640.217
Alex Van Pelt80.2500.634-0.3840.269-0.019
Rich Gannon560.6430.6230.0200.4940.149
Colin Kaepernick70.7140.6210.0930.6140.100
Drew Brees1680.5890.615-0.0250.4760.113
Cam Newton320.4060.612-0.2060.4060.000
Jim Harbaugh50.0000.607-0.6070.297-0.297
Tony Romo920.5980.603-0.0050.4780.120
Matt Ryan760.7110.6020.1080.5130.198
Quinn Gray60.3330.590-0.2570.2850.048
Philip Rivers1120.6160.5880.0280.5340.082
Matt Schaub810.5560.586-0.0300.4700.086
Russell Wilson160.6880.5720.1150.5990.088
Jeff Garcia1030.5240.572-0.0480.4790.046
Ben Roethlisberger1270.6850.5700.1150.5920.093
Daunte Culpepper960.4270.570-0.1430.3770.050
Trent Green980.5100.564-0.0540.4680.043
Kurt Warner970.5770.5620.0150.5260.052
Doug Flutie270.4440.559-0.1150.496-0.051
Chad Pennington830.5180.554-0.0360.4780.040
Donovan McNabb1490.6310.5520.0790.6030.028
Steve McNair980.6120.5490.0630.5140.098
Brett Favre1670.5990.5370.0610.5420.057
Andrew Luck160.6880.5350.1530.3990.288
Elvis Grbac280.5360.5340.0020.5080.027
Jay Fiedler570.6320.5310.1010.5500.081
Joe Flacco790.6840.5260.1580.6460.038
David Garrard750.5330.5230.0100.5130.021
Sage Rosenfels160.4380.522-0.0850.4360.001
Eli Manning1340.5750.5200.0550.5240.050
Carson Palmer1200.4500.519-0.0690.454-0.004
Matt Hasselbeck1480.5470.5170.0300.5050.042
Michael Vick980.5610.5160.0450.5070.055
Randall Cunningham80.3750.515-0.1400.3480.027
Matthew Stafford440.3860.514-0.1270.419-0.033
Shaun Hill270.4810.513-0.0320.4690.013
Jake Plummer990.5350.5120.0240.5090.026
Marc Bulger890.4490.508-0.0590.4270.022
Jay Cutler910.5600.5070.0540.5020.058
Brian Griese690.5650.5060.0590.5510.014
Kordell Stewart400.6500.5050.1450.6040.046
Brad Johnson700.5860.5040.0820.613-0.027
Todd Collins51.0000.4990.5010.8220.178
Aaron Brooks900.4220.497-0.0740.4220.001
Jake Delhomme970.5770.4950.0830.5530.024
Jim Sorgi50.4000.494-0.0940.447-0.047
Vince Young480.6040.4940.1100.4910.114
Kerry Collins1260.4600.491-0.0310.481-0.020
Troy Aikman80.3750.487-0.1120.434-0.059
Mark Brunell810.4440.485-0.0410.535-0.091
Jason Campbell690.4200.484-0.0640.483-0.063
Charlie Batch310.4840.4840.0000.529-0.046
Billy Volek150.3330.482-0.1490.363-0.029
Drew Bledsoe860.4650.479-0.0140.536-0.071
Shaun King200.5000.4790.0210.588-0.088
Todd Bouman60.1670.478-0.3120.333-0.167
Josh Freeman560.4290.478-0.0490.445-0.016
Jon Kitna1080.3520.473-0.1210.393-0.041
Dan Orlovsky130.1540.473-0.3190.305-0.151
Kent Graham50.4000.471-0.0710.569-0.169
Jake Locker110.3640.470-0.1070.416-0.052
Matt Leinart170.2940.470-0.1760.405-0.111
Andy Dalton320.5940.4680.1260.598-0.005
Steve Beuerlein230.4350.467-0.0320.524-0.089
Gus Frerotte430.5810.4670.1150.5760.005
Byron Leftwich490.4690.4640.0050.489-0.019
Matt Cassel650.4620.464-0.0030.502-0.040
Chris Chandler350.4290.463-0.0350.3980.031
Ryan Tannehill150.4000.460-0.0600.495-0.095
Jeff Blake310.4520.457-0.0050.460-0.009
Damon Huard240.4580.4560.0020.489-0.031
Tommy Maddox300.5000.4540.0460.594-0.094
Tyler Thigpen140.0710.454-0.3830.320-0.248
Matt Moore260.5380.4540.0850.610-0.072
Chris Redman140.4290.453-0.0250.3570.071
Josh McCown340.3530.452-0.0990.459-0.106
Jim Miller240.6250.4510.1740.642-0.017
Ryan Fitzpatrick690.3770.449-0.0720.436-0.059
Chad Henne390.3850.448-0.0640.469-0.084
Vinny Testaverde630.4920.4470.0450.497-0.005
Kyle Orton680.4850.4460.0390.518-0.033
Rob Johnson190.3680.446-0.0770.412-0.044
Cleo Lemon110.0910.446-0.3550.312-0.221
Scott Mitchell60.3330.444-0.1100.3290.005
Tony Banks270.5560.4430.1120.604-0.048
Tim Rattay210.2380.440-0.2020.354-0.116
Tarvaris Jackson380.5000.4390.0610.603-0.103
Kelly Holcomb240.3330.438-0.1050.478-0.145
Jeff George70.1430.437-0.2940.247-0.104
Tim Couch430.4420.4360.0060.488-0.046
Troy Smith80.5000.4360.0640.4170.083
Christian Ponder250.4800.4350.0450.497-0.017
Rex Grossman470.5960.4330.1630.637-0.041
Seneca Wallace230.2610.432-0.1710.449-0.189
Kevin Kolb210.3810.431-0.0500.505-0.124
Colt McCoy210.2860.430-0.1450.420-0.134
David Carr780.2950.427-0.1320.393-0.098
Quincy Carter330.5450.4260.1200.564-0.019
Shane Matthews160.5000.4250.0750.4560.044
Brandon Weeden150.3330.419-0.0850.461-0.128
Tim Tebow140.5710.4180.1530.4770.095
Rick Mirer100.2000.418-0.2180.364-0.164
Alex Smith740.5140.4180.0960.576-0.063
Rodney Peete140.5000.4170.0830.590-0.090
Jamie Martin120.3330.416-0.0830.372-0.039
Nick Foles70.1430.415-0.2720.288-0.145
T.J. Yates50.6000.4110.1890.5440.056
Derek Anderson460.4130.4100.0030.443-0.029
Trent Dilfer370.5140.4060.1080.598-0.085
Trent Edwards330.3940.405-0.0110.497-0.103
Patrick Ramsey270.3700.404-0.0330.502-0.131
Mark Sanchez610.5250.4030.1210.624-0.100
Kellen Clemens110.3640.403-0.0390.503-0.139
Brooks Bollinger120.1670.399-0.2320.352-0.186
Joey Harrington710.3660.397-0.0300.499-0.132
Tim Hasselbeck60.1670.394-0.2270.464-0.298
Mike McMahon170.1760.393-0.2160.317-0.140
J.P. Losman340.3240.386-0.0620.431-0.107
Sam Bradford410.3660.382-0.0160.494-0.129
Ray Lucas70.2860.380-0.0950.544-0.259
Kyle Boller490.4490.3800.0690.572-0.123
Curtis Painter80.0000.374-0.3740.222-0.222
Cade McNown80.1250.373-0.2480.289-0.164
Doug Johnson130.1540.370-0.2160.325-0.171
J.T. O'Sullivan70.2860.369-0.0830.502-0.216
Chris Simms150.4000.3690.0310.555-0.155
Anthony Wright220.4090.3670.0430.574-0.165
Drew Stanton70.2860.365-0.0790.432-0.146
A.J. Feeley210.4760.3610.1150.603-0.127
Brodie Croyle90.0000.355-0.3550.321-0.321
Bethel Johnson130.4620.3540.1070.629-0.168
Bruce Gradkowski210.3330.350-0.0160.383-0.050
Charlie Frye210.2860.342-0.0560.463-0.178
Chris Weinke190.1050.335-0.2290.445-0.339
JaMarcus Russell250.2800.330-0.0500.401-0.121
Josh Johnson50.0000.330-0.3300.287-0.287
Brady Quinn180.2220.328-0.1050.397-0.175
Doug Pederson80.2500.326-0.0760.398-0.148
John Skelton160.5630.3170.2450.606-0.044
Blaine Gabbert220.2270.310-0.0830.474-0.247
Danny Kanell60.0000.307-0.3070.429-0.429
Luke McCown100.2000.301-0.1010.439-0.239
Ken Dorsey120.1670.289-0.1220.449-0.283
Chad Hutchinson140.2140.258-0.0430.559-0.345
Ryan Leaf120.0830.254-0.1710.498-0.415
Jimmy Clausen110.0910.244-0.1530.458-0.368
Akili Smith110.1820.243-0.0610.475-0.293
Jonathan Quinn50.0000.223-0.2230.549-0.549
John Beck60.0000.218-0.2180.462-0.462
Andrew Walter110.1820.202-0.0210.599-0.417
Spergon Wynn50.0000.189-0.1890.409-0.409
Ryan Lindley50.2000.1540.0460.684-0.484
Craig Krenzel50.6000.1080.4920.711-0.111
This entry was posted in descriptive, Evaluating QBs Series, Football, offense versus defense, player evaluation, talent distribution, team evaluation and tagged , , , , , , , , , , . Bookmark the permalink.

1 Response to Evaluating QBs: The Truth Behind QB Records

  1. Hi admin, do you monetize your site ? There is easy way to earn extra money every day, just search on youtube – How to earn $25/hour selling articles

Leave a Reply

Your email address will not be published.