Evaluating QBs: The Truth Behind QB Records

*UPDATE: I’ve temporarily changed the blog theme so that the tables in this post will be sortable and searchable.*

With the tedious boring stuff out of the way (if you missed the boring parts, here is boring part 1 and part 2), it’s time for the payoff. I’ll post some results and comment on some of the more interesting findings.

First, the caveats, the fine print. All games from 2000-2012 are included, regular season is assumed unless otherwise noted. From last post, we defined the “QB of record” for each game; that is instead of the starting QB we’ll use the QB who had the most dropbacks for his team in each game (dropbacks = pass attempts + sacks). Again from the previous posts, we defined different phases of the game, which we’ll measure by Expected Points Added (EPA)–despite having my own expected points model, I decided to borrow Brian Burke’s more well-known EP model for this series. Those phases are defense, special teams and offense; most of the time here we’ll be dividing offense into two parts: QB EPA, which are plays where the QB is the passer or rusher, and Non-QB EPA which is all other offensive plays. While part 1 showed that QBs have control over QB EPA but little to no influence over Non-QB EPA, Defensive EPA, or Special Teams EPA that should not be confused with QBs having all control over QB EPA. While that is heavily influenced by the quarterback, receivers, lineman, running backs, the opposing defense, etc. all have some impact as well on these plays.

With the disclaimers out of the way, let’s dive right in.

The Best QBs of the 2000s

While I will save most of the Peyton Manning/Tom Brady analysis for the final post of this series, there’s no way around mentioning them when looking at the best quarterbacks of this millennium. To start, we’ll look at xWin% (reminder from last post, xWin% is a quarterback’s expected winning percentage based on how well he played in that game measured by QB EPA; in other words, how often should we expect the QB’s team to win based on his performance?) for all games from 2000-2013. We’ll set the cutoff at 50 games, or about 3 full seasons of play.

Peyton Manning comes out on top, followed closely by the newest member of the uber-elite QB class, Aaron Rodgers. Tom Brady comes in 3rd, but with a sizable gap between him and our top two. A couple important things: Manning’s career started in 1998 so his rookie and 2nd seasons are not included here. While he had a strong sophomore campaign his rookie year had some growing pains and would certainly drag him down in these rankings. Brady meanwhile, has his worst years from the beginning of his career included here. Rodgers, on the other hand, has the opposite bias: he is still in his prime and hasn’t had to deal with the potential downslope of his career nor sustain his excellence for as long (he also sat behind Brett Favre which both saved him from poor early seasons and hindered his ability to start racking up stats). Regression to the mean is a nasty thing that is nearly impossible to avoid, and it is certainly more impressive for Brady and Manning to sustain excellence over twice as many games. Looking at total expected wins shows Manning (128.3 in 187 games) and Brady (110.7 in 173 games) far ahead of the pack. Brees (103.3 in 168 games) is third with a large gap between those 3 and 4th place Brett Favre.

Those top 3 QBs (Manning, Brady, Rodgers) each own one of the 3 elite QB seasons of past decade-plus. Brady’s 2007 campaign ranks 3rd with and expected win % of 81.4%; Manning’s 2004 is 2nd at 82.2% and Rodgers’ 2011 is the best of the bunch at 82.5%. No other quarterback-season since 2000 tops 77%.

Following the big 3 we get the tail end of Rich Gannon’s career: his best years came in OAK to finish his career from 1999 on, factoring in his earlier years would drop him down the list. Brees has at times been the 3rd amigo with Manning and Brady but he hasn’t had quite the sustained excellence of his peers. Romo, Ryan, Rivers, Schaub, and Garcia round out the top 10. Some might be surprised to see Schaub so high after what some deemed a disappointing season. Still, he ranked in the top 10 in EPA, EPA/Play, and WPA this season, and has done so each of the past 4 years. On the other hand, Roethlisberger just misses out on the top 10 checking in at #11. While his actual .685 win percentage is impressive, he is the beneficiary of the best support of any QB in the top 15. An average QB would be expected to nearly 60% of his games given the support Big Ben got in his career. Schaub’s 47.0% support win %, meanwhile, is the worst of the top 10.

What’s Worse than Bad Play? Promise

Let’s take a peak at the other side of this list, the bottom 7. It takes some special circumstances to rack up 50+ starts with well-below average performances but our magnificent seven were able to do so while posting xWin%’s below 45%. The only thing worse than being bad is having promise that you might get better.

There’s some cautionary tales in that list. Let’s start with Fitzpatrick. After years of poor play–he never posted an xWin% higher than 50%–Fitzpatrick had a strong start to his 2011 season with an xWin% of 67.5% through the first 6 games. The Bills were 4-2 including a huge win over the Patriots. Buffalo got excited, slapped a big check in front of him ($24 million in guaranteed money), and then saw him post a dismal 36.6% xWin% the rest of the season. He followed that up with yet another below-average season (48% xWin% in 2012) and now the Bills are likely looking to start all over.

Kyle Orton is an interesting case. He was a terrible QB in Chicago, with 31 games and an xWin% south of 40%. However, he received tremendous support–his sWin% (the winning percentage we’d expect for a QB given how well the defense, special teams, and running game played) was over 60% each season with the Bears lifting “his” record to a respectable 19-12. Traded to Denver, Orton actually played alright with xWin%’s around 50%, but received terrible support from the Broncos including an unthinkable 25.6% sWin% in 2010. Even the Mannings and Bradys of the world would severely struggle to get a team with that support up to even .500.

David Carr and Joey Harrington are typical cases of top picks getting tons of chances but Alex Smith is one who seems to have actually gotten “it”. But is that true? Here is a chart of his xWin%, sWin%, and actual Win% for each season of his career.

While Smith has improved from the terrible start to his career (two seasons with an xWin% below 25!), he is still just a mediocre QB. What has changed, however, is the support he’s gotten. He’s posted a 19-5 record the past two seasons but most of the credit goes to the best defense over the past two years as SF has put up sWin%’s over 70% for Alex Smith since 2011. While Smith may be able to sustain his average play, it is likely that he will be a disappointment once separated from his dominating defense. Buyer beware.

Oh Mark Sanchez. His signing cost one guy his job, and it likely should have been avoided. Sanchez actually is a double warning: (1) beware of QBs with gaudy records fueled by their support and (2) do not overreact to small sample sizes. In the regular season, Sanchez has been a horrendous QB posting xWin%’s of 40%, 46%, 42%, and finally 33% this year. However, the Jets have had strong defenses, solid running games, and above average special teams units helping to mask Sanchez’s poor play. Despite xWin%’s in the low 40s, Sanchez managed to win at least half his games in each of his first 3 seasons. In the playoffs, Sanchez has actually played rather well in leading his teams to the AFC Championship game two straight years. His xWin% was 56% in 2009 and 58% in 2010 while his support was just average (sWin% of 48% and 51% respectively). However, 46 games of regular season play are a much better indicator of his talent than 6 playoff games and the Jets will pay (literally) for not acknowledging that.

Clutch or Lucky?

While QBs are judged on their regular season record, the ultimate criteria people use are “rings”. We’ll see just how misleading some quarterbacks postseason resumes are.

Here are the quarterbacks (min. 4 playoff games) who received the best support in their playoff careers.

No surprise, Trent Dilfer leads the list thanks to Baltimore’s dominating defense during the 2000 playoffs. The Ravens defense averaged–averaged–(-22.63) EPA during their Super Bowl run (remember, negative EPA is good for the defense). That means in every game they held the offense over 3 TDs below what a normal offense would score, and in the playoffs you’re playing against better offenses than normal. Dilfer for his part was ok–a 45% xWin%–and that was good enough, but nobody should mistake him for a playoff stud.

While most people acknowledge that Dilfer (and to a lesser extent the next two QBs on the list–Brad Johnson and Rex Grossman) were the beneficiaries of incredible support, the other Ravens QB on the list is garnering all the accolades. That’s right, Mr. Elite himself: Joe Flacco. While Flacco has actually played incredibly well this postseason with an xWin% of 77%, he has been below average in each of his previous 4 playoff visits with xWin%’s of 39%, 31%, 41%, and 49%. However, the Ravens supported him incredibly well with sWin%’s of 82%, 61%, 79%, and 52%…that is the reason Flacco and the Ravens were able to post a 5-4 record prior to this season. This year, the support has continued (62%) but Flacco is finally holding up his end of the bargain and the Ravens have benefited with a trip to the Super Bowl.

However, remember the cautionary tale of Mark Sanchez earlier. While Flacco has been better in the regular season, he has been worse in the playoffs. In both cases, his win-loss record is a mirage sustained by the incredible defense, special teams, and running game Baltimore gives him. Thus far in his career Flacco’s record is a stellar 54-25, a 68.4% winning percentage. But a look at his sWin% of 64.6% shows that he has only been able to slightly outperform what any average QB could do given the rest of the Ravens team. Flacco’s own xWin% bears that out, as he’s posted only a slightly above average 52.6%. So given 79 regular season games and 9 more playoff starts prior to this postseason we have overwhelming evidence that Joe Flacco is an average quarterback buoyed by a strong defense and running game. Would you pay Manning/Brady/Brees money for that resume based on 3 outstanding postseason games?

Rising to the Occasion

Here are the top four postseason QBs (min. 4 playoff games) of this century, the only 4 to post xWin%’s of 60% or higher (you’ll see why I cut it off at 4 in the next post…hint hint).

Rodgers and Brees have been outstanding and while they both have a ring to show for it, they also both have had some postseasons cut short due to lack of support. In fact, they are 2 of the 6 quarterbacks with sWin% numbers under 40% in the playoffs. Matt Ryan tops the list; while he hasn’t played great in his 5 playoff games (44.3% xWin%), he has really suffered from terrible defense and special teams (Matt Bryant’s long game-winner notwithstanding) as the Falcons have posted an sWin% of 17.4%. Remember that next time somebody tells you Matt Ryan is a choker…Superman himself would have trouble winning games with that kind of defense.

Final Thoughts

A couple last tidbits that some might be curious about. Who’s been the worst playoff quarterback? It’s not even close, Rex Grossman’s 26% xWin% takes the cake. Amazingly, Chicago’s defense was able to eke out a couple playoff wins in spite of Grossman’s play. Honorable mention to one other QB who didn’t meet the 4-game cutoff, Andy Dalton, who has an xWin% just over 20% in his two playoff starts.

Lastly, what about Tim Tebow? In his first 3-game stint in 2010, Tebow actually played well with a strong 67% xWin%. However, the Broncos support hurt (sWin% of 21%) Tebow’s record and he finished 1-2. The magical 2011 season was a different story. This time, Tebow was a poor 35% xWin% but the rest of the team picked up the slack with a 55% sWin%. Still, the team’s actual record of 7-4 (64% win %) actually outperformed the sWin% despite Tebow’s poor xWin%. I guess there is such thing as a Tebow miracle after all.

Finally, I am obligated to mention Colin Kaepernick since it is Super Bowl week. In his 7 regular season games this year he was very good with an xWin% of 62% and he also benefited from the 49ers defense, special teams and running game, with a 61% sWin%. With a minimum of 5 games, his xWin% would place him 7th in the set.

Here is a list of regular season performance for all quarterbacks with at least 5 games since 2000. “xWin% diff” is the difference between actual and xWin%, higher numbers mean the QB actually won more than expected based on his performance and usually this implies a higher sWin% (i.e. the QB was lucky). “sWin% diff” is the difference between actual and sWin%, higher numbers mean the QB actually won more than expected based on the support he received and usually implies a higher xWin% (i.e. the better the QB).

*UPDATE: The table below is now fully sortable and searchable. You can sort by any column or type part of a player’s name into the search box to find a specific quarterback.*