Based on their respective records this might sound crazy. Brady has three rings, five total Super Bowl appearances, and a record 17 playoff victories. Manning on the other hand: a below .500 playoff record, just one Super Bowl ring, and a record eight “one-and-outs”. How could anybody in their right mind choose the latter over the former? It’s amazing what a little perspective can do. Let’s start from the beginning.
Manning v Brady: Regular Season
One way to compare players’ careers is to plot their seasons side-by-side, but rearrange them from best to worst. Instead of putting their first seasons together or their 2011 seasons together, you can put their best seasons together. For instance, Manning’s best season (by xWin%*) was his 2004 campaign while Brady’s best was the magnificent 2007 season. As you’ll see below those two seasons are quite comparable. The rest of the seasons are done in the same manner, from 2nd through 12th best seasons.
*xWin%, as explained in Part II, is the expected winning percentage for a QB based on his EPA (again using Brian Burke’s Expected Points model) in that game.
As the graph shows, Manning and Brady’s best 3 seasons are all nearly identical. However, starting with their 4th best years Manning starts to separate himself. By their 7th best, Peyton is around 70% xWin% while Brady is at 60%. One thing to note is that Brady has one fewer season, so in this sample he only had 11 seasons while Manning had 12. We are also missing Manning’s 98 and 99 seasons but in this analysis adding those in could only help him.
Overall, Peyton’s xWin% was 69% to Brady’s 64%. Manning’s actual regular season win % was worse than Brady’s–73% to 78%–but that is because he was given worse support. An average QB given Peyton’s defense/special teams/running game would win slightly less than half their games: 48%. Brady, meanwhile, benefited from very good support with an sWin%* of 56%. Based on that, Manning has outperformed his support by nearly 25%, compared to Brady’s also-outstanding 22%, the top two numbers of the past decade-plus.
*sWin% is how often a quarterback would win based on everything besides his own EPA (that is, his defense, special teams, and offensive plays he was not involved in).
This comparison, however, is only getting tighter. Manning’s three best seasons, in order, were 2004, 2005, and 2009. Brady, meanwhile, has had his best seasons in 2007, 2011, and 2012. From 2000 through 2006, Manning dominated Brady with an xWin% of 68% to Brady’s 56%. At this point, the perception was nowhere near reality. Brady had 3 rings and a sterling postseason record, with many preferring him to Manning who had no rings (he’d get his first following the 06 regular season) despite his multiple MVP awards. From that point on, however, it’s been Brady who has been the best QB in the league with a 73% xWin% to Manning’s 2nd-place 69%. With Manning a year and a half older and Brady playing the best football of his career, the gap could shrink in the coming years.
Into the Postseason
Last time we cut off our top postseason performers after the top 4, and that’s because of this:
#5 Peyton Manning: 59.4%
#6 Tom Brady: 58.5%
*Each player is missing a game–Manning’s first playoff game in 1999 came before my set and Brady was not the QB of record in the 2001 AFC Championship game when Bledsoe relieved him after he was injured and Bledsoe had more pass attempts.
In their playoff careers, Peyton Manning has actually played–ever so slightly–better than Tom Brady. Manning should have won just over 11 of his 19 games on average, for an expected record of 11.3 wins and 7.7 losses. Brady’s expected record is 13.5 and 9.5. So what explains the difference between their expected and actual records? Their support. An average QB would win just 40% of the games that Manning played while the support alone would be good enough for an average QB to win over half the time (52% sWin%) in Brady’s games.
As we did with their regular seasons, this time let’s plot Manning and Brady’s postseason careers on a game-by-game basis with their best games first. In these graphs, the line shows each quarterback’s games from best to worst. Green dots represent wins and red ones losses. The gray bars represent the sWin% for that game, with above average support above the line and below average support below.
In Manning’s 19 career playoff games, he has received positive support just 5 times. Amazingly, the best three support games all came during the Colts 2006 Super Bowl run. In those 3 games (#14-16 above), Manning turned in three of his worst postseason games, all with xWin%’s under 50% including the Super Bowl itself. The only game where Manning played well was the famous AFC Championship game where he led a huge comeback win over the Patriots, his 74% xWin% was good enough to overcome the 45% sWin%. In the other three games, Manning was bailed out by incredible support: all those games saw him receive support with an sWin% over 90%. The only other game where he received significant help from the rest of the team was the 2003 AFC Championship game where Manning turned in by far his worst postseason performance (this was the famous 4-INT snow game).
We can really divide Manning’s career into three parts. In Manning’s best 6 games–all wins–he totaled 5.3 expected wins, giving him +0.7 wins thanks to support and luck (mostly luck in this case as he only received positive support in one game). The second section is Manning’s next 6 best games where he was above average, all between 55% and 71% xWin%. In those 6 games, we would expect him to win nearly 4 of them (3.8 expected wins). He won zero. The support was so bad that an average QB would win only 1.3 of those 6 games. What poor support didn’t take care of, bad luck did. For instance, of those 6 games, the best sWin% and worst xWin% came in the same game (#12 above). That was the 2005 loss to the Steelers when Mike Vanderjagt missed the potential game-tying FG.
The third section is Manning’s 7 below average playoff games. Here, we’d expect him to win only 2.1 games while he actually won 3 (+0.9 wins). The support in these games, thanks mostly to the 2006 playoff run, would have helped an average QB win 4.3 games, meaning Manning cost the team 1.3 wins with his poor performance.
Add those up: in section 1, Manning benefited from support and/or luck to the tune of +0.7 wins. In the 3rd section, Manning lucked into another +0.9 wins. But in the middle section, Manning was unlucky by -3.8 wins. That’s a total of -2.2 wins.
While Manning received positive support in just 5 of his 19 postseason games, Brady has received better than average support in over half (12) of his 23 games. Like Manning, Brady won all of his 6 best games. His expected wins of 5.1 means he was helped by +0.9 wins. In the Brady’s 6 worst games, he won 2 of them, exactly in line with his expected win total in those games.
The big difference comes in the middle. Brady played 11 games where he managed an xWin% between 49% and 70%. Based on his play, his expected win total in those 11 games is 6.4. He won 8. Some of that was support, with a slightly above average sWin% of 52%, and some of that was good luck (Tuck Rule game, anyone?).
That’s the difference. In games where Manning and Brady play great, they win. When they play poorly, they lose as often as expected. But when they play well enough to win, Brady’s defense, special teams, and running game have supported him, while Manning’s support has failed him. These are things out of a quarterback’s control that have a huge impact on the outcome of games, and therefore their legacies.
Let’s split their careers in 2004. In both halves, Manning and Brady played nearly equal (Brady was better early 60% to 56% xWin%, Manning was better later 61% to 57%). In the three Super Bowl runs, the Patriots defense put up a 60% sWin%; Manning’s sWin%: 29%. Brady’s teams went 8-0, Manning’s went 3-4. Since then, Brady is finding out just how hard it is to win without great support. Both quarterbacks have received similar below-average support (46% for Manning; 48% for Brady) and both have won about half their games (6-6 for Manning; 8-7 for Brady).
It’s Not Over
This analysis is not the end-all, be-all to end this debate. However, it does move the discussion forward and future analysis can build on this. We used EPA, which is well-suited to this analysis, but so would WPA which is similar to EPA but also accounts for time and score. A cursory look at Advanced NFL Stats playoff pages for Brady and Manning shows that Manning averages 0.22 WPA per playoff game compared to 0.18 for Brady.
EPA also has some flaws. For instance, it only compares the start of the play to the end of the play. It doesn’t separate out Brady’s interception to Marlon McCree and McCree’s ensuing lost fumble forced by Troy Brown. It just sees a 4th and 5 followed by a 1st and 10, a play in which Brady was involved in. EPA doesn’t see that Troy Polamalu’s interception of Peyton Manning should have been upheld as a turnover; it just sees an incomplete pass. These are all very minor quibbles, but worth pointing out.
The biggest caveat is that while we were able to separate out non-QB EPA that the QB has no control over from QB EPA which the QB does have some control over, that QB EPA certainly does not tell the full story. For one thing, the quaterback has some control over the outcome of those plays, but certainly the rest of the offense deserves some credit. Quarterbacks with better receivers or offensive linemen would be helped by this analysis. Brady certainly had some poor passing game support early in his career, which could have hurt his QB EPA. Some further research would have to be done to determine exactly how much of the QB EPA should be credited to the QB. The quality of the defense quarterbacks face also affects their performance. A quick look shows that Manning has faced higher quality defenses in the postseason than Brady. Manning’s opposing defenses averaged -4 EPA per game in the regular season (remember, negative defense EPA is good) while Brady’s opposing defenses averaged -2 EPA. What about weather? Manning played many games in domes while Brady’s home games were outside in the cold, which has been shown to have an effect on QB play. There are certainly other biases and factors that could have some effect on this analysis.
What we’ve done here is control for the biggest factor outside of a quarterback’s control in determining the outcome of games. Based on their QB EPA, Manning and Brady have played remarkably similar over their playoff careers. The difference between their performance and their actual postseason record is nearly all explained by the support they received.
You can argue that Brady has been a better postseason QB than Manning, but your argument can’t have anything to do with their win-loss record while ignoring the fact that Brady has benefited from much better defensive support. The fact is that among the two best quarterbacks of their generation, Manning is not only the better regular season quarterback but also that–ever so slightly–Peyton Manning has been a better playoff quarterback than Tom Brady.