The Anatomy of a Decision

4th and 2. Up 6. 2:08 remaining. Ball on your own 28. As a head coach, what do you do? More importantly, what process do you go through in order to make a decision.

Many of you will recognize the above situation: it is the famous 4th and 2 play from the 2009 game between the Patriots and Colts. I am not interested in discussing the validity of this particular decision as it has been dissected more than any other play of the past few years. Instead, I am simply going to use it as a lens to discuss how decisions should be made.

As a Colts fan, I debated the decision countless times (defending it). Many times, people would deride my use of “numbers” when I would lay out my argument. What they were really criticizing, however, were two very different things that many people often lump together and dismiss as “numbers”.

In decision-making, there are really two dimensions: (1) the first is defining the question you want answered and identifying the parameters that you’ll need in order to answer it and (2) the other is using statistics to estimate the unknown variables in that resulting equation. As I’ll show, criticism of the latter can be valid and depends heavily on the situation, but the former is not debatable but an indisputable truth.

The Decision-Making Process

Part I: IDENTIFYING THE PARAMETERS

STEP 1: Ask the ideal question you want answered.

The key here is to forget about all the constraints and simply ask the question you want answered if you had perfect information. Many times in football, the question will be simple: “Which decision gives my team the best chance of winning the game?”

In the example I’m using, we want to know which decision gives the Patriots the best chance of winning the game. Our question will be: “Which decision gives the Patriots the higher probability of winning: punt or go for it?”

STEP 2: Express the parameters of your question.

In other words, simplify your question to just the parameters you need in order to answer the question. What information do you need in order to make the best decision?

Expressed as an equation using words, we’d say: is the probability of winning if we choose to punt less than or greater than the probability of winning if we choose to go for it. I’ll express this in more mathematical terms as: PW(punt) ?= PW(goforit)*. If you knew exactly those probabilities, in a perfect world, you’d simply choose the one that is higher. This equation is just a more precise representation of what we already expressed in words.

*The funky “?=” you see just means we don’t know if the two sides of the equation are equal or if one is greater than the other.

Note that right now, this is an indisputable fact. If we knew the exact values for PW(punt) and PW(goforit), we would have the exact correct decision for our question. This is an important point, as many people hear “probability” or see the equation and dismiss this. Instead, they’d talk about trusting your defense, or how Peyton Manning is on the other sideline, or mention that the Pats offense is hot. There is nothing wrong with that (as we will see later), but all of those points are just inputs into our equation! Every point made MUST affect at least one side of the equation. Your offense is hot? That increases the chances you win if you go for it. You have a good punter? That increases your chances of winning if you punt. You trust your defense? Well, that affects both sides, though probably not equally. All of these are factors in our equation, and will affect the final decision, but they do not change the validity of the equation itself.

STEP 3: Factor out the parameters of your equation.

At this point, we can break down the equation into smaller parts in order to make it easier to estimate the parameters of the equation and arrive at a final decision. You many not know the probability of winning with much accuracy, but you may have a very good idea of your team’s ability to convert on 3rd down, or your FG kicker’s range, or how good your defense is at shutting down long passes. By factoring out each variable, we end up with a much more manageable equation.

Continuing with my example, we might break down our equation like so:

PW(punt) ?= PW(goforit)

PW(punt) ?= P(convert) * PW(success) + (1-P(convert)) * PW(failure)

Estimating the probability of winning if I go for it may be tough, but I can make better estimates of the probability that NE converts 4th and 2, the probability they win if they convert, and the probability they win if the Colts stop them.

STEP 4: Making simplifying assumptions about our equation.

Up until this point, we have been talking about our ideal question. Since we live in an imperfect world, many times we’ll make assumptions to simplify our question. We may later go back and challenge these assumptions, but for everything we do AFTER this point, we will assume that our assumptions are true and proceed accordingly.

Let’s make a few assumptions to simplify our example equation. For example, I may make the following assumptions: (1) if the Patriots convert 4th down, they win; (2) if the Colts get the ball and score, they win; (3) the Patriots punt is not blocked and the result will be that of an average punt in that situation; (4) if the Patriots don’t convert, they gain no yards and the Colts take over at the same line of scrimmage.

Granted, we are leaving out some factors, but in doing so we are simplifying and focusing on the major factors that go into the decision. We may end up with this as our simplified equation.

(1-P(Colts score TD from own 27)) ?= P(convert) + (1-P(convert)) * (1-P(Colts scored TD from NE 28))

The PW(punt) is now just 1 minus the probability of the Colts scoring a TD from their own 27 (the result of an average punt), or said another way, the probability that the Patriots stop the Colts after the punt. On our “go-for-it” side of the equation, we add together the probability that Patriots convert 4th down (and thus, clinch the win) plus the probability that they don’t convert but still stop the Colts from scoring on the ensuing drive.

So we just need to know the chances that the Colts score from their own 27 and the NE 28, plus the odds of the Patriots converting the 4th and 2. Still, we haven’t done ANYTHING that involves numbers.

PART II: ESTIMATING THE VARIABLES

STEP 5: Estimating the unknown variables in our equation.

So we now have our equation, but we don’t know the value of any of the variables. What should we do? We can use numbers to estimate them. We can use logic. We can use instinct, or gut, or intuition. We may each have our preferences, and that is fine. But whatever we use to derive values for our variables, they must fit into our structure.

You want to incorporate Tom Brady’s clutchness? Increase the probability of NE converting 4th down. Think the Pats defense will be deflated if Belichick chooses to go for it and fails? That increases the chances the Colts score from the NE 30. Think the Colts defense has an advantage knowing that giving up a 2-yard gain for a 1st down is the same as giving up a big play for a TD? That’ll affect NE’s conversion rate downward. You get the point.

Here’s an example of how two different people may estimate NE’s probability of converting.

PERSON A: First, look at the historical average of all teams converting 4th and 2 between the 20s, say it’s 52%. Now I want to adjust that upward because NE has a good offense and the Colts have a bad defense: 55%. The Colts are at home (?) and get to play aggressively on 4th and 2, making it more like a 2P converstion attempt (succes rate of around 40%): decrase the odds to 48%.

PERSON B: The Patriots have Tom Brady, who I think is clutch, and have a lot of weapons on 4th and short: 60%. The Pats have been unstoppable today: 65%. Belichick will make a great play call and the offense will be emotionally elevated knowing that their coach believes in them: 72%.

Both of these are reasonable ways to approach estimating this number. The point is that we have a framework and now anyone can estimate the variables as they see fit, plug them in, and get an answer. This resulting conclusion cannot be denied. You can’t add anything else in or take anything out. You can change your variable estimates, or revisit your simplifying assumptions, but if you accept your assumptions and your estimates, you MUST accept the conclusion.

CONCLUSION

To summarize, the anatomy of a decision can be broken down into two parts:

PART I: Identifying the parameters of the decision. Ask the right question, determine the appropriate parameters, make smart assumptions. These are the indisputable constraints of the problem, not just one way to approach it but THE ONLY WAY. If you want a different answer, you have to ask a different question.

PART II: Estimating the parameters of the decision. Once the framework is in place, use the best information you have to plug into your equation. This information may be data and statistics, personal observation, intuition, or any other source of information out there. Some will be better than others, based on the situation, but there is room for healthy debate here.

Decision making is one of the most important parts of football. Every play presents a variety of decisions that need to be made, from personnel to play calling to 4th down decisions. Sometimes, numbers can help us estimate the parameters of our decision, sometimes they can’t. But we can always ask the correct question, determine the necessary parameters, and–with some smart assumptions and educated estimates–arrive at the best possible decision.