Can NFL teams win more games by instructing the ball carrier to stop short of the first down? Believe it.
The first downThis year marks the 100th anniversary of the National Football League. The game has undergone many changes, but statistical analysis of strategy has been something of a late starter. For example, a good chunk of that history expired before a serious discussion of punting strategy occurred and to this day, some armchair statisticians are driven to distraction by suboptimal fourth-down decision making.
In this post, I go after another sacred cow - the first down. I'm going to argue that players are, perhaps unwittingly, making a mid-play strategy error on the first down that is materially impacting their chance of winning the game. Their sin isn't a lack of aggression this time, but too much. They are finding their way to 10 yards too often. Yes, that's right. Teams are mistakenly reaching for too many first downs.
What should they be doing instead, you ask? Not getting first downs?
Exactly! On a first-down play, no offensive player should want a first down unless he can get some extra yards in the process. Here's Alvin Kamara in the divisional making a terrific run, but also a tactical error.
From my casual observation of American Football, it seems that commentators, fans, and coaches alike uniformly encourage their players to lunge, stretch, hurdle, or bulldoze their way across that last yard to move the chains. Getting the first down is a motherhood issue. It's applauded. For example, how often do we see a wide receiver break the imaginary yellow plane in an acrobatic fashion, often one arm outstretched, as they careen out of bounds?
Using data from the 2009-2013 NFL seasons, we shall investigate this "decision" (or perhaps we should call it a non-decision). It is a decision made every time a player decides to put their body on the line to reach the first down marker.
Let me be clear about this scenario. First, we are only talking about decisions made on the first of four downs. (Readers unfamiliar with the NFL may wonder about the terminology. The "first down" refers to both the first of four attempts to advance 10 yards, but also refers to the act of achieving said 10-yard advance, thereby resetting the down count.)
Second, we are referencing a decision made when a player with the ball is certain he can't progress terribly far beyond the first down marker. We're talking about that last effort to break the plane of the first down marker. Obviously, a player in full flight who is likely to achieve the first down and also many extra yards would be silly to stop one yard short.
Under these conditions, in choosing to complete the first down, or not, the ball carrier is determining which of these two outcomes will ensue:
The second possibility seems to be the strongly preferred option, given two seeming advantages:
However, there is one big disadvantage. The number of yards required for the next down reset will be 10, rather than one.
It's our duty to convince the NFL offensive coordinators to order their players to do what doesn't come naturally. Instead of reaching for the first down, they should slide, run out of bounds, or otherwise stop the advance at the nine-yard line. This is a tough case to make and makes for quite the break in tradition. As it happens, the logic is easy, although we'll need an upper bound on the value of possession relative to yards to be convincing.
Thus I invite you on a slight detour to determine, very roughly, the value of possession. A slightly naive analysis would suggest that since teams drive 35-40 yards on average, a possession can't be worth more than about 40 yards. Let's be more careful, however. Some people might find the above rationale suspicious. I wouldn't like the findings to depend on it, so I'll offer you another way to arrive at roughly the same answer.
We ask a different question. How many points is a yard worth? By this, I refer to a yard of field position, assuming we are on the first down.
Of course, of course, of course... the value of a yard depends on this and that. I'm not going to get into endgame analysis or special situations, nor the value of the clock when one team is ahead. I shall be content with somewhat typical field and game position, which, if you prefer, can be assumed to occur in the first three-quarters of a relatively even game (that is, not so lopsided that one team has significantly diverged from a strategy that, roughly speaking, maximizes mean points scored).
As a an aside: here is an illustration of why points per yard may vary even under benign settings. Suppose we are in field goal range on the third down. The value of a yard is reasonably represented by the blue line's slope in this plot I stole from a 538 article on punters (thanks Benjamin Morris). The plot shows the probability of a successful field goal. Since the slope clearly varies quite a lot, so does the value of a yard on third down when, let's assume, we must take the field goal. The value of a yard on first down will include a kind of average of these results, somewhat smoothing out the differences in slope, but we will still have different values of points per yard.
This kind of variation shouldn't deter us. I will be lazy and borrow from our good friends at Advanced Football Analytics from where our next plot is taken. This particular analysis is the work of Brian Burke, and it shows a value function derived from a Markov Model for football. With some caveats, this really is the real value of field position and it takes into account the possession after this one, and the possession after that, all the way to the end of the game.
As an aside, look closely and you will see the field goal effect (the varying marginal value of a yard) as we get to the red zone. At first, it wiggles up, representing the relatively small benefit of getting closer when your kicker is highly likely to make the kick. Then, the expected points drop faster. You also see effects when touchdowns have a high probability - albeit in the other direction since one-yard means all the world when a team is two yards from the end zone.
Here's an example of reading the plot. Notice that the intercept occurs at 85 yards, approximately. This means that a Herculean punter who can always pin an opposition to their own 15-yard line (when punting from their own 15-yard line for a net 70-yard punt) would be a handy asset. A team transferring possession in this manner has lost absolutely nothing, according to this plot.
In the absence of a physics-defying punter, we can use the plot to draw conclusions that are more reasonable. For example, let's say we read off 66 yards (i.e., our own 34-yard line) corresponding to +1 point on that plot. We are +1 points because we have possession. That's how my eye reads it, anyway. Now imagine we punt it to our opposition. They start their next play on their own 34-yard line after an entirely plausible, if unspectacular, net punt of 32 yards. So now they are +1 points. Net, we have lost two points.
Or perhaps the punt is better. They start at their own 20 instead, corresponding to an expected 0.5 points per possession. Net, we have lost 1.5 points. I put it to you that a possession is worth not more than two points, or perhaps a tad more, but not much more. You get the idea.
Similarly, we can read the value of a yard. You can see that between the red zones it takes about 60 yards to go from four points to zero. This means about 15 yards per point. Put those together and you see that a possession is worth about 30 yards - with the caveats above - and probably not much more. The 30-yard estimate seems small, don't you think? Well then, make it 40.
If you worry this might not apply to college football, refer to this analysis by Saiem Gilani. I think you will find that the relationship between yards and possession is pretty close to what I postulate. Gilani cites the history of this kind of work, going back to Virgil Carter's paper in 1971.
Armed with the seeming fact that a possession is worth no more than 40 yards, and probably considerably less, we turn to the third down. We assume one yard remains and we set about determining how to value this situation from the offensive perspective.
I could not find any study of third down and one situations so alas, I had to do a tiny amount of work myself (see the notebook which you can use to reproduce my database queries and rudimentary arithmetic). I was pretty lazy and used a CSV prepared by Ben Dilday (Github homepage) containing all plays from the 2009-2013 seasons. I'm sure the skeptical reader will be able to enlarge the dataset should they wish to do so.
We need to understand third and one for the obvious reason that an unsuccessful second and one play will probably leave us in that position (the astute reader will note there is a small chance of a loss on the play - but we can use running plays in this argument if we need to, where this effect is very small).
What most interested me was the number of yards gained on a successful third and one play. That's a conditional average, not the average including unsuccessful plays that don't advance the line of scrimmage. When passing, this conditional average gain is 12.75 yards. When rushing, the average gain is five yards.
Another aside: this is a large discrepancy between passing and running and it suggests that teams might consider passing on third and one more often than they do. Passing plays are only successful 61% of the time, versus 72% when rushing (and I am led to believe, 80% for quarterback sneaks - though that was not in the data I had). However, an 11% chance of lost possession corresponds to only a few yards, according to our analysis above. And this gets swamped by the massive 7.75 yard differential when passing. Side conclusion: teams should run passing plays on third and one!
Third and one strategy is certainly interesting, but the only thing we need to take from this analysis is that a successful third and one results in an advance of field position of five yards even if you choose to run the ball. Perhaps that's worth repeating for those who fear the third down. Third and one is probably first and six, give or take.
With the knowledge of what third and one will entail for us, let's roll back one play. How should we feel about finding ourselves in second and one? When we look at second and one plays, rather than third and one, we find that the rushing play percentage goes even higher - up to 80% - though the average number of yards drops slightly (down to 4.73).
Another aside: the differential between passing and rushing yards gained decreases, with passing leading to only 6.5 yards of gain on average compared with 7.75 for passing on third down. Side conclusion: the data suggests that the natural urge to pass on second and run on third might be working against the better interest of teams. Interesting no? But for another time. All we need for our analysis are the following "facts":
Perhaps it is apparent why, sometimes, you don't want those chains moved. But let's close out the argument...
Finally, the moment of truth has arrived. Our star receiver takes a catch a yard short of the first down. Flat-footed, he turns to see a defensive player bearing down at great speed. Risking a season-ending injury he can, most certainly, dive forward with outstretched hands and make the hero play - securing the first down. Alternatively, he can casually step out of bounds, leaving his team at second and one.
Or perhaps a tight end has broken one tackle and staggers toward the first down marker dragging a defender who has grasped his leg. Should he break the plane or voluntarily stop his progress, if he knows he won't get any extra yards beyond the first down?
For a potentially controversial topic, the calculus is alarmingly easy. Starting at second and one, our team will get the first down 94.5% of the time. But what is more important is that our team will advance field position in the process, by an average of almost five yards. This means that the first down will take place four yards further into enemy territory than it otherwise would. Four yards (or even 3.75) is not to be sneezed at. And this assumes rushing, which may not be optimal on third and one, as we have discussed.
We can now frame the decision in terms of implied yards per possession. The wide receiver's lunge suggests that a 5.5% chance of losing possession in this series of downs is more important than 3.75 yards of field position. He is wrong! Since 5.5% is roughly 1 in 18, this means that possession must be 17 times more important than 3.75 yards. The receiver implies a value of possession of 17*3.75=63.75 yards! But as we have seen, there is no way in God's green football field that a possession is worth 64 yards. It is closer to half that number.
A Century of Football Strategy out the window?
It gets worse. Interestingly, going for the first down isn't even worth it if you get to the 11-yard line. The calculus would then read 2.75*17=46.75, which is still way too high a value (in yards) to put on possession. We have to get to the 11.5-yard line to get something closer to a sensible value of yards per possession (38.25). So the moral of the story is, unless you think you can advance two yards past the first down, take a knee.
Now that we know what is optimal, let's take a look at what really happens on the field. Here is a plot of first down yards gained rushing, where for simplicity we are restricting attention to cases where it is first and 10. One would think that with a rushing play, the offensive ball carrier would have good ability to aim for nine yards or 12, but not accidentally end up in between.
The data, which shows some mass moved from 10-yard gains back to nine yard gains, seems to indicate one of two things:
I'm not sure how we can disentangle these two effects. However, one thing is clear, and that is that there is plenty of room for improvement. Defensive teams could very easily allow players to make a 10-yard advance, and they are clearly not doing that. Conversely, offensive teams really should have very little probability on the 10- yard advances.
From the perspective of the offensive team, the fact that 12-yard advances are less common than 11-yard advances is also a clear sign of poor strategy. That mass on the 11-yard gain should be moved back to 9 yards. Injury risking heroics used to get to 10 yards should, in fact, be reserved for going from 11 to 12, or 12 to 13, when the opportunity arises.
Interestingly, the data is so similar for passing plays that I'll save one plot. Whether we are talking passing or rushing, both offensive and defensive teams clearly need to improve their strategy. Defensive teams can no doubt benefit from coverage patterns that deliberately allow the first down if a player is likely to make nine yards (while simultaneously reducing the chance of being caught flat footed by a long pass, most likely).
A more scathing indictment of strategy, both offensive and defensive, is delivered by the following plot, which shows first down yards gained on the rarer occasions when, due to a defensive penalty, we are at first and five. There ought to be more ability to control yardage gained, and design a play to achieve four yards with high probability. Instead, teams are mistakenly attempting to get to five yards.
What are they thinking?! My data suggest that teams' chances of making four yards when they want it is significantly higher than their chance of making five yards (when they think they want it) - about 10% higher. For this, we have to use second down data due to a dearth of first and four situations - though there are in fact two in the database, and on both occasions four yards were achieved. It is clear that teams are running riskier plays to achieve a worse outcome.
One day this craziness will stop. We'll know it because the distributions you see here will converge toward the optimal choices instead. We'll see more four-yard gains on first and five than we see five-yard gains, for example. I'm not holding my breath, however.
Thanks for the questions and comments on this in other forums.
Question: Doesn't one have to analyze future drives to determine whether the decision to stop at nine yards is indeed correct?
Response: There is certainly enough data to do this, and it would improve the analysis by providing the reader with an alternative path. However, I am certain the result will be the same. The key point in this, or any other analysis of the sort, is that the value function (plot of point advantage versus field position) already takes future plays into account - thus providing a shortcut (again, here's my lay introduction to value functions). And while it may not be immediately intuitive that a possession can be measured in yards, that too is a shortcut to the final result.
Question: Is it not against the spirit of the game to stop shy of the first down?
Response: It is beyond my remit to answer that one. However, I fail to see how a game is playable if intent is to be inferred or even policed for something so basic as the first down play (we aren't talking about bowling underarm here). After all, if a defensive team chooses a pattern that puts slightly less emphasis on stopping the first down, would that also be considered unethical? If you believe that choosing the best strategy is unethical then the great game of football may just be fundamentally flawed. It could be argued that it is already flawed, given the seemingly distasteful fact I have shown here - that a play's benefit to a team is non-monotonic in yards gained. The fixes don't seem very elegant (e.g. a team making 10 yards on the first down could be offered an option of second and one - but I don't see that catching on any time soon).
Consider following microprediction on LinkedIn to justify my existence.