Penalty Yards Awarded for Defensive Pass Interference in the NFL are Unjustified

I watch way too much college football, but I have a limited interest in the NFL. I will watch NFL games featuring Lamar Jackson, Deshaun Watson, the Jaguars with Fournette and Josh Allen, and the Chief’s offense—when those games are actually available in my viewing area. Blackouts are one of myriad reasons a person might develop a distaste for the NFL (e.g., stadium costs, handling of violence against women, ticket costs, nonguaranteed contracts, obstructing the publishing of concussion problems, treatment of retired players, and Roger Goodell). In full disclosure, I loathe how defensive pass interference (DPI) penalties are enforced in the NFL, the topic of this post. Regardless, I am specifically curious if the yardage granted by an enforced DPI call in the NFL is justifiable statistically.  

Longtime POTH readers know I am an aspiring defensive back. Hence, I will typically be biased against many DPI calls, but I recognize that DPI does genuinely occur. My concern about DPI enforcement in the NFL is that it is a spot foul. The offensive team is awarded a first down at the yard line where the DPI was committed. It thus assumes that the receiver would have caught the pass were it not for the DPI. A spot foul 10 or 15 yards down field seems reasonable to me, but 30 or 40 yards seems like far too much field position to simply gift the offense on what may or may not have been a catch if there were no PI. In other words, it is unfair to award the offense, say, 40 yards because of DPI given that one of several events could have led to an incompletion if there were not DPI. My thesis is that NFL DPI penalty yardage becomes increasingly unjustifiable as the spot of the foul gets farther from the line of scrimmage. But, I don’t know that and that’s why I’m exploring the matter. 

any act by a player more than one yard beyond the line of scrimmage that significantly hinders an eligible player’s opportunity to catch the ball

I needed play-by-play data. It had to include depth of target, the distance from the line of scrimmage to the yard line where the pass is caught or comes closest to the targeted receiver. I found nothing in the open-source arena but ArmchairAnalysis does provide a sample of their thorough NFL charting data, which I used. Specifically, it is a sample of about 4013 plays from two weeks in the 2019 NFL season. Of those, about 2430 are passing plays. I removed 82 throwaways, 157 sacks, and 11 spiked balls, because these events preclude a pass to a receiver.  This left 2180 passing plays eligible for analysis. 

First, I compared the completion % of the 2273 passing plays that were not sacks, which is 65%, to the NFL average % for all other weeks in 2019 (through week 12), which is 63.8%.  This way we can assess if this sample of passes is somehow dissimilar from passes in the remainder of the season. It was not, χ² = 1.49, p = 0.22, 95% CI [0.63, 0.67]. 

Figure 1. Top panel shows distribution of completions (magenta) and incompletions (brown) by depth of target. Bottom panel shows completion % by depth of target (dotted black) and estimated probability of completion by depth of target (green).

Figure 1 shows the raw completion percentage by depth of target (dotted black) and the estimated probability of a completion when accounting for random variance due to defense and targeted receiver (green).¹ The farther a targeted receiver is from the line of scrimmage, the less likely the pass is to be completed. Indeed, this provides some support for my thesis that awarding a first down at the spot of the DPI is increasingly unjustifiable when the depth of target is farther and farther from the line of scrimmage. This is because passes targeted farther down the field are simply less likely to be completed.

Figure 2. Expected yards per pass attempt by depth of target (dotted black) and the expected yards when controlling for random variance due to defense and offense units (orange).

Another way to frame the issue is in terms of yards per attempt (Y/A). That is, how many receiving yards are expected on a pass to a given depth of target. Y/A is a widely used measure of passing efficiency. Figure 2 shows the Y/A by depth of target (dotted black) and the expected Y/A when accounting for random variance due to defense and offense (orange).  This provides additional support for my thesis that awarding a spot foul for DPI is increasingly unwarranted when the DPI is farther from the line of scrimmage. For example, a target of 32 yards down field is expected to gain only 15.9 yards. This might seem odd because 15.9 is less than 32, but Y/A accounts for the probability of the pass being completed. Again, passes target farther down field are less likely to be caught thus a spot foul is less justifiable.

Summarily, this exploratory post yields evidence suggesting that the penalty yards awarded for DPI in the NFL are unwarranted. Although some random variance was accounted for in the models, the major shortcoming is that other factors that affect completion percentage and receiving yards were not accounted for in the analysis. This includes factors such as QB pressure, pass coverage, field position, score differential, and others. Nevertheless, the results demonstrate that, by being a spot foul, the penalty yards awarded to the offense following a DPI (with an uncaught pass) in the NFL are incommensurate with the yardage that would be expected given the depth of target. We here at POTH have no delusions that the enforcement of penalty yardage for DPI will be subject to change. Likewise, we are not anarchists; we respect the game and know that parameters are needed to standardize competition. However, we do feel it necessary to present evidence that directly contradicts any notion of rules designed to ensure a fair game that is decided on the field, by the players.

1 Computed using GLMM specifying a binomial distribution. Depth of Target is fixed effect, with defense and targeted receiver as random effects. QB and offense were considered as random effects but were essentially null and excluded from the model. The model explained about 12% of the variance in completion %. Depth of Target was significant, reducing the log odds of completion by -0.059 for each one yard from the line of scrimmage.

2 Computed using LMM. Simple linear regression had R2 = 0.108 and a smooth regression line had R2 = 0.11, so I used a linear model for simplicity. Depth of Target is fixed effect, with defense and offense as random effects, each with a random intercept for depth of target. The model explained about 18.4% of the variance in A/Y. Depth of target was significant, increasing the A/Y by about 1.03 yards for every three yards of depth of target.

How do College Quarterbacks Perform After Throwing an Interception?

So, you’re watching the game on any day that is or should be Saturday and your QB throws an interception. Waves of fury and grief crash through your viscera. Moments earlier, the stodgy commentator was waxing anachronistically about the merits of establishing the running game but has since transitioned to divining the psychological state of your beloved QB. You will be haunted by the potential of him throwing an interception when your offense retakes the field. You are a fan, after all, and the potential for your QB to throw interceptions looms over your every daily activity and dreamscape from August to January, anyways. But what about your QB? The objective of this report is to examine how (collegiate) QBs perform after they throw an interception.

Did I conduct a literature review? Yes. We know that, in the NFL at least, team interception rates are weakly (r = 0.08)  to modestly correlated (0.27)  within seasons. Among individual NFL QBs, there is a modest relationship (0.27)  in year-to-year INT rates with the same team but it approaches negligibility after changing teams (0.10); but the best NFL QBs do consistently throw interceptions at a rate slightly less than the League average in a given year.  Other work shows there is a moderate relationship (0.43) between a QB’s INT rate and his true quarterbacking ability.  We also know that a QB’s year-to-year completion percentage is pretty stable with the same team (0.58)  but less so when changing teams (0.25).  Together, the prior research suggests that, [a] although INTs are wildly random, some QBs are inevitably more prone to throwing INTs and [b] that some QBs are more accurate than others. Given that interceptions are random, the majority of collegiate QBs are (relatively) well-practiced, and that completion percentage is moderately stable, we might expect that a QBs performance is minimally affected after throwing an INT. 

For the present study, we shall use a massive data set with plays from NCAA games in years 2001 and 2003 through mid-November 2018. I removed all offensive plays of FCS teams but retained plays with FBS offenses against FCS defenses. Because I do not own a supercomputer or even a particularly powerful machine, I removed all plays without a pass attempt or sack in the description. This means we cannot account for QB scrambling ability/threat in the analysis. 

Table 1. Descriptive Data for Cumulative Interceptions

Cumulative INTs	Completion %	Interception %	Sack %	Completions	Interceptions	Sacks	Pass Attempts	Drop Backs
0	0.609	0.029	0.052	286334	13575	25643	470461	496104
1	0.589	0.032	0.057	101146	5511	10372	171704	182076
2	0.577	0.037	0.060	28199	1785	3130	48866	51996
3	0.563	0.040	0.061	6828	490	785	12121	12906
4	0.560	0.034	0.064	1435	87	174	2564	2738
5	0.559	0.043	0.051	209	16	20	374	394
6	0.455	0.036	0.068	25	2	4	55	59
7	0.333	0.000	0.000	2	0	0	6	6

Nevertheless, let our dependent variable—how we’ll measure performance—be whether a given play was a completion. Table 1 shows that completion % decreases after throwing 1, 2, and 3 interceptions but it remains constant at about 56% after the second INT (ignoring the 6th and 7th INTs which are rare scenarios). Now, these differences would be significant statistically because of our sample size; however, is the difference in completion rate between 1 and 2 INTs meaningful? I would wager that a difference of 5 percentage-points, such as between no interceptions (~61%) and 3 interceptions (~56%), might be meaningful. Likewise, interceptions become slightly more likely once a QB has thrown an interception. Although a multitude of factors contribute to QB performance after an interception, some of those factors probably led to cumulative INTs, anyhow. Many of the factors are simply unknowable including individual tendencies of each QB such as perseverance or resilience, self-confidence, and intelligence. There are also in-game factors such as down, distance, field position, time remaining, score differential, team and opponent quality, and others. Let us account for these on-field factors (as best we can), which are listed and defined at the close of the post.

We’ll use a logistic regression to predict the probability of a completion before and after throwing an interception, while controlling for all of the factors we can. Indeed, we should actually use a (generalized) mixed model because there are many plays from each season, game, team unit, and QB—as well as plays for each team unit and QB nested within seasons and games, and QBs nested within teams—but my computer would explode and burn down this apartment building, destroying all of my stuff. 

Our model indicated no main effect of cumulative interceptions on completion %. However, there is a significant effect of cumulative interceptions when approaching the defense’s endzone (i.e., when the offense is getting closer to scoring). Interestingly, QBs were slightly but significantly more likely to complete passes when close to the defense endzone after having thrown an interception. The estimated (marginal) probabilities of completion can be seen in Figure 1. The difference in expected completion % is, at most, 1-2 percentage points from one INT to the next. To me, this is virtually meaningless. Additionally, both poorer offensive lines and better defenses reduced the likelihood of completing a pass. Having thrown more interceptions in the season to date also reduced the likelihood of a completion, albeit meagerly. Completions were less likely on 3rd and 4th downs and as games progressed. QBs at home were slightly more likely to complete passes. The model has a mess of interactions between down, distance, etc., but a summary can be seen here.

Figure 1. Expected Marginal Completion percentage as a function of field position, Time Remaining in the Game, and Cumulative INTs, with other covariates held constant at mean.

Summarily, this report provides evidence that quarterback performance is essentially unchanged after throwing an interception. Critical in-game factors, home-status of QB, and team quality were controlled for in the analysis. The sample size was >700K and the slight effects that were observed are probably ecologically meaningless. That said, I would like to note that if performance is reduced after throwing an INT, say, in a subset of QBs or for some QBs in some games, it is at least somewhat related to psychological processes. For example, some neuroscience research suggests that performance would be impaired immediately following an error, such as on the QB’s next few passes after an INT.  This research also suggests that for well-learned tasks, such as quarterbacking in our case, errors in performance may be more related to activity in the prefrontal cortex (located in your forehead, above your eyes, until about your temple) than in the circuitry of the brain largely responsible for volitional motor movements (but, in the study, electrodes were not placed on the circuits more responsible for learned motor movements). The prefrontal cortex is related to, among myriad other activity, planning and decision making. Nonetheless, you are probably best-served by disregarding that atavistic commentator. Your precious quarterback will probably be fine, but it will never feel that way for you and that is part of what makes college football so delightful.

Acknowledgements
On behalf of all of us here at POTH, I would like to thank my colleague CK for her insight and suggestions.
Covariates
Below are variables included in the analysis. OS% and ONR% are thought to reflect offensive line quality. Passes dropped by receivers and yards after the catch are not described in the play-by-play and thus there was no way to measure the ability receiving units. DS%, DC%, and DPD% are thought to reflect defensive unit quality. Specifically, %DS might reflect D-Line pass rush quality; DPD% might reflect ability of LBs and secondary to defend passes; and DC% might reflect the overall ability of a defensive unit to prevent successful passing. 

• Offensive Sack % (OS%): the proportion of the offensive team’s QB drop-backs that resulted in sacks, for the season on which a play occurred. Adjusted for sack % and FBS/FCS-status of opposing defenses.
• Offensive Negative Rush % (ONR%): the proportion of the offensive team’s rushing plays that resulted in negative yards, for the season. Adjusted for negative rush % and FBS/FCS-status of opposing defenses.
• Defensive Sack % (DS%): the proportion of the defensive team’s opponents’ QB drop-backs that results in sacks, for the season on which a play occurred. Adjusted for sack % of all offenses a defense faced.
• Defensive Completion % (DC%): the season average completion percentage against the defense on a given play. Adjusted for completion % of opposing defense.
• Defensive Passes Defensed % (DPD%): the proportion of passes broken up or intercepted by the defense, in the season on which the play occurred. Bayesian-average adjustment for quantity of passes faced by the defense in that season based on FBS averages.
• Cumulative Interceptions, Game: the quantity of interceptions thrown by the offensive team in the current game up to but not including a given play. That is, if a team had thrown 2 interceptions before a given play, the cumulative value would be 2 for this play. This way we can compute the probability of throwing an interception when none have been thrown. I had to settle for cumulation at the team level instead of the QB level because, again, I do not have a supercomputer.
• Cumulative Interceptions, Season: the quantity of interceptions thrown by the offensive team in the season up to but not including a given play. Used team to be consistent with above.
• Whether QB is playing at home or away.
• A variety of on-field factors noted in the main body of the text.