Penalty Kicks and Pressure

According to an article on BBC, there is a great difference in penalty kick conversion rates at various points in a World Cup shootout. When a kick would win the shootout, players in the World Cup have converted 93 percent of their opportunities, but when facing elimination with a miss, players have converted just 44 percent of the time. The article doesn't cite sample sizes for these situations, but we do know there were 204 penalties taken over 23 shootouts in the data set. Every shootout has to include at least one such chance--either a chance to clinch or a chance to choke--so a conservative estimate would be sample sizes of 10. And in fact, all we'd need for statistical significance are sample sizes of 10. Check!

Even with statistical significance covered, there could very well be some selection bias here, as perhaps the best PK takers are saved for clinching moments. The combination of small sample sizes and selection bias might explain a lot of the discrepancy in conversion rates, but that's just not a fun conclusion. So let's assume there is some effect of pressure.

In my mind, a PK to clinch a shootout should have some pressure associated with it, just as a PK to avoid elimination would. But what this data suggests is that it's the pressure to avoid elimination that really gets to players.

So I thought I'd check it out in MLS. The only problem is that we don't have nearly enough shootouts that I can access. So instead I will look at in-game PK conversion rates in scenarios where the shooting team is either down one or tied, controlling for which half and whether or not the kick taker is at home.

The results of a logit binomial regression led me to a few conclusions. First, taking a PK at home doesn't significantly alter its chances of going in, but there is significant interaction between the gamestate and the half. There are four scenarios that seem to matter for PK conversion: tied in the first half, tied in the second half, down one in the first half, and down one in the second half. Here's a chart that summarizes those outcomes in MLS:

Gamestate	Half	Goals	Attempts	Percentage
0	1	38	43	88.4%
-1	2	19	24	79.2%
0	2	18	24	75.0%
-1	1	5	13	38.5%

From our knowledge of World Cup shootouts it was predictable that the highest conversion rate belongs to the situation with the least pressure. Tied in the first half, a miss still leaves the team with a lot of time to win, and there the PK probabilities are highest. What's somewhat baffling to me is the rest of the chart. Like for instance the incredibly low conversion rate when down a goal in the first half. Though a sample size of 13 is small, the difference between 88.4 and 38.5 percent is still very statistically significant (p = 0.0002). Or how about why facing a deficit seems to matter in the first half but not the second.

I find it hard to blame selection bias here for our findings. Teams that go down a goal in the first half are likely to be worse teams with potentially worse penalty kick takers, but then that wouldn't explain why they are able to perform well from the spot in the second half. And teams that are tied in the first half have no reason to be the better teams on average, though it's that group that has converted 88.4 percent of its penalties. I'm left to wonder if I don't understand psychology, or if this is all a type-I error. After all, if we ignore deficits in the first half, then there is no statistical significance between the other three scenarios.