Pass Completion Rate: Does it Really Matter?

By Jared Young (@jaredeyoung)

During week two of the MLS season Orlando City SC completed 87% of their passes on their way to a 1-1 draw with Chicago. At the other end of the spectrum, D.C. United completed just 63% of their passes. They earned the same result with a 0-0 tie at New England. Hmmmm. What's that all about? United failed to complete more than a third of their passes while Orlando only failed once in eight attempts. Is there anything to take away from that very wide range of pass completion rates?

To better understand pass completion rates let's take a quick look at the 2015 season, where there is a bigger sample size. Orlando SC wasn't too far off of last year's pass completion single game high, which was 87.9% by the New York Red Bulls. The low of the season was 56.6% by Sporting Kansas City. Looking at the New York Red Bulls performance reveals that they played a lot of short passes on the ground. They attempted 653 passes in that game and only 41 of them were longer than 25 yards. Only 3% of those passes were attempted off of a players head. Contrast that with Sporting Kansas City who attempted 101 passes longer than 25 yards, over 36% of their attempts. They also used their respective heads to make the pass 12% of the time. Longer passes and passes off the head are obviously more difficult to complete. Could it simply be the type of passes that drive the difference?

Taking these observations a step further I looked at the collection of pass types that a team makes throughout a game and attempted to predict what the pass completion rate would be. The model turns out to be reasonably good just using pass types with 78% of the variance in per game pass completion rates being described just by looking at the types and indirectly the difficulty of the passes.

For the curious I'll provide details about the model results but here's a look at week two pass completion results:

As you can see the model using pass types does a pretty good job of predicting what the pass completion percentage will be. Even LA Galaxy's seemingly off night with a 65% pass completion rate was mostly expected. Pass completion rate and the difference from the expected rate was not predictive of results this past week, but over the course of last season playoff teams outperformed their expected pass completion rate by 0.14% while non-playoff teams under performed their expected  pass completion rate by 0.13%. Surprising? Significant? Barely.

The takeaway is that pass completion rate is not by itself an indication of a good or bad game by a team. The number needs to be put into context. What pass completion rate really indicates is how difficult the passes a team was attempting. A high completion rate indicates short safe passes on the ground, while a low completion rate indicates longer passes more likely in the air. Pass completion rate is more indicative of risk taking than it is ability on any given night. 

Model Validation

The model was built on 2015 game data. I created a training data set with 70% of the game results and tested it on the remaining 30%. I used just pass different pass types like throw ins, crosses, headers, etc as well as a home and away flag, which was not significant.  

The training and validation results rank order fairly well. The validation results flatline from forecast deciles 5 to 8 because of a spike in decile 5, but otherwise look good.  

The actuals track the forecast very well on the validation set. To summarize, pass completion rate is a very predictable metric and because of that a fan should not be fooled to think that a 67% pass completion rate by their team was solely because of an off night. That certainly may be true but most of that seemingly off night can be described by the riskiness of the pass attempts. Perhaps the defense influenced that level of risk taking or perhaps it was a tactical decision, but that fact, and not a poor physical performance, is more than likely the cause of the low percentage.

Special thanks to Kevin Minkus (@kevinminkus) for help with this post