Roster Consistency in MLS: How much does it actually matter?
/By Dustin Nation (@D_Naish)
EDITOR'S NOTE: Due to a flaw in the method used in the initial ordering of games, we're updating this post with more accurate graphs and a few minor edits to the content. All the main conclusions still hold, so stay tuned for part two in the coming weeks!
Recently, while trying to put a finger on what exactly happened with FC Dallas last season, “lack of squad consistency” was mentioned as a reason for their remarkable plummet down the table. This made me wonder both how much their roster actually changed from week-to-week and how much week-to-week rotation is the right amount.
The following is my initial exploration into roster consistency.
What is Rotation?
For the purposes of this exploration, I’m going to start by defining squad rotation as simply the number of players who are in the starting lineup of a team but weren't in the previous starting lineup. For now, I’m intentionally ignoring player positions and looking only at the raw numbers for each team. In the future, it would be interesting to look at the effects of players switching positions from week-to-week, but I digress.
What is Team Success?
To answer the question of “how much rotation is the right amount of rotation?” we need to define what the measurement of a team’s performance is in order to find that “right amount of rotation.” There are lots of ways to measure a team’s performance, but ultimately the only true measure of success is winning.
Some wins indicate better performance than other wins, sure, but over the course of the season these tend to balance out. So let's measure a team's performance by their final position in the Supporters’ Shield race (number of points accumulated by teams across both conferences).
The Data
All of that having been said, let’s take a look at the squad rotation over the past three years.
2015 saw a very obvious increase in week-to-week changes from the teams at the top of the table to the teams at the bottom. This seems to indicate that the more changes a team made, the worse they were (or the worse they were, the more changes they made, depending on how you want to look at it). We’ll explore this more in a bit.
2016 was different. It had fewer overall changes than the year prior, and the distribution seems to be more random. However when you look at the linear trend line, it still increased (albeit barely) from first place FC Dallas to last place Chicago Fire.
2017 was a return to the higher correlation between table position and the number of week-to-week changes a team made. There was more variation from position to position than in 2015, but the overall trend is for the number of changes to increase as you move towards the bottom of the table.
Breaking it Down
The first pattern to emerge here is that 2015 and 2017 have more overall week-to-week roster changes than 2016. It seems reasonable that this is due to ‘15 and ‘17 being Gold Cup years, causing teams to necessarily make more changes to try and cope with the loss of their international players.
| Season | Total Changes | Diff from 2016 | Gold Cup 23-Man Roster Players |
|---|---|---|---|
| 58 | 45 | ||
| -- | -- | ||
| 156 | 51 |
Simply removing and reinserting the 45 Gold Cup players in ‘15 would be more than the difference between ‘15 and ‘16. However, ‘17 saw almost triple that difference from ‘16. As we’ll see in a bit, more Gold Cup players in ‘17 came from teams in the 2nd half of the table than in ‘15. Since these teams are less likely to be able to cope with the loss of international level players, it’s possible that the extra changes could be chalked up to teams searching for ways to replace the players who’ve left.
The shape of the charts for ‘15 and ‘17 are also curious. You would expect the better teams to have more national team players, and thus more changes. In reality, the teams at the top of the table tended to have less changes than teams at the bottom, which is even more interesting when you factor in the distribution of national team players.
The players leaving their clubs for the Gold Cup are actually distributed quite equally across the league in both years. You can see peaks and valleys in the changes when teams had more or less players out for the Gold Cup, but despite the equally distributed player loss, the trend is still for worse teams to make more changes. But why? Perhaps the better teams are simply deeper and thus able to handle the loss of key players without having to try out a lot of new player combinations.
Another pattern in the data is that both ‘15 and ‘17 are similar in how close teams were to the linear trend line. The trend line is a representation of what you’d expect a team’s changes to be based upon its table position. Both years showed that top teams veered farther from that trend line than teams at the bottom of the table.
While in ‘16 teams at the top of the table were closer to the trend line than teams at the bottom.
Note: “Distance from linear trend” is an absolute value. So while positions 1-4 in ‘15 and ‘17 are far from the trend, it’s because they’re mostly below the trend.
Both ‘15 and ‘17 saw some of teams in the 1-4 positions make far fewer changes, and some of teams in the 5-8 positions make far more changes than the trend would predict. This might suggest that not only do the very top teams have more depth and cohesion than the teams towards the middle of the table, but they actually have MUCH more roster depth and team cohesion than the class of teams beneath them.
All of these assertions place a lot of importance on the linear trend from the “Changes vs Table Position” data. However, let’s take a step back and look at the coefficients of those trend lines.
These low coefficients tell us that the difference in each table position is only two or fewer week-to-week roster changes… over an entire season. Using that as a predictor would be quite the stretch and wouldn’t really be that accurate. This is likely a case where the overall story is more important than how it applies to any individual position on the table or the actual number of week-to-week changes.
What about FC Dallas?
| Season | Table Position | Changes |
|---|---|---|
| 71 | ||
| 80 | ||
| 100 |
FC Dallas made 20 more changes in ‘17 than they did during their Supporter’s Shield winning campaign in ‘16. That’s a swing in roster consistency almost as large as their drop in the table. Given these numbers, it’s quite possible that roster consistency was indeed a factor.
Perhaps FC Dallas wasn’t deep enough to handle the loss of their international players during the Gold Cup and had to search for answers. Or, judging by the xG data on this site showing FC Dallas at the bottom of the of xG/G differentials list, the reason for their drop in form could simply be that, as Jason Poon so eloquently put it on the Dallas Soccer Show, the players that went to the Gold Cup “just forgot how to kick a soccer ball when they got back.”
Final Thoughts
There are a million reasons why teams would need to make a roster change from one week to the next, and these numbers certainly do not paint a complete picture. For instance, rotation amongst different positions might be more common or have less effect on a team’s success. Also, each team and manager are different so a team’s high or low week-to-week rotation might be the norm, causing there to be no correlation between roster changes and table position.
We also don’t know if these patterns and analysis hold true in the years prior to 2015 in order to know if they’ll hold true in the future. However, 2018 is a World Cup year. It will be interesting to see if that affects MLS roster rotation similarly to the Gold Cup in ‘15 and ‘17, especially since the USA won’t be participating. Sadface.
For the next step in my exploration, I’ll get a bit more granular and take a look at what the number of changes in an individual week meant to that week’s performance. Stay tuned!
Data for this article can be found in csv format at https://github.com/d-nation/mls-roster-consistency