Is Manchester United Still a Top Club?

By Soham Choudhury

Introduction

Manchester United Football Club, an organization which is recognized across the globe is also one with the richest of histories in its own country, England, and amongst the highest valued sports franchises in the world, according to Forbes.

Despite the financial success of the club, Manchester United have been going through a title drought ever since the year 2013. At the end of the 2012/13 season, which started in August 2012 and ended in May 2013, the club’s longest serving manager/head coach retired after 28 years of service given towards that team. His most impressive accolades include a record 13 English Premier League (EPL) titles since the league’s inception in 1992. With 20 teams competing in the league every year/season, it is fair to say that Manchester United are spoiled by success having failed to win another EPL since 2013. Clearly, the manager’s absence is a void that the club have not filled and are suffering from it.

Thus, the question arises: Are Manchester United still a top club? Of course, I have already laid out evidence showing that they are doing well financially, at the time of writing, and they have kept their reputation throughout England and other soccer-loving countries. But, I will be focusing on the on-field performances of the team ever since the year 2013 and attempting to determine from those findings if they are indeed still a top club.

We must first start by gathering all the required data from online sources. I have selected fbref.com as my source since they host the necessary league tables and added information for me to perform my analysis later on. However, since league tables are recorded by many different sites and databases, they should have the same common information across all sources.

Data Scraping

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn
from bs4 import BeautifulSoup
import sklearn
import requests as rq
from sklearn.linear_model import LinearRegression
import warnings

warnings.filterwarnings('ignore')

# the following links contain the final league tables for the English Premier League for 8 consecutive seasons
source_urls = ["https://fbref.com/en/comps/9/669/2013-2014-Premier-League-Stats", 
               "https://fbref.com/en/comps/9/733/2014-2015-Premier-League-Stats",
               "https://fbref.com/en/comps/9/1467/2015-2016-Premier-League-Stats",
               "https://fbref.com/en/comps/9/1526/2016-2017-Premier-League-Stats",
               "https://fbref.com/en/comps/9/1631/2017-2018-Premier-League-Stats",
               "https://fbref.com/en/comps/9/1889/2018-2019-Premier-League-Stats",
               "https://fbref.com/en/comps/9/3232/2019-2020-Premier-League-Stats",
               "https://fbref.com/en/comps/9/Premier-League-Stats"]

league_tables = []


for i in source_urls:
    # here, we get the page
    page = rq.get(i)
    soup = BeautifulSoup(page.text)
    tables = soup.findAll('table')
    
    # the first table in each webpage is what we need
    df = pd.read_html(str(tables))[0]
    
    # adding the current dataframe to the list of league tables
    league_tables.append(df)

Tidying the Data

We have brought in the league tables for the 13/14, 14/15, … 20/21 seasons. But the league tables for the last 4 seasons (17/18 to 20/21) have 2 added pieces of information that we are interested in.

Missing Data

This means that the earlier 4 seasons of the 8 we are using will not have these 2 pieces of data reported for any of the teams which are Expected Goals (xG) and Expected Goals Allowed (xGA). The missing data will mean that the earlier 4 seasons will not be used for our Data Analysis when calculating attack and defense scores. For now, we are only interested in the following columns of the table, which will each be explained later.

# creating a list of NaNs
empty_array = np.empty((20,1))
empty_array = np.NaN

# tidying the tables for the earlier years
x = range(0, 4)
for i in x:
    curr_table = league_tables[i]
    league_tables[i] = curr_table[["Rk", "Squad", "MP", "W", "D", "L", "GF", "GA", "GD", "Pts"]]
    
    # setting the values for xG and xGA to NaN for the earlier seasons
    league_tables[i]["xG"] = empty_array
    league_tables[i]["xGA"] = empty_array
    
# tidying the tables for the latter years
x = range(4, 8)
for i in x:
    curr_table = league_tables[i]
    league_tables[i] = curr_table[["Rk", "Squad", "MP", "W", "D", "L", "GF", "GA", "GD", "Pts", "xG", "xGA"]]

It should be noted that placing a team like Manchester United next to any other team would not be a fair comparison. Thus, it is important to compare Manchester United and its rivals, otherwise known as the Big 6 clubs, to each other.

These teams compete against each other in terms of reputation, financial stability, and the competitions they play in. These teams are also widely accepted across the country as the teams who have the highest chances to win trophies. If you would like to learn more about why I made the decision to filter down to only these six clubs, you can read more about it here in an article by Goal.

#Making a list of all the big 6 clubs
big6 = ["Manchester City", "Liverpool", "Chelsea", "Arsenal", "Tottenham", "Manchester Utd"]

# getting rid of all the irrelevant clubs
x = range(0, 8)
for i in x:
    drop_rows = []
    for ind in league_tables[i].index:
        if ((league_tables[i]["Squad"][ind] in big6) == False):
            drop_rows.append(ind)
    
    # this step deletes all rows which doesn't concern
    # any of the big 6 clubs
    league_tables[i] = league_tables[i].drop(drop_rows)

I also add a column to these tables to indicate which season these results took place in as it will be important to differentiate the seasons when looking at plots.

# creating list of all seasons
seasons = ["13/14", "14/15", "15/16", "16/17", "17/18", "18/19", "19/20", "20/21"]

# adding the Seasons column into each table
x = range(0, 8)
for i in x:
    season_inp = np.empty((6,1))
    season_inp = seasons[i]
    league_tables[i]["Season"] = season_inp

I have chosen to use a statistic added to the table by myself called Points per Game (PPG). PPG is simply calculated by dividing the points accrued by a team throughout a whole season by the amount of games they played, which is 38 for all teams in every season.

Thus, a team’s rank in the final league table remains the same if it was to be sorted by PPG rather than Points. This is a better way to calculate the performance of a team throughout the season since the measurement for PPG is condensed between values of 0 to 3. If I had left the analysis to be done with Points, the values could have ranged from 0 to 114.

# adding the points per game (PPG) column into each table
x = range(0, 8)
for i in x:
    league_tables[i]["PPG"] = league_tables[i]["Pts"] / 38 

Now that we have a Season column to differentiate between one team’s campaigns in various years, we can finally combine the individual league tables to make one large one.

# combining all tables from the 8 different seasons
table = league_tables[0]
for i in range(1,8):
    table = table.append(league_tables[i], ignore_index = True)
    
table

	Rk	Squad	MP	W	D	L	GF	GA	GD	Pts	xG	xGA	Season	PPG
0	1	Manchester City	38	27	5	6	102	37	65	86	NaN	NaN	13/14	2.263158
1	2	Liverpool	38	26	6	6	101	50	51	84	NaN	NaN	13/14	2.210526
2	3	Chelsea	38	25	7	6	71	27	44	82	NaN	NaN	13/14	2.157895
3	4	Arsenal	38	24	7	7	68	41	27	79	NaN	NaN	13/14	2.078947
4	6	Tottenham	38	21	6	11	55	51	4	69	NaN	NaN	13/14	1.815789
5	7	Manchester Utd	38	19	7	12	64	43	21	64	NaN	NaN	13/14	1.684211
6	1	Chelsea	38	26	9	3	73	32	41	87	NaN	NaN	14/15	2.289474
7	2	Manchester City	38	24	7	7	83	38	45	79	NaN	NaN	14/15	2.078947
8	3	Arsenal	38	22	9	7	71	36	35	75	NaN	NaN	14/15	1.973684
9	4	Manchester Utd	38	20	10	8	62	37	25	70	NaN	NaN	14/15	1.842105
10	5	Tottenham	38	19	7	12	58	53	5	64	NaN	NaN	14/15	1.684211
11	6	Liverpool	38	18	8	12	52	48	4	62	NaN	NaN	14/15	1.631579
12	2	Arsenal	38	20	11	7	65	36	29	71	NaN	NaN	15/16	1.868421
13	3	Tottenham	38	19	13	6	69	35	34	70	NaN	NaN	15/16	1.842105
14	4	Manchester City	38	19	9	10	71	41	30	66	NaN	NaN	15/16	1.736842
15	5	Manchester Utd	38	19	9	10	49	35	14	66	NaN	NaN	15/16	1.736842
16	8	Liverpool	38	16	12	10	63	50	13	60	NaN	NaN	15/16	1.578947
17	10	Chelsea	38	12	14	12	59	53	6	50	NaN	NaN	15/16	1.315789
18	1	Chelsea	38	30	3	5	85	33	52	93	NaN	NaN	16/17	2.447368
19	2	Tottenham	38	26	8	4	86	26	60	86	NaN	NaN	16/17	2.263158
20	3	Manchester City	38	23	9	6	80	39	41	78	NaN	NaN	16/17	2.052632
21	4	Liverpool	38	22	10	6	78	42	36	76	NaN	NaN	16/17	2.000000
22	5	Arsenal	38	23	6	9	77	44	33	75	NaN	NaN	16/17	1.973684
23	6	Manchester Utd	38	18	15	5	54	29	25	69	NaN	NaN	16/17	1.815789
24	1	Manchester City	38	32	4	2	106	27	79	100	86.4	22.8	17/18	2.631579
25	2	Manchester Utd	38	25	6	7	68	28	40	81	60.3	41.6	17/18	2.131579
26	3	Tottenham	38	23	8	7	74	36	38	77	67.1	35.0	17/18	2.026316
27	4	Liverpool	38	21	12	5	84	38	46	75	72.8	34.0	17/18	1.973684
28	5	Chelsea	38	21	7	10	62	38	24	70	57.5	35.1	17/18	1.842105
29	6	Arsenal	38	19	6	13	74	51	23	63	68.4	50.6	17/18	1.657895
30	1	Manchester City	38	32	2	4	95	23	72	98	82.0	25.3	18/19	2.578947
31	2	Liverpool	38	30	7	1	89	22	67	97	74.7	30.5	18/19	2.552632
32	3	Chelsea	38	21	9	8	63	39	24	72	58.2	35.0	18/19	1.894737
33	4	Tottenham	38	23	2	13	67	39	28	71	57.9	45.1	18/19	1.868421
34	5	Arsenal	38	21	7	10	73	51	22	70	57.5	54.1	18/19	1.842105
35	6	Manchester Utd	38	19	9	10	65	54	11	66	59.6	48.3	18/19	1.736842
36	1	Liverpool	38	32	3	3	85	33	52	99	71.5	40.0	19/20	2.605263
37	2	Manchester City	38	26	3	9	102	35	67	81	93.0	34.7	19/20	2.131579
38	3	Manchester Utd	38	18	12	8	66	36	30	66	59.4	37.4	19/20	1.736842
39	4	Chelsea	38	20	6	12	69	54	15	66	66.6	37.9	19/20	1.736842
40	6	Tottenham	38	16	11	11	61	47	14	59	46.1	52.0	19/20	1.552632
41	8	Arsenal	38	14	14	10	56	48	8	56	49.2	56.6	19/20	1.473684
42	1	Manchester City	38	27	5	6	83	32	51	86	73.3	31.4	20/21	2.263158
43	2	Manchester Utd	38	21	11	6	73	44	29	74	60.2	42.2	20/21	1.947368
44	3	Liverpool	38	20	9	9	68	42	26	69	72.6	45.3	20/21	1.815789
45	4	Chelsea	38	19	10	9	58	36	22	67	64.0	32.8	20/21	1.763158
46	7	Tottenham	38	18	8	12	68	45	23	62	54.5	49.5	20/21	1.631579
47	8	Arsenal	38	18	7	13	55	39	16	61	53.5	44.3	20/21	1.605263

As mentioned before, we have made another set of tables which include xG and xGA in order to perform further analysis on the teams’ attacking and defending scores.

xG and xGA represent how many goals the team should have scored and should have conceded, respectively, across the season. This is calculated through other factors in the game such as key chances created, player form, distance shots are taken from goal, etc. You can learn more about it here.

Therefore, how well a team has performed according to their expectations can be derived from the following equations in the code.

For example, a value of 1 being resulted for Attack Rating (“AtkRt”) when Goals For (“GF”) is divided by xG means that the team lived up to their attacking expectations, greater than 1 if they surpassed it, and less than 1 if they underperformed.

# making a 2nd table for rating the attacking performance of each club
atk_tbl = table[24:]
atk_tbl = atk_tbl[["Squad", "GF", "xG", "PPG", "Season"]]
atk_tbl["AtkRt"] = atk_tbl["GF"] / atk_tbl["xG"]
atk_tbl.head()

	Squad	GF	xG	PPG	Season	AtkRt
24	Manchester City	106	86.4	2.631579	17/18	1.226852
25	Manchester Utd	68	60.3	2.131579	17/18	1.127695
26	Tottenham	74	67.1	2.026316	17/18	1.102832
27	Liverpool	84	72.8	1.973684	17/18	1.153846
28	Chelsea	62	57.5	1.842105	17/18	1.078261

# making a 3rd table for rating the defensive performance of each club
def_tbl = table[24:]
def_tbl = def_tbl[["Squad", "GA", "xGA", "PPG", "Season"]]
def_tbl["DefRt"] = def_tbl["xGA"] / def_tbl["GA"]
def_tbl.head()

	Squad	GA	xGA	PPG	Season	DefRt
24	Manchester City	27	22.8	2.631579	17/18	0.844444
25	Manchester Utd	28	41.6	2.131579	17/18	1.485714
26	Tottenham	36	35.0	2.026316	17/18	0.972222
27	Liverpool	38	34.0	1.973684	17/18	0.894737
28	Chelsea	38	35.1	1.842105	17/18	0.923684

Now is a good time to review the meaning behind each column in the tables...

1. Rnk is the final position of a team in the league for a given season. It is determined by whoever has the most points with 1st being the winner of the EPL
2. Squad is just another way of saying team or club. It is the name of the sport's team
3. MP is matches played which is 38 for all teams
4. W, D, and L stand for wins, draws, and losses. A team gets 3, 1, and 0 points for a win, draw, or loss, respectively
5. GF,GA, and GD stand for Goals For, Goals Allowed, and Goal Difference throughout a league season. GD=GF-GA
6. Pts, and PPG: points are used to determine the final place for each team at the end of the season. PPG=Pts/38
7. Season is the edition of the league that is reported. The league starts in the fall of the beginning year and ends in late spring of the following year.

Exploratory Data Analysis and Visualization

Now that we have our data tidied up, we can look at certain plots to see if there is a trend which suggests if Manchester United is underperforming or not.

plt.subplots(figsize = (15, 8))

# plotting all the points available and differentiating each season by color
i = 0
for s in table["Season"].unique():
    curr_season = seasons[i]
    this_season = table[(table == s).any(axis = 1)]
    
    # excluding Man Utd so that we can make a different marker for them
    this_season = this_season[~this_season["Squad"].str.contains("Manchester Utd")]
    plt.scatter(marker = "o", x = this_season["GF"], y = this_season["PPG"], label = str(curr_season))
    i += 1

# finding the Man Utd data
for ind in table.index:
    squad = table["Squad"][ind]
    ppg = table["PPG"][ind]
    gf = table["GF"][ind]
    if(squad == "Manchester Utd"):
        plt.scatter(marker="x",x = gf, y = ppg)
        plt.annotate("Man Utd " + table["Season"][ind], (gf, ppg))

plt.legend()

# creating regression line

# we get the transpose of the column extracted from the table so that
# it can fit in the array
gf = np.array(table["GF"]).reshape(-1, 1)
ppg = np.array(table["PPG"]).reshape(-1, 1) 
reg = LinearRegression().fit(gf, ppg)
plt.plot(gf, reg.intercept_ + reg.coef_ * gf, '-')

x = "Goals For"
y = "Points per Game"
plt.xlabel(x)
plt.ylabel(y)
plt.title(y + " vs. " + x)
plt.show()

From looking at this plot, we can see that Manchester United is one of the worst attacking teams in the Big 6 because of the low number of goals they score and, as a result, they have a low PPG throughout all the 8 seasons for which we have collected data for. There has not been even one standout season where they can say that their attack has been amongst the best in the Big 6. Let’s now look at if their defense has proved useful during this time.

plt.subplots(figsize = (15, 8))

# plotting all the points available and differentiating each season by color
i = 0
for s in table["Season"].unique():
    curr_season = seasons[i]
    this_season = table[(table == s).any(axis = 1)]
    
    # excluding Man Utd so that we can make a different marker for them
    this_season = this_season[~this_season["Squad"].str.contains("Manchester Utd")]
    plt.scatter(marker = "o", x = this_season["GA"], y = this_season["PPG"], label = str(curr_season))
    i += 1

# finding the Man Utd data
for ind in table.index:
    squad = table["Squad"][ind]
    ppg = table["PPG"][ind]
    ga = table["GA"][ind]
    if(squad == "Manchester Utd"):
        plt.scatter(marker="x",x = ga, y = ppg)
        plt.annotate("Man Utd " + table["Season"][ind], (ga, ppg))

plt.legend()

# creating regression line

# we get the transpose of the column extracted from the table so that
# it can fit in the array
ga = np.array(table["GA"]).reshape(-1, 1)
ppg = np.array(table["PPG"]).reshape(-1, 1) 
reg = LinearRegression().fit(ga, ppg)
plt.plot(ga, reg.intercept_ + reg.coef_ * ga, '-')

x = "Goals Against"
y = "Points per Game"
plt.xlabel(x)
plt.ylabel(y)
plt.title(y + " vs. " + x)
plt.show()

This plot for the defensive side of the teams show more encouragement for Manchester United; In the 17/18 season, we can see that the low number of goals that the team conceded resulted in one of the higher PPGs reported by a Big 6 team during this time. However, that was only one season.

The 16/17 season also saw them concede a low amount of goals, but their PPG was far astray from the Line of Best Fit. Perhaps this can be attributed to the fact that they simply are unable to score decisive goals to turn draws into wins due to their lackluster attacking force. All the other seasons for Manchester United displayed either a poor or mediocre defense in comparison to other Big 6 teams.

Testing

Now comes the time to create a standard to hold these Big 6 teams to, and even compare them with that standard to see who is going above and below expectations. It is safe to say that there have not been any outstanding performances from Manchester United. This is finally where we use our 2 tables which contain our values AtkRt and DefRt.

# constructing a violin plot for attacking scores
plt.subplots(figsize = (15,8))
seaborn.violinplot(x=atk_tbl["Season"], y=atk_tbl["AtkRt"])

<AxesSubplot:xlabel='Season', ylabel='AtkRt'>

The violins in this plot look to be normally distributed with only one spike (i.e. unimodal). The mean appears to be central as well. Therefore, we can proceed with standardization.

# constructing a violin plot for defending scores
plt.subplots(figsize = (15,8))
seaborn.violinplot(x=def_tbl["Season"], y=def_tbl["DefRt"])

<AxesSubplot:xlabel='Season', ylabel='DefRt'>

I am unable to conclude that the DefRt show a normal distribution as there even seems to be a bimodal distribution in the 20/21 season. Hence, we must forego the standardization for this variable and only attempt to report the Standardized Attacking Score.

After standardization, it will be apparent that the teams who have a higher than average AtkRt will have a positive Standardized Attacking Score; this is a result Manchester United should easily achieve if they were to be considered a “top” club in the EPL, at least in terms of on-field performances.

std_vals = []

for s in atk_tbl["Season"].unique():
    curr_season = atk_tbl[(atk_tbl == s).any(axis = 1)]
    
    # calculating mean and standard deviation
    mean = curr_season["AtkRt"].mean()
    std = np.std(curr_season["AtkRt"])
    
    # associating a standardized score with each team for every season
    for ind in curr_season.index:
        atk_rating = curr_season["AtkRt"][ind]
        if (std == 0): 
            std_atk_rt = 0
        else:
            std_atk_rt = (atk_rating - mean)/std
        std_vals.append(std_atk_rt)

# adding the scores to the attack table
atk_tbl["std_atk_rt"] = std_vals

atk_tbl.head()

	Squad	GF	xG	PPG	Season	AtkRt	std_atk_rt
24	Manchester City	106	86.4	2.631579	17/18	1.226852	1.923165
25	Manchester Utd	68	60.3	2.131579	17/18	1.127695	-0.016916
26	Tottenham	74	67.1	2.026316	17/18	1.102832	-0.503385
27	Liverpool	84	72.8	1.973684	17/18	1.153846	0.494754
28	Chelsea	62	57.5	1.842105	17/18	1.078261	-0.984130

Let’s plot the table into a visual representation so that we can compare Manchester United’s standings to other Big 6 clubs, season by season.

i = 4
# plot the information by year, and player.
for s in atk_tbl["Season"].unique():
    plt.subplots(figsize = (15,8))
    
    curr_season = atk_tbl[(atk_tbl == s).any(axis=1)]

    for ind in curr_season.index:
        squad = curr_season["Squad"][ind]
        ppg = curr_season["PPG"][ind]
        std_atk_rt = curr_season["std_atk_rt"][ind]

        # plot that specific year of data
        if (squad == "Manchester Utd"):
            plt.scatter(marker = "x", x = ppg, y = std_atk_rt, label = seasons[i])
            plt.annotate(squad , (ppg, std_atk_rt))
        else :
            plt.scatter(marker = "o", x = ppg, y = std_atk_rt, label = seasons[i]) 
    
    # creating regression line

    # we get the transpose of the column extracted from the table so that
    # it can fit in the array
    std_atk_rt = np.array(atk_tbl["std_atk_rt"]).reshape(-1, 1)
    ppg = np.array(atk_tbl["PPG"]).reshape(-1, 1) 
    reg = LinearRegression().fit(ppg, std_atk_rt)
    plt.plot(ppg, reg.intercept_ + reg.coef_ * ppg, '-')

    y = "Standardized Attack Score"
    x = "Points per Game"
    plt.xlabel(x)
    plt.ylabel(y)
    plt.title(y + " vs. " + x + " in " + seasons[i])
    plt.show()
    
    i += 1

Deriving a Conclusion

Despite the final season, 20/21 season, showing that Manchester United performed above the standard set by the Big 6 clubs, it is hard to refute the notion that they are no longer the top club in the EPL and are struggling to keep pace with the likes of Manchester City. Long gone are the days where Manchester United could display their sheer dominance over rivals and win the Premier League as often as they did with their successful manager. Their performances in attack are simply not up to par to compete with the best of the Big 6 and their defence was seen to be unreliable or inconsistent.

Nevertheless, we have only shown the downfall of Manchester United’s on-field showings; their spending power and their value as a global brand suggest that they are still the biggest club in England.

This study, however, cannot be deemed the final judgement on the fate of Manchester United. Different competitions have not been taken into account; for example, Manchester United won a continental competition known as the Europa League in the 16/17 season. That title, albeit not the EPL, is regarded as a major trophy by soccer fans and clubs in Europe. And yes, we did indeed see a rise in Manchester United’s performances in the league last season in 20/21 when they finished runners-up. It is also noteworthy that their team is comprised of young talents, some of whom already have tons of first-team experience, and their board of directors are only going to facilitate bringing in players of that calibre.

Thank you to everyone who gave some of their time to look at this report behind the failures behind Manchester United.