Baseball attendance model

OLS regression, and the input that made the fit look too good

Author

Arsy Nas (group coursework, UBC Sauder COMM 414)

Published

2022

NoteSummary

Context. COMM 414 Assignment 4, a group assignment from 2022. The brief asked for a regression that predicts attendance at Atlanta Braves home games.

Approach.

  • Data: 461 home games (2011 to 2016), joined to daily weather data.
  • Method: OLS regression with new variables. Train on the seasons before 2015, test on 2015 and 2016.

Findings.

  • The model that we chose has an R² of 0.991 on the training seasons and 0.985 on the test seasons.
  • One input divides tickets sold by attendance, so it contains the value that the model predicts. Without that input, the R² is about 0.95.
  • Weather explains less than 1% of the variation in attendance.
  • A Friday or Saturday game has about 300 more guests.

Limit. This is coursework. It makes no causal claim about attendance or revenue.

1 Setup

import numpy as np
import pandas as pd
import plotly.graph_objects as go
from _plot import show
import statsmodels.formula.api as smf
from sklearn.metrics import r2_score, mean_squared_error, mean_absolute_error
import warnings

warnings.filterwarnings('ignore')
pd.set_option('display.max_columns', None)
pd.set_option('display.precision', 3)

TRAIN_BEFORE = 2015
BLUE, RED = '#2E86AB', '#E84855'

2 Data

Show code
baseball = pd.read_csv('../baseball.csv')
weather = pd.read_csv('../weather.csv')
print(f"Game table: {len(baseball):,} games")
print(f"Weather table: {len(weather):,} records")
Game table: 461 games
Weather table: 2,126 records
Show code
baseball.drop(columns=baseball.columns[0]).head()
Table 1: The first rows of the game table
season opponent event_id event_date event_time win attendance tickets_sold tickets_resale opponent_payroll opponent_rank opponent_win_pcnt braves_payroll braves_rank braves_win_pcnt weekday is_night
0 2011 Arizona Diamondbacks 6347 2011-08-20 19:10:00 1 31147 39783 5052 53.640 28 0.401 87.003 7 0.562 Saturday 1
1 2011 Arizona Diamondbacks 6348 2011-08-21 13:35:00 1 24645 35274 2425 53.640 28 0.401 87.003 7 0.562 Sunday 0
2 2011 Arizona Diamondbacks 6346 2011-08-19 19:35:00 1 23327 30599 3078 53.640 28 0.401 87.003 7 0.562 Friday 1
3 2011 Baltimore Orioles 6323 2011-07-02 19:10:00 1 33468 38766 6759 85.304 27 0.407 87.003 7 0.562 Saturday 1
4 2011 Baltimore Orioles 6324 2011-07-03 13:35:00 0 18801 24580 2415 85.304 27 0.407 87.003 7 0.562 Sunday 0
Show code
df = baseball.join(weather.set_index('date'), on='event_date', how='left')
print(f"Joined table: {len(df):,} rows x {len(df.columns)} columns")
print(f"Missing values: {int(df.isnull().sum().sum())}")
Joined table: 461 rows x 20 columns
Missing values: 0

3 Data preparation

Show code
df['weather'] = df['weather'].astype('category')
df['opponent'] = df['opponent'].astype('category')
df['weekday'] = df['weekday'].astype('category')
df['event_date'] = pd.to_datetime(df['event_date'])
df['event_time'] = pd.to_datetime(df['event_time'], format='%H:%M:%S')
df['weekday'] = df['weekday'].cat.reorder_categories(
    ['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday']
)

4 New variables

We made new variables from the raw columns:

  • Time: the month, and a weekend flag for Friday and Saturday games.
  • Team strength: the difference in win percentage and in rank.
  • Money: the payroll ratio and the payroll difference.
  • Tickets: the log of tickets sold, and the log of tickets sold divided by attendance.
  • Weather: a rain flag and a hot-day flag.
Show code
def engineer_features(df):
    f = df.copy()
    f['month'] = f.event_date.dt.month.astype('category')
    f['is_weekend'] = f['weekday'].apply(lambda x: 1 if x in ['Friday', 'Saturday'] else 0)
    f['win_pct_diff'] = np.abs(f['opponent_win_pcnt'] - f['braves_win_pcnt'])
    f['rank_diff'] = f.opponent_rank - f.braves_rank
    f['payroll_ratio'] = f['braves_payroll'] / (f['braves_payroll'] + f['opponent_payroll'])
    f['payroll_diff'] = np.abs(f['opponent_payroll'] - f['braves_payroll']).astype(int)
    f['ticket_ratio'] = f['tickets_sold'] / f['attendance']          # contains the target
    f['log_ticket_ratio'] = np.log(f['ticket_ratio'])
    f['log_tickets_resale'] = np.log(f['tickets_sold'])              # the name is wrong: this is the log of tickets sold
    f['precipitation'] = (f['weather'] == 'Rain') * 1
    f['is_hot'] = f['mean_temp'] > 21
    return f

df = engineer_features(df)
print(f"Columns after the new variables: {len(df.columns)}")
Columns after the new variables: 31
WarningTwo problems in these variables
  • log_ticket_ratio uses attendance, the value that the model predicts. The section “The input that contained the answer” shows the effect.
  • log_tickets_resale has the wrong name. It holds the log of tickets sold, in the 2022 notebook and here.

5 Exploratory analysis

Show code
numeric_cols = df.select_dtypes(include=[np.number]).columns.drop(df.columns[0])
corr = df[numeric_cols].corr()[['attendance']].sort_values(by='attendance', ascending=False).head(15)

fig = go.Figure(data=go.Heatmap(
    z=corr.values, x=['Attendance'], y=corr.index, colorscale='RdBu_r', zmid=0,
    text=np.round(corr.values, 3), texttemplate='%{text}', colorbar=dict(title='Correlation'),
))
fig.update_layout(title='Variables by correlation with attendance', height=560)
show(fig)
Figure 1: Correlation of each numeric variable with attendance

Tickets sold has the strongest correlation with attendance. The weather variables have a weak correlation.

6 Models

6.1 Split by season

Show code
df_train = df.query(f'season < {TRAIN_BEFORE}')
df_test = df.query(f'season >= {TRAIN_BEFORE}')
print(f"Training set: {len(df_train)} games (2011 to 2014)")
print(f"Test set: {len(df_test)} games (2015 and 2016)")
Training set: 324 games (2011 to 2014)
Test set: 137 games (2015 and 2016)

6.2 Six models

Show code
model_1 = smf.ols(data=df_train, formula='attendance ~ opponent + event_time + tickets_sold + tickets_resale + '
                  'opponent_rank + opponent_win_pcnt + braves_rank + braves_win_pcnt + weekday + is_night').fit()
model_2 = smf.ols(data=df_train, formula='attendance ~ log_tickets_resale + tickets_sold + is_weekend + log_ticket_ratio').fit()
model_3 = smf.ols(data=df_train, formula='attendance ~ log_tickets_resale + tickets_sold + event_time').fit()
model_4 = smf.ols(data=df_train, formula='attendance ~ weather').fit()
model_5 = smf.ols(data=df_train, formula='attendance ~ mean_temp').fit()
model_6 = smf.ols(data=df_train, formula='attendance ~ weather + mean_temp').fit()
Show code
models = {
    '1. All variables': model_1,
    '2. The model that we chose': model_2,
    '3. Tickets and start time': model_3,
    '4. Weather type': model_4,
    '5. Temperature': model_5,
    '6. Weather type and temperature': model_6,
}
comparison = pd.DataFrame({
    'Model': list(models),
    'Parameters': [int(m.df_model) for m in models.values()],
    'R²': [m.rsquared for m in models.values()],
    'Adjusted R²': [m.rsquared_adj for m in models.values()],
    'AIC': [m.aic for m in models.values()],
})
comparison.style.format({'R²': '{:.3f}', 'Adjusted R²': '{:.3f}', 'AIC': '{:,.0f}'}).hide(axis='index')
Table 2: The six models on the training seasons
Model Parameters R² Adjusted R² AIC
1. All variables 51 0.959 0.952 5,909
2. The model that we chose 4 0.991 0.991 5,326
3. Tickets and start time 18 0.948 0.945 5,924
4. Weather type 2 0.002 -0.004 6,848
5. Temperature 1 0.001 -0.003 6,846
6. Weather type and temperature 3 0.002 -0.007 6,849

We chose model 2. It has the highest adjusted R² and the lowest AIC, with four inputs.

Show code
print(model_2.summary())
                            OLS Regression Results                            
==============================================================================
Dep. Variable:             attendance   R-squared:                       0.991
Model:                            OLS   Adj. R-squared:                  0.991
Method:                 Least Squares   F-statistic:                     8787.
Date:                Wed, 18 Nov 2026   Prob (F-statistic):               0.00
Time:                        10:07:01   Log-Likelihood:                -2657.9
No. Observations:                 324   AIC:                             5326.
Df Residuals:                     319   BIC:                             5345.
Df Model:                           4                                         
Covariance Type:            nonrobust                                         
======================================================================================
                         coef    std err          t      P>|t|      [0.025      0.975]
--------------------------------------------------------------------------------------
Intercept           1.106e+05   8570.933     12.901      0.000    9.37e+04    1.27e+05
log_tickets_resale -1.141e+04    915.561    -12.461      0.000   -1.32e+04   -9607.946
tickets_sold           1.1595      0.030     39.061      0.000       1.101       1.218
is_weekend           300.2873    127.885      2.348      0.019      48.682     551.893
log_ticket_ratio   -1.789e+04    443.367    -40.354      0.000   -1.88e+04    -1.7e+04
==============================================================================
Omnibus:                      130.180   Durbin-Watson:                   1.843
Prob(Omnibus):                  0.000   Jarque-Bera (JB):             1774.720
Skew:                          -1.256   Prob(JB):                         0.00
Kurtosis:                      14.187   Cond. No.                     5.73e+06
==============================================================================

Notes:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
[2] The condition number is large, 5.73e+06. This might indicate that there are
strong multicollinearity or other numerical problems.

7 Interpretation

Show code
coeffs = pd.DataFrame({
    'Variable': model_2.params.index,
    'Coefficient': model_2.params.values,
    'Std error': model_2.bse.values,
    'p-value': model_2.pvalues.values,
})
coeffs.style.format({'Coefficient': '{:,.2f}', 'Std error': '{:,.2f}', 'p-value': '{:.4f}'}).hide(axis='index')
Table 3: Coefficients of model 2
Variable Coefficient Std error p-value
Intercept 110,569.88 8,570.93 0.0000
log_tickets_resale -11,409.25 915.56 0.0000
tickets_sold 1.16 0.03 0.0000
is_weekend 300.29 127.89 0.0195
log_ticket_ratio -17,891.64 443.37 0.0000
  • Tickets sold. Each ticket sold adds about 1.16 to the predicted attendance, with the other inputs held constant.
  • Weekend. A Friday or Saturday game has about 300 more guests.
  • log_tickets_resale. This is the log of tickets sold (see “New variables”). It is not a resale effect.
  • log_ticket_ratio. This input contains attendance. Its large coefficient is the leak, not a business effect.

8 Evaluation on the test seasons

Show code
cols = ['log_tickets_resale', 'tickets_sold', 'is_weekend', 'log_ticket_ratio']
y_pred = model_2.predict(df_test[cols])
y_actual = df_test['attendance']

r2 = r2_score(y_actual, y_pred)
rmse = np.sqrt(mean_squared_error(y_actual, y_pred))
mae = mean_absolute_error(y_actual, y_pred)
print(f"Test R²: {r2:.4f}")
print(f"RMSE: {rmse:,.0f} guests")
print(f"MAE: {mae:,.0f} guests")
Test R²: 0.9855
RMSE: 1,062 guests
MAE: 636 guests
Show code
fig = go.Figure()
fig.add_trace(go.Scatter(x=y_pred, y=y_actual, mode='markers', name='Games',
                         marker=dict(size=8, color=BLUE, opacity=0.6)))
lo, hi = min(y_pred.min(), y_actual.min()), max(y_pred.max(), y_actual.max())
fig.add_trace(go.Scatter(x=[lo, hi], y=[lo, hi], mode='lines', name='Perfect prediction',
                         line=dict(color=RED, dash='dash')))
fig.update_layout(title=f'Predicted and actual attendance (R² = {r2:.3f})',
                  xaxis_title='Predicted attendance', yaxis_title='Actual attendance', height=520)
show(fig)
Figure 2: Predicted and actual attendance on the 2015 and 2016 games (model 2, with the leaked input)

These values are too good. The next section shows why.

9 The input that contained the answer

log_ticket_ratio is the log of tickets sold divided by attendance. Attendance is the value that the model predicts, so this input gives the model a part of the answer. A team does not know the attendance before the game.

The model below is the same model without that input. It was added in the 2026 check.

Show code
model_clean = smf.ols(data=df_train, formula='attendance ~ np.log(tickets_resale) + tickets_sold + is_weekend').fit()
pred_clean = model_clean.predict(df_test)

leak = pd.DataFrame({
    'Model': ['With the leaked input (submitted)', 'Without the leaked input (2026 check)'],
    'R², training seasons': [model_2.rsquared, model_clean.rsquared],
    'R², test seasons': [r2, r2_score(y_actual, pred_clean)],
    'MAE on the test seasons (guests)': [mae, mean_absolute_error(y_actual, pred_clean)],
})
leak.style.format({'R², training seasons': '{:.3f}', 'R², test seasons': '{:.3f}',
                   'MAE on the test seasons (guests)': '{:,.0f}'}).hide(axis='index')
Table 4: Model 2 with and without the leaked input
Model R², training seasons R², test seasons MAE on the test seasons (guests)
With the leaked input (submitted) 0.991 0.986 636
Without the leaked input (2026 check) 0.953 0.955 1,289

The fit to quote for this model is about 0.95.

10 Weather

Show code
weather_tbl = pd.DataFrame({
    'Model': ['Weather type', 'Temperature', 'Weather type and temperature'],
    'R²': [model_4.rsquared, model_5.rsquared, model_6.rsquared],
    'Smallest p-value of a weather variable': [
        model_4.pvalues.drop('Intercept').min(),
        model_5.pvalues.drop('Intercept').min(),
        model_6.pvalues.drop('Intercept').min(),
    ],
})
weather_tbl.style.format({'R²': '{:.3f}', 'Smallest p-value of a weather variable': '{:.2f}'}).hide(axis='index')
Table 5: The three weather models on the training seasons
Model R² Smallest p-value of a weather variable
Weather type 0.002 0.49
Temperature 0.001 0.68
Weather type and temperature 0.002 0.48

Weather explains less than 1% of the variation in attendance, and no weather variable is significant. A forecast for this team does not need the weather.

11 What a team can use

  1. Tickets sold is the first signal for staff and stock plans. It explains most of the attendance.
  2. Plan for more guests on Friday and Saturday games.
  3. Do not change the forecast for the weather.
  4. Track the resale ratio for each game as a sign of demand.

12 Limits and next steps

  • Home games of one team, 2011 to 2016. The model does not apply to other teams or stadiums without a new fit.
  • No data on promotions or on star players of the opponent.
  • A linear model does not show interactions between variables.
  • Next steps: add promotion days, fit the model without the leaked input from the start, and compare with a tree model.
Show code
residuals = y_actual - y_pred
print(f"Residuals on the test seasons (model 2): mean {residuals.mean():,.0f}, "
      f"std {residuals.std():,.0f}, min {residuals.min():,.0f}, max {residuals.max():,.0f}")
Residuals on the test seasons (model 2): mean 164, std 1,053, min -2,380, max 4,982