Win prediction

Logistic regression, compared with a simple baseline

Author

Arsy Nas (group coursework, UBC Sauder COMM 414)

Published

2022

NoteSummary

Problem. COMM 414 Assignment 4, a group assignment from 2022. Predict a win or a loss for each Atlanta Braves home game from values that are known before the game.

Method. Logistic regression for four groups of variables, then a final model. A random 70/30 split with a fixed seed.

Results.

  • The final model is correct for 56.1% of the test games.
  • The team won 51.1% of the test games. A model that predicts a win for each game gets that score, so the model adds about five points.
  • Team performance is the best group of variables. Payroll alone, the schedule and the weather add almost nothing.

Limit. This is coursework. The model is a small step above the baseline and is not a tool to predict games.

1 Setup

Libraries, data and new variables
import numpy as np
import pandas as pd
import statsmodels.formula.api as smf
import plotly.graph_objects as go
import plotly.express as px
from _plot import show
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix, accuracy_score


baseball = pd.read_csv('../baseball.csv')
weather = pd.read_csv('../weather.csv')
df = baseball.join(weather.set_index('date'), on='event_date', how='left')

df['weather'] = df['weather'].astype('category')
df['is_night'] = df['is_night'].astype('category')
df['weekday'] = df['weekday'].astype('category')
df['opponent_rank'] = df['opponent_rank'].astype('int64')
df['braves_rank'] = df['braves_rank'].astype('int64')

df['is_weekend'] = df['weekday'].apply(lambda d: 1 if d in ['Friday', 'Saturday'] else 0)
df['payroll_ratio'] = df['braves_payroll'] / (df['braves_payroll'] + df['opponent_payroll'])

print(f"Games: {len(df):,} | share of wins: {df['win'].mean():.1%}")
Games: 461 | share of wins: 54.9%

2 Split

Show code
train, test = train_test_split(df, test_size=0.3, shuffle=True, random_state=0)
print(f"Training set: {len(train)} games | test set: {len(test)} games")
print(f"Share of wins: training {train['win'].mean():.1%} | test {test['win'].mean():.1%}")
Training set: 322 games | test set: 139 games
Share of wins: training 56.5% | test 51.1%

3 Four groups of variables

Idea: a higher payroll buys better players, so the team wins more.

Show code
model_1 = smf.logit(data=train, formula='win ~ braves_payroll + opponent_payroll').fit(disp=0)
model_1.summary()
Logit Regression Results
Dep. Variable: win No. Observations: 322
Model: Logit Df Residuals: 319
Method: MLE Df Model: 2
Date: Wed, 18 Nov 2026 Pseudo R-squ.: 0.004169
Time: 10:07:05 Log-Likelihood: -219.53
converged: True LL-Null: -220.45
Covariance Type: nonrobust LLR p-value: 0.3989
coef std err z P>|z| [0.025 0.975]
Intercept 1.7008 1.119 1.520 0.129 -0.493 3.895
braves_payroll -0.0138 0.012 -1.145 0.252 -0.037 0.010
opponent_payroll -0.0014 0.003 -0.525 0.600 -0.007 0.004

Idea: the rank and the win percentage of the two teams show which team is stronger.

Show code
model_2 = smf.logit(data=train, formula='win ~ opponent_rank + opponent_win_pcnt + braves_rank + braves_win_pcnt').fit(disp=0)
model_2.summary()
Logit Regression Results
Dep. Variable: win No. Observations: 322
Model: Logit Df Residuals: 317
Method: MLE Df Model: 4
Date: Wed, 18 Nov 2026 Pseudo R-squ.: 0.01952
Time: 10:07:05 Log-Likelihood: -216.14
converged: True LL-Null: -220.45
Covariance Type: nonrobust LLR p-value: 0.07175
coef std err z P>|z| [0.025 0.975]
Intercept -11.4821 13.089 -0.877 0.380 -37.135 14.171
opponent_rank 0.0487 0.080 0.610 0.542 -0.108 0.205
opponent_win_pcnt 5.5960 10.413 0.537 0.591 -14.812 26.004
braves_rank 0.0638 0.134 0.476 0.634 -0.199 0.327
braves_win_pcnt 13.9357 17.828 0.782 0.434 -21.006 48.878

Idea: weekend games and night games have a different result.

Show code
model_3 = smf.logit(data=train, formula='win ~ is_weekend + is_night').fit(disp=0)
model_3.summary()
Logit Regression Results
Dep. Variable: win No. Observations: 322
Model: Logit Df Residuals: 319
Method: MLE Df Model: 2
Date: Wed, 18 Nov 2026 Pseudo R-squ.: 0.007022
Time: 10:07:05 Log-Likelihood: -218.90
converged: True LL-Null: -220.45
Covariance Type: nonrobust LLR p-value: 0.2127
coef std err z P>|z| [0.025 0.975]
Intercept 0.4595 0.235 1.958 0.050 -0.001 0.920
is_night[T.1] -0.3915 0.270 -1.450 0.147 -0.921 0.138
is_weekend 0.3037 0.246 1.233 0.217 -0.179 0.786

Idea: temperature and rain change how the teams play.

Show code
model_4 = smf.logit(data=train, formula='win ~ weather + mean_temp').fit(disp=0)
model_4.summary()
Logit Regression Results
Dep. Variable: win No. Observations: 322
Model: Logit Df Residuals: 318
Method: MLE Df Model: 3
Date: Wed, 18 Nov 2026 Pseudo R-squ.: 0.004247
Time: 10:07:05 Log-Likelihood: -219.51
converged: True LL-Null: -220.45
Covariance Type: nonrobust LLR p-value: 0.5993
coef std err z P>|z| [0.025 0.975]
Intercept 1.3332 1.008 1.323 0.186 -0.642 3.309
weather[T.Rain] -1.0369 0.822 -1.261 0.207 -2.649 0.575
weather[T.Sunny] -1.0002 0.815 -1.227 0.220 -2.597 0.597
mean_temp -0.0034 0.026 -0.132 0.895 -0.054 0.047
Show code
ms = {'1. Payroll': model_1, '2. Team performance': model_2, '3. Schedule': model_3, '4. Weather': model_4}
comparison = pd.DataFrame({
    'Model': list(ms),
    'Pseudo R²': [m.prsquared for m in ms.values()],
    'Log-likelihood': [m.llf for m in ms.values()],
    'AIC': [m.aic for m in ms.values()],
})
comparison.style.format({'Pseudo R²': '{:.4f}', 'Log-likelihood': '{:,.2f}', 'AIC': '{:,.2f}'}).hide(axis='index')
Table 1: The four models on the training games
Model Pseudo R² Log-likelihood AIC
1. Payroll 0.0042 -219.53 445.05
2. Team performance 0.0195 -216.14 442.29
3. Schedule 0.0070 -218.90 443.80
4. Weather 0.0042 -219.51 447.02

Team performance has the highest pseudo R² and the lowest AIC. All the values are small: no group of variables explains much of the result of one game.

4 The final model

The final model uses the team-performance variables and adds a payroll ratio.

Show code
model_logit = smf.logit(
    data=train,
    formula='win ~ opponent_rank + opponent_win_pcnt + braves_rank + braves_win_pcnt + payroll_ratio',
).fit(disp=0)
model_logit.summary()
Logit Regression Results
Dep. Variable: win No. Observations: 322
Model: Logit Df Residuals: 316
Method: MLE Df Model: 5
Date: Wed, 18 Nov 2026 Pseudo R-squ.: 0.02046
Time: 10:07:05 Log-Likelihood: -215.94
converged: True LL-Null: -220.45
Covariance Type: nonrobust LLR p-value: 0.1082
coef std err z P>|z| [0.025 0.975]
Intercept -9.4877 13.459 -0.705 0.481 -35.866 16.891
opponent_rank 0.0371 0.082 0.452 0.651 -0.124 0.198
opponent_win_pcnt 3.4608 10.939 0.316 0.752 -17.980 24.902
braves_rank 0.0578 0.135 0.429 0.668 -0.206 0.322
braves_win_pcnt 13.4928 17.852 0.756 0.450 -21.497 48.482
payroll_ratio -0.9362 1.451 -0.645 0.519 -3.780 1.907

The probability of a win is

P(win)=11+e−z P(\text{win}) = \frac{1}{1 + e^{-z}}

where zz is the sum of each coefficient times its variable. The win percentage of the Braves has the largest coefficient. Read it as a direction only: this is a coursework model, and the coefficient is not a causal estimate.

5 Evaluation

Show code
cols = ['opponent_rank', 'opponent_win_pcnt', 'braves_rank', 'braves_win_pcnt', 'payroll_ratio']
y_test = test['win']
y_prob = model_logit.predict(test[cols])
y_pred = (y_prob > 0.50).astype(int)

cm = confusion_matrix(y_test, y_pred)
acc = accuracy_score(y_test, y_pred)

fig = go.Figure(data=go.Heatmap(
    z=cm, x=['Predicted: loss', 'Predicted: win'], y=['Actual: loss', 'Actual: win'],
    text=cm, texttemplate='%{text}', colorscale=[[0, '#EEF2F7'], [1, '#2E86AB']], showscale=False,
))
fig.update_layout(title=f'Confusion matrix (accuracy {acc:.1%})', height=400, width=520)
show(fig)
Figure 1: Confusion matrix on the test games, threshold 0.5
Show code
baseline = max(y_test.mean(), 1 - y_test.mean())
acc_09 = accuracy_score(y_test, (y_prob > 0.90).astype(int))
result = pd.DataFrame({
    'Rule': ['Final model, threshold 0.5', 'Always predict a win (baseline)', 'First draft: final model, threshold 0.9'],
    'Accuracy': [acc, baseline, acc_09],
    'Games correct': [int((y_pred == y_test).sum()), int(y_test.sum()), int(((y_prob > 0.90).astype(int) == y_test).sum())],
    'Wins predicted': [int(y_pred.sum()), len(y_test), int((y_prob > 0.90).sum())],
})
result.style.format({'Accuracy': '{:.1%}'}).hide(axis='index')
Table 2: The model against two simple rules on the test games
Rule Accuracy Games correct Wins predicted
Final model, threshold 0.5 56.1% 78 102
Always predict a win (baseline) 51.1% 71 139
First draft: final model, threshold 0.9 48.9% 68 0
WarningRead the accuracy against the baseline

The team won 51.1% of the test games. A model that predicts a win for each game is correct that often. The correct comparison is 56.1% against 51.1%, not against a coin flip.

Our first draft used a threshold of 0.9. No game had a win probability that high, so the draft predicted a loss for each game.

Show code
fig2 = px.histogram(x=y_prob, nbins=40, color_discrete_sequence=['#2E86AB'],
                    labels={'x': 'Predicted win probability'}, title='Predicted win probabilities')
fig2.add_vline(x=0.5, line_dash='dash', line_color='#E84855', annotation_text='Threshold 0.5')
fig2.update_layout(height=380, yaxis_title='Games')
show(fig2)
Figure 2: Predicted win probability for each test game

Most probabilities are near 0.5. The model is not sure about most games.

6 Findings

  1. The season record of the team is the strongest signal for one game.
  2. Payroll alone does not predict a win.
  3. Schedule and weather add almost nothing.
  4. Next steps: add the starting pitchers, the results of the last ten games and injuries. We did not test these.