Win prediction: five points better than a simple guess

A group assignment. We built a logistic regression that predicts a win or a loss for each home game, and compared it with a simple baseline.

Role
Group member (coursework)
Period
2022
Stack
Python, statsmodels, Logistic regression
Evidence
Published
Question
Which groups of variables help to predict a win, and how much better than a simple guess is the model?
Data
The same 461 home games, with payroll, team rank, win percentage, schedule and weather variables.
Method
We fitted one logistic model for each group of variables. Then we fitted a final model with rank, win percentage and a payroll ratio. We used a 70/30 random split and a 0.5 threshold.
ResultPublished
The final model was correct for 56.1% of the test games. The team won 51.1% of the test games, so the model adds about five points.

Coursework output from a group assignment. The model is only a small step above the baseline. It is not a tool to predict games.

56.1%Test accuracy
51.1%Baseline: always predict a win
0.5Threshold
70 / 30Train / test split

1 · Business problem

Predict a win or a loss for each Atlanta Braves home game from values that are known before the game. The assignment also asked which groups of variables help.

2 · Data and split

  • The 461 home games of 2011 to 2016. The team won 54.9% of them.
  • A random 70/30 split with a fixed seed: 322 training games and 139 test games.
  • The team won 56.5% of the training games and 51.1% of the test games.

3 · Four groups of variables

We fitted one logistic model for each group. The value is the pseudo R² on the training games.

  • Payroll of the two teams: 0.0042.
  • Team performance (rank and win percentage of the two teams): 0.0195.
  • Schedule (weekend and night games): 0.0070.
  • Weather (type and mean temperature): 0.0042.

4 · The final model

Team performance was the best group, so the final model uses it and adds a payroll ratio.

The win percentage of the Braves has the largest coefficient (13.49). The payroll models show that payroll alone does not predict a win in one game.

All the pseudo R² values are small. No group of variables explains much of the result of one game.

5 · Evaluation

On the 139 test games, the model was correct for 78: 56 wins and 22 losses. That is 56.1%.

An accuracy value has no meaning without a baseline. A model that predicts a win for each game is correct 51.1% of the time on this test set. The correct comparison is 56.1% against 51.1%, not against zero.

6 · The first draft predicted only losses

Our first draft used a threshold of 0.9. No game had a win probability that high, so the model predicted a loss for each game.

We changed the threshold to 0.5. Then the model predicted wins and losses, and the accuracy became 56.1%.

7 · Findings

  • The season record of the team is the strongest signal for one game.
  • Payroll alone does not predict a win.
  • Schedule and weather add almost nothing.
  • Next steps: add the starting pitchers, the results of the last ten games and injuries.

Accuracy on the test games

Accuracy on the test games
ItemValue (%)
Logistic model56.1
Always predict a win51.1
Final model and threshold (from the assignment notebook)
train, test = train_test_split(df, test_size=0.3, shuffle=True, random_state=0)

model_logit = smf.logit(
    data=train,
    formula='win ~ opponent_rank + opponent_win_pcnt + braves_rank + braves_win_pcnt + payroll_ratio',
).fit()

yhat = model_logit.predict(X)
prediction = (yhat > 0.50).astype(int)   # the first draft used 0.90