Resale alerts and a cinema screen plan

A classification tree and a Monte Carlo simulation

Author

Arsy Nas (group coursework, UBC Sauder COMM 414)

Published

2022

NoteSummary

COMM 414 Assignment 5, a group assignment from 2022. It has two separate parts.

Part A: ticket resale alerts (classification tree)

  • The tree gives each baseball game a resale alert level: low, medium or high.
  • At depth 6 it is correct for 82.1% of the test games. A tree with no depth limit is correct for 87.5%. The most frequent level is correct for 58.9%.
  • The inputs include the two ticket counts that define the alert level, so a high accuracy is the expected result.

Part B: cinema screen plan (Monte Carlo simulation)

  • 100,000 simulated nights for each plan, with Poisson demand.
  • Expected revenue goes from about $5,755 with the plan of the manager to about $5,974 with our plan (modelled).
  • Our plan also makes the $6,000 target more probable. The size of that change depends on the demand model, so this page does not give one value for it.

1 Setup

Show code
import numpy as np
import pandas as pd
import plotly.graph_objects as go
import plotly.express as px
from _plot import show
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import confusion_matrix, accuracy_score

2 Part A: ticket resale alerts

2.1 Context

A baseball team watches the resale market for its tickets. A high share of resale tickets can show high demand, a low price or resellers at work. The brief defines an alert level for each game from the resale ratio.

2.2 Data preparation

Resale ratio and alert level
baseball = pd.read_csv('../baseball.csv')
baseball['resale_ratio'] = baseball['tickets_resale'] / baseball['tickets_sold']

def alert_level(resale_ratio):
    if resale_ratio <= 0.075:
        return 'low'
    elif resale_ratio <= 0.1:
        return 'medium'
    return 'high'

baseball['alert'] = baseball['resale_ratio'].apply(alert_level)
alert_counts = baseball['alert'].value_counts().reindex(['low', 'medium', 'high'])
print("Games in each alert level:")
print(alert_counts.to_string())
print(f"\nMean resale ratio: {baseball['resale_ratio'].mean():.3f}")
Games in each alert level:
alert
low       209
medium    161
high       91

Mean resale ratio: 0.083
Show code
fig = px.bar(x=alert_counts.index, y=alert_counts.values, color=alert_counts.index,
             color_discrete_map={'low': '#3DDA84', 'medium': '#F4B942', 'high': '#E84855'},
             labels={'x': 'Alert level', 'y': 'Games'}, title='Games in each resale alert level')
fig.update_layout(showlegend=False, height=380)
show(fig)
Figure 1: Games in each resale alert level, 2011 to 2016

2.3 Split by season

Show code
train_bb = baseball.query('2011 <= season <= 2015')
test_bb = baseball.query('season == 2016')
features = ['attendance', 'tickets_sold', 'tickets_resale', 'opponent_payroll', 'opponent_rank', 'opponent_win_pcnt']
x_train, y_train = train_bb[features], train_bb['alert']
x_test, y_test = test_bb[features], test_bb['alert']
print(f"Training set: {len(x_train)} games (2011 to 2015)")
print(f"Test set: {len(x_test)} games (2016)")
Training set: 405 games (2011 to 2015)
Test set: 56 games (2016)

2.4 Tree with no depth limit

Show code
clf_full = DecisionTreeClassifier(criterion='entropy', random_state=0).fit(x_train, y_train)
train_acc_full, test_acc_full = clf_full.score(x_train, y_train), clf_full.score(x_test, y_test)
print(f"Training accuracy: {train_acc_full:.1%}")
print(f"Test accuracy: {test_acc_full:.1%}")
Training accuracy: 100.0%
Test accuracy: 87.5%

The tree learns the training games fully. The gap between the two values shows that it fits the training seasons more closely than the test season.

2.5 Depth tuning

Show code
depths = list(range(1, 15))
train_scores, test_scores = [], []
for d in depths:
    clf_d = DecisionTreeClassifier(criterion='entropy', random_state=0, max_depth=d).fit(x_train, y_train)
    train_scores.append(clf_d.score(x_train, y_train))
    test_scores.append(clf_d.score(x_test, y_test))

fig = go.Figure()
fig.add_trace(go.Scatter(x=depths, y=train_scores, mode='lines+markers', name='Training', line=dict(color='#2E86AB', width=2)))
fig.add_trace(go.Scatter(x=depths, y=test_scores, mode='lines+markers', name='Test', line=dict(color='#3DDA84', width=2)))
fig.add_vline(x=6, line_dash='dash', line_color='#E84855', annotation_text='Depth that we chose', annotation_position='top right')
fig.update_layout(title='Accuracy for each tree depth', xaxis_title='Maximum depth', yaxis_title='Accuracy',
                  yaxis=dict(tickformat='.0%'), height=420)
show(fig)
Figure 2: Training and test accuracy for each tree depth

2.6 The tree that we chose: depth 6

Show code
clf_opt = DecisionTreeClassifier(criterion='entropy', random_state=0, max_depth=6).fit(x_train, y_train)
y_pred = clf_opt.predict(x_test)
cm = confusion_matrix(y_test, y_pred, labels=['low', 'medium', 'high'])
acc = accuracy_score(y_test, y_pred)

fig = go.Figure(data=go.Heatmap(
    z=cm, x=['Predicted: low', 'Predicted: medium', 'Predicted: high'], y=['Actual: low', 'Actual: medium', 'Actual: high'],
    text=cm, texttemplate='%{text}', colorscale=[[0, '#EEF2F7'], [1, '#2E86AB']], showscale=False,
))
fig.update_layout(title=f'Confusion matrix, depth 6 (accuracy {acc:.1%})', height=400)
show(fig)
Figure 3: Confusion matrix of the depth-6 tree on the 2016 games
Show code
baseline = y_test.value_counts(normalize=True).max()
results = pd.DataFrame({
    'Model': ['Tree with no depth limit', 'Tree with depth 6 (submitted)', 'Most frequent level (baseline)'],
    'Training accuracy': [f'{train_acc_full:.1%}', f'{clf_opt.score(x_train, y_train):.1%}', '-'],
    'Test accuracy': [f'{test_acc_full:.1%}', f'{acc:.1%}', f'{baseline:.1%}'],
})
results.style.hide(axis='index')
Table 1: The two trees and the baseline
Model Training accuracy Test accuracy
Tree with no depth limit 100.0% 87.5%
Tree with depth 6 (submitted) 94.8% 82.1%
Most frequent level (baseline) - 58.9%
WarningTwo limits of this result
  • Depth 6 is not the most accurate tree on this test season. The tree with no limit has a higher test accuracy. The group chose depth 6 as the balance between the two accuracy values. The depth-6 tree is smaller and easier to read.
  • The inputs contain the rule. The alert level is calculated from resale tickets and tickets sold, and the two counts are inputs to the tree. A useful alert model would predict the level before the resale count is known.

2.7 The resale market: for and against

For: fans can buy and sell tickets easily and can change games. A lower resale price can bring more guests, who also buy food and merchandise.

Against: resellers make a profit on the product of the team. The team has less control of prices, season tickets can lose value, and the market is difficult to monitor.

3 Part B: cinema screen plan

3.1 Context

A cinema has five screens and shows three films on one night:

  • Avatar: $19.99 for an IMAX seat.
  • Black Adam: $12.99 for a seat.
  • The Fabelmans: $12.99 for a seat.

The manager gives each screen a film before she knows the demand. Her revenue target for the night is $6,000.

3.2 Simulation parameters

Show code
lam = 400          # mean number of guests in one night
sims = 100_000     # simulated nights for each plan
np.random.seed(42)

p_avatar, p_black_adam, p_the_fabelmans = 0.4, 0.4, 0.2
price_imax, price_reg = 19.99, 12.99
cap_A, cap_B, cap_C, cap_D, cap_E = 120, 120, 80, 80, 40   # 440 seats

# As in the submitted notebook: one Poisson draw for each film
exp_avatar = np.random.poisson(lam, sims) * p_avatar
exp_black_adam = np.random.poisson(lam, sims) * p_black_adam
exp_fabelmans = np.random.poisson(lam, sims) * p_the_fabelmans

print(f"Simulated nights: {sims:,}")
print(f"Mean demand: Avatar {exp_avatar.mean():.0f} | Black Adam {exp_black_adam.mean():.0f} | The Fabelmans {exp_fabelmans.mean():.0f}")
Simulated nights: 100,000
Mean demand: Avatar 160 | Black Adam 160 | The Fabelmans 80

3.3 The two plans

Screen Film Type Seats
A Avatar IMAX 120
B Avatar IMAX 120
C Black Adam Regular 80
D Black Adam Regular 80
E The Fabelmans Regular 40
Show code
s_A = np.minimum(exp_avatar, cap_A)
s_B = np.minimum(exp_avatar - s_A, cap_B)
s_C = np.minimum(exp_black_adam, cap_C)
s_D = np.minimum(exp_black_adam - s_C, cap_D)
s_E = np.minimum(exp_fabelmans, cap_E)
s_rev = (s_A + s_B) * price_imax + (s_C + s_D + s_E) * price_reg
sarah_avg = s_rev.mean()

print(f"Expected revenue: ${sarah_avg:,.2f}")
print(f"Mean seats filled: {(s_A + s_B + s_C + s_D + s_E).mean():.0f} of 440")
for name, s, cap in [('A, Avatar IMAX', s_A, cap_A), ('B, Avatar IMAX', s_B, cap_B), ('C, Black Adam', s_C, cap_C),
                     ('D, Black Adam', s_D, cap_D), ('E, The Fabelmans', s_E, cap_E)]:
    print(f"  Screen {name}: {s.mean():.0f} of {cap}")
Expected revenue: $5,754.86
Mean seats filled: 357 of 440
  Screen A, Avatar IMAX: 120 of 120
  Screen B, Avatar IMAX: 40 of 120
  Screen C, Black Adam: 80 of 80
  Screen D, Black Adam: 77 of 80
  Screen E, The Fabelmans: 40 of 40

This plan gives 240 seats to Avatar for about 160 guests, and 40 seats to The Fabelmans for about 80 guests.

Screen Film Type Seats
A Avatar IMAX 120
B Black Adam Regular 120
C Avatar Regular 80
D The Fabelmans Regular 80
E Black Adam Regular 40

Avatar gets 200 seats, Black Adam gets 160 and The Fabelmans gets 80.

Show code
o_A = np.minimum(exp_avatar, cap_A)
o_B = np.minimum(exp_black_adam, cap_B)
o_C = np.minimum(exp_avatar - o_A, cap_C)
o_D = np.minimum(exp_fabelmans, cap_D)
o_E = np.minimum(exp_black_adam - o_B, cap_E)
o_rev = o_A * price_imax + (o_B + o_C + o_D + o_E) * price_reg
opt_avg = o_rev.mean()

print(f"Expected revenue: ${opt_avg:,.2f}")
print(f"Mean seats filled: {(o_A + o_B + o_C + o_D + o_E).mean():.0f} of 440")
for name, s, cap in [('A, Avatar IMAX', o_A, cap_A), ('B, Black Adam', o_B, cap_B), ('C, Avatar', o_C, cap_C),
                     ('D, The Fabelmans', o_D, cap_D), ('E, Black Adam', o_E, cap_E)]:
    print(f"  Screen {name}: {s.mean():.0f} of {cap}")
Expected revenue: $5,973.75
Mean seats filled: 395 of 440
  Screen A, Avatar IMAX: 120 of 120
  Screen B, Black Adam: 120 of 120
  Screen C, Avatar: 40 of 80
  Screen D, The Fabelmans: 78 of 80
  Screen E, Black Adam: 37 of 40

3.4 Revenue distribution

Show code
bins = np.linspace(min(s_rev.min(), o_rev.min()), max(s_rev.max(), o_rev.max()), 61)
mid = (bins[:-1] + bins[1:]) / 2
fig = go.Figure()
fig.add_trace(go.Bar(x=mid, y=np.histogram(s_rev, bins)[0], name='Plan of the manager', opacity=0.65, marker_color='#E84855'))
fig.add_trace(go.Bar(x=mid, y=np.histogram(o_rev, bins)[0], name='Our plan', opacity=0.65, marker_color='#3DDA84'))
fig.add_vline(x=6000, line_dash='dash', line_color='#2E86AB', line_width=2,
              annotation_text='Target', annotation_position='top right')
fig.update_layout(barmode='overlay', bargap=0, title='Revenue in one night: the two plans',
                  xaxis_title='Revenue in one night ($)', yaxis_title='Simulated nights', height=440,
                  legend=dict(yanchor='top', y=0.99, xanchor='left', x=0.01))
show(fig)
Figure 4: Revenue in 100,000 simulated nights for each plan

3.5 Result

Show code
results = pd.DataFrame({
    'Plan': ['Plan of the manager', 'Our plan', 'Difference'],
    'Expected revenue': [f'${sarah_avg:,.2f}', f'${opt_avg:,.2f}', f'+${opt_avg - sarah_avg:,.2f}'],
})
results.style.hide(axis='index')
Table 2: Expected revenue in one night (model output)
Plan Expected revenue
Plan of the manager $5,754.86
Our plan $5,973.75
Difference +$218.88

The submitted 2022 run was not seeded. It gave $5,756.26 and $5,974.61. This page uses a fixed seed, so its values differ by about one dollar.

NoteWhy this page gives no probability for the target

The chart shows that our plan passes the $6,000 target on more nights. The size of that change depends on how the demand for the three films is drawn: one draw for each film (as here and in the submission) or one shared draw for the night. The revenue gain is the same in the two models, so it is the result on this page.

3.6 Recommendations

  1. Use the plan that fits seats to demand.
  2. Do not give The Fabelmans only 40 seats. About 80 guests want it.
  3. Keep one Avatar screen in IMAX, because that seat has the highest price.
  4. Use the same simulation when the films change. Only the demand shares need an update.

3.7 Limits

  • The demand shares are fixed values from the brief.
  • A guest who finds no seat does not go to a different film.
  • The model has one night and one showing for each screen.
  • Prices do not change with demand.