This is a teaching reconstruction. Ride-hailing companies publish the broad shape of their marketplace systems (spatial grids, short-horizon demand and supply forecasts, a matching and pricing layer, and the move from multiplier-based to itemised pricing) but not their live models, features or coefficients. What follows has the right shape and mathematics for the problem, on synthetic Mumbai data. Treat it as a well-posed engineering problem, not a description of any company's code.
1 · The problem, stated properly
Split the city into 500 m grid cells and time into 5-minute buckets. Within a cell-bucket you observe open requests $R$ and available drivers $S$. As $R/S$ climbs, waiting times rise, riders abandon, and drivers are in the wrong place. Price is the fastest lever: raise the fare and some riders defer while nearby drivers are pulled in.
The objective is not "maximise revenue". It is closer to:
$$ \min_{\text{price}} \;\; \Big| \frac{\hat R(\text{price})}{\hat S(\text{price})} - 1 \Big| \quad\text{subject to}\quad \widehat{\text{wait}} < 5\ \text{min}, \;\; \text{surge} \le \text{cap}. $$Both $\hat R$ and $\hat S$ are forecasts, and forecasts are regressions.
2 · The architecture
Three models set the price, in series. The third takes the first two's output as input, which matters when reasoning about error propagation.
3 · The features, and their inconvenient correlations
For each cell-bucket we collect:
x = [ hour, is_peak, is_weekend, is_rain, is_event_nearby, is_airport_pickup,
traffic_speed_kmph, drivers_available_500m, open_requests_500m,
historical_demand_same_hour_last_week ]These are not independent, and not remotely close to it. In our Mumbai data:
| Pair | r | Why |
|---|---|---|
is_peak × traffic_speed | −0.75 | Evening peak is exactly when the roads seize up. |
is_rain × traffic_speed | −0.66 | Rain slows everything, and Mumbai rain slows everything a lot. |
is_peak × drivers_available | −0.60 | Drivers are already occupied when demand peaks. |
is_peak × is_rain | +0.25 | Monsoon downpours cluster in the late afternoon and evening. |
So when the fare rises on a rainy Tuesday at 7 pm, which variable caused it? Statistically the question is close to unanswerable, which is the point of Theory §1.1. OLS answers anyway, confidently, and differently each week.
4 · Which estimator does which job
MLE / OLS The version that fails first
Regressing surge on is_rain and traffic_speed works
until the two move together, which in Mumbai is most of the monsoon. On six training rows their
correlation is $-0.94$, and a single extra trip shifts a coefficient by ₹11 per standard
deviation. Task 1 asks you to reproduce this, and then a pair
where $X^\top X$ is exactly singular.
Ridge Demand and supply forecasting
For $\hat R$ and $\hat S$ every feature carries some signal: rain matters off-peak, an event matters in light traffic. Nothing should be deleted, and the forecast must not lurch between five-minute refits, since the rider's price derives from it. That is the Ridge brief. With $\sigma = 10$ and $\tau = 1$ the assignment's prior gives $\lambda = 100$, checked against cross-validation in Task 2.
Lasso Launching in a new city
A new market has hundreds of candidate features and a few weeks of trips: $p \gg n$. OLS is undefined, and Ridge returns hundreds of small non-zero weights nobody can act on. The operations team needs a list of which signals move price. That is selection, which is what the $\ell_1$ penalty does. Task 3 runs this on 120 features and 60 trips, and asks which real drivers Lasso misses and which decoys it keeps.
Elastic Net The production surge model
The live model needs both properties. It must drop dead features, since a
weekend flag with no signal should not appear in a fare breakdown. And it must keep correlated
groups together: is_rain, traffic_speed and is_bad_weather
describe one situation, and a model that keeps a random one will explain the same fare
differently each week. Task 4 compares all three.
Full posterior What the rider is shown
₹50 and ₹50 ± ₹30 are different products. The second justifies showing a fare range, and justifies suppressing surge when the interval is too wide to defend. The bonus task computes these from the closed-form posterior in Theory §7.
5 · Multiplier or line item?
The older design multiplies: $\text{fare} = \text{base} \times m$, with $m = 1.8\times$ shown to the rider. The newer industry-wide pattern is additive and itemised:
$$ \text{fare} = \underbrace{\text{base} + c_d\,\text{distance} + c_t\,\text{time}}_{\text{deterministic}} \;+\; \underbrace{\text{surge}(x)}_{\text{your regression}} .$$Three consequences, all of them modelling consequences rather than marketing ones:
- The multiplier penalises long trips disproportionately. A $1.8\times$ on a ₹600 airport run adds ₹480; on a ₹120 hop it adds ₹96. Surge prices driver scarcity at that moment, which has little to do with trip length. An additive term prices it once.
- Additive means linear means interpretable. "₹60 for high demand" is one coefficient times one feature, and fits on a receipt. A multiplier's attribution does not.
- It changes the regression target. Under a multiplier you would model
$\log(\text{fare})$; additive surge is modelled directly in rupees. Our dataset's target
surge_additive_inrreflects the newer design.
6 · The cap, and what it does to your data
The assignment imposes two policy rules:
if demand_supply_ratio < 0.8: surge = 0 # never surge into a soft market
surge = min(surge, 150) # hard cap, ₹150The first is uncontroversial. The second has a statistical consequence that is easy to miss: 21 of the 200 trips sit exactly at ₹150. Those rows are censored, since the true willingness-to-price was higher and never observed. Fit an ordinary linear model to censored data and you underestimate the coefficients on whatever drives extreme surge.
Notice the censoring; you are not asked to fix it, as a Tobit model is out of scope. A
submission that says "21 observations are at the cap, so my is_airport and
is_rain coefficients are biased toward zero, in this direction" is doing applied
statistics. One that reports RMSE and stops is not.
The cap is also an ethical instrument. Uncapped scarcity pricing during a flood or an evacuation turns a crisis into a willingness-to-pay auction. Caps, and the regulatory disabling of surge during declared emergencies, exist because the profit-maximising price and the defensible price differ. The model computes the first; policy decides the second. Keeping the two layers separate in code is not just tidiness.
7 · Worked example: BKC, 7 pm, raining
The scenario from Task 5. Change the inputs and the model — fitted live in your browser on the training split — re-prices the trip.
Where the surge comes from
Per-feature contribution $w_j (x_j - \bar x_j)/s_j$: the receipt breakdown the additive design makes possible.
8 · So what would you actually deploy?
This is Deliverable 3. There is no single right answer, but there are wrong ones. A defensible argument names the constraint first and the estimator second.
The case for Elastic Net
Drops dead features so the fare breakdown stays short, and keeps the weather group together so the explanation is stable across refits. Costs one extra hyperparameter.
The case for plain Ridge
One hyperparameter, a closed form, no convergence to monitor, and no feature silently vanishing after an overnight retrain. If the requirement is predictability, the accuracy difference here is within noise anyway.
Check that claim before arguing it: the lab's comparison table shows how close the held-out RMSEs are. If two models are within a rupee of each other, the tie-breaker is operational rather than statistical, and saying so explicitly earns the marks.