1 · What is in it
| Column | Type | Meaning |
|---|---|---|
trip_id | int | 1–200, in time order. The split is time-based: 1–150 train, 151–200 test. |
zone | string | Pickup area — BKC, Andheri East, Powai, CSMIA T2 and seven others. Not used as a feature; it is there for grouping and plots. |
hour | int 0–23 | Hour of pickup. |
is_peak | 0/1 | Derived: 08–10 or 18–21. |
is_weekend | 0/1 | True coefficient is exactly zero. This is the null feature Lasso should delete first. |
is_rain | 0/1 | Rain probability is raised during peak hours, which is what makes peak and rain correlated. |
is_event_nearby | 0/1 | Stadium, concert or similar within the cell. |
is_airport_pickup | 0/1 | CSMIA pickups carry a structural premium. |
traffic_speed_kmph | float | Mean speed in the cell. Falls in peak and in rain — correlated with both. |
drivers_available_500m | int | Supply. Enters the true model as $50/(S+1)$, i.e. non-linearly. |
open_requests_500m | int | Demand. |
historical_demand | float | Same hour last week — a near-duplicate of open_requests. |
distance_km, time_min | float | Trip length. Drive the base fare, not the surge. |
base_fare_inr | float | $40 + 12\,d + 2\,t$. |
surge_additive_inr | float | The target. Clipped to [0, 150]. |
final_fare_inr | float | base + surge. |
demand_supply_ratio | float | requests / drivers. Used by the policy layer, not by the model. |
is_bad_weather | 0/1 | Derived: is_rain AND traffic_speed < 25. Deliberately correlated with both parents — this is the Task 4 group. |
2 · The data generating process
Synthetic data, so the truth is known and an estimator can be checked against it. Each trip's surge is drawn as
$$ \text{surge} = 40\,\texttt{is\_peak} + 30\,\texttt{is\_rain} + 20\,\texttt{is\_event} + 25\,\texttt{is\_airport} + \frac{50}{S+1} + 2\,(60 - \texttt{traffic}) + \varepsilon, \quad \varepsilon \sim \mathcal{N}(0, 10^2), $$then clipped to $[0, 150]$. Three consequences carry into every task.
Two terms are non-linear
$50/(S+1)$ and the traffic term are not linear in the raw feature, so a linear model cannot be exactly right. The best achievable RMSE is bounded below by more than the noise σ = 10.
One feature is pure noise
is_weekend has a true coefficient of exactly zero, so any
non-zero estimate is fitted noise. That makes it a good probe for whether the penalty is
working.
The target is censored
— trips sit exactly at the ₹150 cap. Their true surge was higher and never recorded, so coefficients on the drivers of extreme surge are biased toward zero.
3 · The correlation matrix
Blue is positive, red negative; hover a cell for the exact value. Most of what Ridge, Lasso and Elastic Net do differently on this dataset traces back to this figure.
traffic_speed correlates with almost everything, since peak and rain cause it and
it in turn drives surge, which makes it the feature most likely to absorb its neighbours'
coefficients. historical_demand and open_requests are near-duplicates, a
good pair for the "Lasso picks one arbitrarily" demonstration. is_weekend correlates
with nothing, as a null feature should.
4 · Distributions
Histogram
Scatter
5 · The raw table
Click a header to sort. Filter to see just the rows you care about.