Ridge and Lasso, same budget
Real fits, computed in your browser on the 200-trip Mumbai dataset. Comparing the two at equal α would be unfair, since a squared penalty and an absolute-value penalty bite at different rates. The slider instead fixes the total coefficient mass each model may spend, so both are always equally shrunk and the only difference left is how they spend it.
Ridge Gaussian prior
Shrinks together, never to exactly zero.
Lasso Laplace prior
Shrinks and deletes. Watch the count.
Eight modules
Each module is one lecture plus one lab. The mathematics is developed on the left, the production system is built on the right, and they meet in the middle.
The linear model & least squares
Design matrices, the normal equations, projection geometry, and what goes wrong when $X^\top X$ is close to singular.
Maximum likelihood
Gaussian noise gives least squares directly from the likelihood. Estimating $\sigma^2$, and what MLE guarantees asymptotically.
Bayes & the MAP estimator
Posterior ∝ likelihood × prior. Taking logs turns a prior into a penalty — the result everything else depends on. Built step by step in Walkthrough 2.
Ridge — the Gaussian prior
Closed form, shrinkage in the SVD basis, effective degrees of freedom, and why correlated features share weight.
Lasso — the Laplace prior
Soft-thresholding derived by hand, the diamond-corner argument, coordinate descent, and the caveats about selection.
Elastic Net
The grouping effect, Zou & Hastie's double-shrinkage correction, and when the extra hyperparameter is worth it.
Beyond the point estimate
The full Gaussian posterior, credible intervals, predictive uncertainty, and empirical Bayes for λ.
Case study: surge pricing
Demand and supply forecasts, additive pricing, the price cap, and which estimator to deploy.
Watch the algorithms run
Step through the normal equations, the MAP construction, ridge shrinkage, Lasso coordinate descent and Bayesian updating, one line of arithmetic at a time.
Where this ships
Recommenders, genomics, credit scoring, ad ranking and sensor fusion, each mapped to the estimator it relies on.
Which estimator, and when
A starting rule of thumb. The rest of the course is about when to depart from it.
| Situation | Estimator | Prior on $w$ | Penalty | What it buys you |
|---|---|---|---|---|
| $n \gg p$, features roughly independent | MLE / OLS | none (flat) | none | Unbiased and efficient, with nothing to tune. |
| Features correlated, all plausibly matter | Ridge | $\mathcal{N}(0,\tau^2)$ | $\lambda\lVert w\rVert_2^2$ | Stability: correlated columns split the weight rather than competing for it. |
| $p \gg n$; you need 20 features out of 500 for a compliance review | Lasso | $\text{Laplace}(0,b)$ | $\lambda\lVert w\rVert_1$ | Exact zeros, giving a model that fits on one page. |
| Correlated groups, and you still need sparsity | Elastic Net | Gaussian × Laplace | $\lambda_1\lVert w\rVert_1+\lambda_2\lVert w\rVert_2^2$ | Drops irrelevant groups, keeps relevant ones intact. |
| You must quote an uncertainty, not just a number | Full posterior | $\mathcal{N}(0,\tau^2)$ | — | "₹50 ± ₹10" rather than "₹50". |
What you will hand in
A working surge engine
By week six: a model that takes (hour, rain, traffic, drivers,
requests) for a 500 m grid cell and returns an additive surge in rupees, with a
price cap, a no-surge rule and an uncertainty band.
Five graded tasks plus a Bayesian bonus. Full brief on the assignments page.
Four estimators you wrote yourself
OLS by normal equations, Ridge in closed form, Lasso by coordinate
descent, Elastic Net by extending it. scikit-learn is for checking answers,
not producing them.
Starter notebooks and full solutions on the notebooks page.
Theory has the derivations. Algorithms runs them one step at a time, drawing each matrix entry, soft-threshold and coordinate update as it happens; read a derivation, then watch it execute. Lab applies the estimators to the full dataset. Data holds the correlation matrix that explains most of the results.