The Bias-Variance Tradeoff

Module 5 — Machine Learning for Time Series Forecasting
Created by Dr. Pedram Jahangiry | Enhanced with Claude

This tool refits a polynomial to a fresh random sample every time you click "Draw Sample," so you can watch low-degree fits stay stable but miss the curve while high-degree fits chase the noise instead. Bias, variance, and the resulting mean squared error update from the accumulated draws, letting you see the tradeoff build up sample by sample rather than take it on faith.

Understanding Bias and Variance Through Repeated Sampling

The true relationship between x and y is a quadratic function (degree 2) plus random noise. You don't know this — you only observe noisy samples of 20 points. Each time you click "Draw Sample", a fresh sample of 20 points is generated and a polynomial of your chosen degree is fit to it. By drawing many samples, you can see how the fitted model behaves across repeated experiments.

Bias = how far the average prediction is from the truth (systematic error). Variance = how much predictions scatter across different samples (instability). A simple model (degree 0 or 1) is stable but consistently wrong — high bias. A complex model (degree 3 or 4) has extra parameters and changes more between samples — high variance.

Bias² — Systematic Error
Bias[f̂(x)] = E[f̂(x)] − f(x)
How far is the average prediction
across all samples from the truth?
Variance — Instability
Var[f̂(x)] = E[(f̂(x) − E[f̂(x)])²]
How much do individual predictions
vary around their average?
E[Test Error] = Bias² + Variance + σ²   (irreducible noise)
2
Draws: 0
Good Balance
Expected Test Error = Bias² + Variance + Irreducible Noise =
Bias²
Draw samples to estimate
Variance
Draw samples to estimate
Irreducible Noise (σ²)
9.00
Cannot be reduced
Total Expected Error
Bias² + Variance + σ²
Fitted Models on Repeated Samples
Bias-Variance Tradeoff Curve