Time Series Bootstrapping | Dr. Pedram Jahangiry

Each tab resamples the same monthly airline passenger series with a different bootstrap method. Start with the i.i.d. foil to see what goes wrong, then drag the block-size (or mean block length) and sample-count sliders to regenerate the paths, see exactly which stretches of history got reused, and compare block resampling with fitting an AR(1) model to the residuals. Below the tabs, the payoff: what each method does to the uncertainty of a statistic.

⚠ Why Not Standard (i.i.d.) Bootstrap?

Standard bootstrap draws observations independently with replacement — this destroys the temporal dependence structure that defines time series data. Autocorrelations, trends, and seasonal patterns would be completely lost. Time series bootstrapping must preserve the local dependence by resampling in blocks or through model-based approaches. The first tab lets you watch it happen.

How every tab builds a bootstrap path

1. Split log(passengers) = trend + seasonal + residual

→

2. Resample the residuals only Blocks (MBB, CBB, SB) or an AR(1) model shuffle the small leftover wiggles

→

3. Add the rest back Each month keeps its own trend and seasonal value, plus a borrowed wiggle

Why not cut blocks from the raw counts? A summer 1960 block (about 600 passengers) would land in 1949 (about 120). The residuals have no trend, so their blocks can go anywhere. The catch: trend and seasonality are held fixed, so the bootstrap only captures uncertainty in the wiggles, not in how the trend and seasonal pattern were estimated. That is why the paths stay so close to the original.

i.i.d. Bootstrap (the foil)

The classic bootstrap (Efron, 1979) draws T observations one at a time, independently, with replacement. It works beautifully for independent data. For a time series it shuffles away the order, so everything that depends on order (trend, seasonality, autocorrelation) disappears.

Draw T indices uniformly from 1, ..., T with replacement
Raw series mode: resample the passenger counts directly
Residuals mode: resample the decomposition residuals, then add trend and seasonality back (the same pipeline the block tabs use)

Resample:

Bootstrap samples: 5

Key Insight: Two Ways to Fail

In raw mode the damage is obvious: trend and seasonality vanish and every path is noise around the average. Residuals mode is sneakier. Adding trend and seasonality back makes the paths look reasonable, but the residuals have lost their memory: their lag 1 autocorrelation falls to about zero. The paths look right and are quietly wrong, which is exactly what the bootstrap distribution panel below the tabs exposes.

Moving Block Bootstrap (MBB)

The MBB (Kunsch, 1989; Liu & Singh, 1992) samples overlapping blocks of fixed size m from the original series. Each block preserves the local dependence structure within it. Blocks are drawn uniformly from the T − m + 1 possible starting positions, then concatenated and trimmed to length T.

Block library: B_i = [y_i, y_i+1, ..., y_i+m−1] for i = 1, ..., T−m+1
Draw k = ⌈T/m⌉ blocks uniformly with replacement
Concatenate and trim to length T
For non-stationary data (like this series): decompose into Trend + Seasonal + Residual, cut the blocks from the residuals only, then add trend and seasonality back. The first chart below shows the shuffle; the second shows the result.

Block size (m): 24

Bootstrap samples: 5

Tip: twice the seasonal period (m = 24 for monthly data, marked on the slider) is a common default, used by Bergmeir, Hyndman & Benítez (2016). Shorter blocks break the autocorrelation more often; longer blocks give less variety.

Key Insight — Edge Bias

Observations near the boundaries (y₁ and y_T) appear in fewer possible blocks than observations in the middle. For example, y₁ only appears in B₁, while a middle observation y_t appears in up to m different blocks. This creates an edge bias where boundary observations are undersampled — a limitation addressed by CBB.

Circular Block Bootstrap (CBB)

The CBB (Politis & Romano, 1992) wraps the series into a circle so that y_T+1 = y₁, y_T+2 = y₂, etc. This creates exactly T possible blocks of any size m (vs. T−m+1 for MBB), ensuring all observations are sampled with equal probability.

Circular extension: y_t^circ = y_{((t−1) mod T) + 1}
Block library: T blocks (all starting positions valid)
Eliminates edge bias: every observation appears in exactly m blocks

Block size (m): 24

Bootstrap samples: 5

Tip: m = 24 (twice the seasonal period, marked on the slider) is a common default for monthly data.

Key Insight — Circular Wrap

By connecting y_T back to y₁, the CBB ensures that observations near the series boundaries have equal representation. This eliminates the edge bias of MBB. The trade-off is that the wrap-around junction is artificial — the connection from the end back to the start may not reflect genuine temporal dynamics, especially for non-seasonal data.

Stationary Bootstrap (SB)

The SB (Politis & Romano, 1994) uses random block lengths drawn from a Geometric(p) distribution with mean block length 1/p. At each step, with probability p a new random block begins, and with probability 1−p the current block continues (with circular wrapping).

Block length: L ~ Geometric(p), so E[L] = 1/p
Conditional on the data, the resampled series is stationary, a property the fixed-block methods lack
No edge bias: all observations equally likely at any position
The trade-off: random lengths make it less sensitive to the exact mean block length, but it is usually less efficient (higher MSE) than MBB/CBB at the same average block length (Lahiri, 1999)

Mean block length (1/p): 24

Bootstrap samples: 5

Tip: start the mean block length near 24 (marked on the slider), then see how much the paths change as you move it.

Key Insight — Stationarity Preservation

Among the classic block methods, the SB is the one whose resampled series is stationary (conditional on the data) when the original is stationary. The memoryless Geometric block lengths mean block joins can land anywhere, so no position in the resample is special. MBB and CBB put their joins at fixed positions (every m steps), so their resamples are not stationary even when the data are. The price is efficiency: the extra randomness in block lengths adds variance, so for the same average block length the SB is usually a bit less accurate than MBB or CBB.

Model-based Bootstrap — Decomposition + AR(1)

This tab uses the same classical decomposition as the block methods (Trend + Seasonal + Residual on log data), but instead of resampling residuals in blocks, it fits an AR(1) model to carry the autocorrelation and then generates brand new residual paths from that model. The only question is where the new shocks come from, and the toggle gives you both answers:

Semiparametric (residual bootstrap): resample the fitted innovations ε̂_t with replacement. The AR(1) is assumed, the shock distribution is not.
Fully parametric: draw shocks from N(0, σ̂²). Both the AR(1) and the Normal shape are assumed.

Step 1: Decompose log(Y) = Trend + Seasonal + Residual (same as block methods)
Step 2: Fit AR(1) to residuals: r_t = φ·r_t−1 + ε_t
Step 3: Extract & center innovations ε̂_t, and estimate σ̂
Step 4: Draw shocks ε*_t (resampled ε̂_t or N(0, σ̂²)), then build r* with the AR(1) recursion
Step 5: Reconstruct: Y* = exp(Trend + Seasonal + r*)

Shocks:

Bootstrap samples: 5

Key Insight — Two Kinds of Shock, One Model

Both flavors trust the AR(1) to carry the memory, so neither has block joins, and both can keep running past the last month. They differ only in the shocks. The semiparametric version reuses the fitted innovations, so their real shape (skew, fat tails, the odd outlier) carries over. The fully parametric version draws clean Normal shocks: exactly right if the errors really are Normal, blind to anything that shape misses. Compare the histogram with the Normal curve above to judge how much that matters here. If the AR(1) itself is wrong, both inherit the mistake; the lag 12 test in the panel below shows a case where it is. Block methods (MBB, CBB, SB) skip the model, which makes them more robust but less efficient.

What Is It For? The Bootstrap Distribution of a Statistic

One resampled path is just a picture. The real job of a bootstrap is to redraw the series hundreds of times, recompute a statistic on every copy, and read off its spread: a standard error or a confidence interval. Here the statistic is computed on the residuals of every resample, for every method at once. A good time series bootstrap should give copies whose autocorrelation sits around the original (the dashed line). The block size below applies to MBB, CBB and the SB mean block length.

Statistic:

Block size (m): 24

Replications:

Key Insight — What to Look For

Lag 1: the i.i.d. bootstrap collapses to about zero, because it erased the memory. The block methods sit a little below the original: only pairs that land inside the same block keep their link, roughly (m − k)/m of the lag k pairs, so every join drags the autocorrelation down. Slide m from 3 up to 24 and watch them climb. The AR(1) methods land on target, but only because they were fit to lag 1. Lag 12: the fairer test. The residuals still carry some yearly pattern (about 0.35). Blocks of 24 months keep about half of it, longer blocks keep more, and an AR(1) keeps essentially none, since it implies φ¹² ≈ 0. That gap is model misspecification made visible. Std. deviation: every method gets the spread of the residuals roughly right, which is why the i.i.d. bootstrap can look fine on simple statistics and still fail on anything that depends on time order.

Comparison of Methods

Property	i.i.d.	MBB	CBB	Stationary Bootstrap	Model-based AR(1)
Block length	1	Fixed m	Fixed m	Random ~ Geom(p)	N/A (model-based)
Possible blocks	T (single points)	T − m + 1	T	T starts, random lengths	N/A
Keeps autocorrelation	No	Within blocks	Within blocks	Within blocks	Through the fitted AR(1)
Edge bias	No	Yes	No	No	No
Resampled residuals stationary*	Yes (but no memory)	No	No	Yes	Approximately, if \|φ\| < 1
Equal obs. probability	Yes	No	Yes	Yes	N/A (draws shocks)
Model-free	Yes	Yes	Yes	Yes	No
Shock distribution	Empirical	Empirical	Empirical	Empirical	Empirical (semiparametric) or Normal (fully parametric)
Block boundary artifacts	Every point is a jump	Yes, every m steps	Yes, every m steps	Yes, at random positions	No
Can extend residuals past T**	No	No	No	No	Yes (AR recursion)

* This row is about the resampled residuals, conditional on the data. Once trend and seasonality are added back, no Y* path is stationary, for any method.
** A forecast path also needs the trend extended past T, which needs its own model. Block bootstraps reach forecasting a different way, through bagging: resample the series many times, refit a forecaster on each copy, and average the forecasts (Bergmeir, Hyndman & Benítez, 2016).

Where This Goes Next: Bagging

The Bagging the Airline Passengers tool trains many trees on bootstrap resamples and averages their forecasts. It resamples rows of lag features, which works because each row carries its own recent history. When you need whole new series instead, for example to bag exponential smoothing models, you resample with the block methods on this page.