Part 4

Turn history into measurable memory.

A lag aligns today's target with values that were already known. Autocorrelation summarizes how strongly the series resembles delayed versions of itself.

Feature question

To predict Thursday at Wednesday night, which are valid: Thursday demand, Wednesday demand, last Thursday demand, or the average ending on Thursday?

Supervised framing

\[x_t=\begin{bmatrix}y_{t-1}&y_{t-2}&\cdots&y_{t-p}\end{bmatrix},\qquad \text{target}=y_t\]

Time	Lag 1	Lag 7	Previous 7-day mean	Target
Day \(t\)	\(y_{t-1}\)	\(y_{t-7}\)	\(\frac17\sum_{j=1}^{7}y_{t-j}\)	\(y_t\)

Safe rolling feature: calculate the window from shifted data. The row for \(t\) may summarize \(t-1\) through \(t-7\), never \(t\) itself.

Plot prediction

For weekly demand, at which lags should the ACF show repeated strength?

Autocorrelation function

For a weakly stationary process, the population autocorrelation at lag \(k\) is the ordinary correlation between observations \(k\) time steps apart:

\[\rho_k=\operatorname{Corr}(Y_t,Y_{t-k})=\frac{\operatorname{Cov}(Y_t,Y_{t-k})}{\sqrt{\operatorname{Var}(Y_t)\operatorname{Var}(Y_{t-k})}}\]

Because stationarity gives both variables the same variance, the denominator is simply \(\operatorname{Var}(Y_t)\). From a finite series we estimate \(\rho_k\) with the sample ACF:

\[r_k=\frac{\sum_{t=k+1}^{T}(y_t-\bar y)(y_{t-k}-\bar y)}{\sum_{t=1}^{T}(y_t-\bar y)^2}\]

At lag \(k\), line up the series with a copy shifted by \(k\), keep the overlapping pairs, and calculate their linear similarity. Values near \(+1\) mean high values tend to follow high values; values near \(-1\) mean high values tend to follow low values; values near zero mean little linear association at that lag.

Small numerical example

Let \(y=[10,20,10,20,10,20]\), so \(\bar y=15\). At lag 2, every overlapping value matches the value two steps earlier:

\[r_2=\frac{4(5^2)}{6(5^2)}=\frac{100}{150}\approx0.67\]

The estimate is strongly positive because the high-low pattern repeats every two observations.

Lag-7 meaning

For daily bike rentals, pair every day with the same weekday one week earlier. A large positive \(r_7\) supports weekly repetition.

Red bars mark weekly multiples 7, 14, and 21. Trend can also keep many correlations positive, so an ACF must be read alongside the original series and differenced versions.

Direct-effect question

If lag 2 correlates with today only because lag 2 influences lag 1, should both lags receive equal credit?

Partial autocorrelation function

PACF at lag \(k\) removes the linear information carried through the intermediate lags \(1,\ldots,k-1\). Regress \(Y_t\) on those intermediate lags and keep residual \(e_t\). Do the same for \(Y_{t-k}\) and keep residual \(e_{t-k}\). PACF is the correlation between what remains:

\[\operatorname{PACF}(k)=\operatorname{Corr}(e_t,e_{t-k})\]

Equivalently, in an autoregression containing lags 1 through \(k\), PACF at lag \(k\) is the coefficient on the final lag after the shorter lags have already received credit:

\[Y_t=c+\phi_{k1}Y_{t-1}+\cdots+\phi_{kk}Y_{t-k}+\varepsilon_t,\qquad \operatorname{PACF}(k)=\phi_{kk}\]

A lag-2 numerical example

Suppose \(\rho_1=0.8\) and \(\rho_2=0.64\). The ACF says lag 2 is strongly related to today. But for lag 2 the partial correlation is:

\[\operatorname{PACF}(2)=\frac{\rho_2-\rho_1^2}{1-\rho_1^2}=\frac{0.64-0.8^2}{1-0.8^2}=0\]

Interpretation: all of the lag-2 association can be explained by the chain \(Y_{t-2}\rightarrow Y_{t-1}\rightarrow Y_t\). Lag 2 has no additional direct linear information once lag 1 is known. If \(\rho_2\) were 0.70 instead, PACF(2) would be about 0.17, revealing a small remaining direct relationship.

ACF at lag \(k\)

Total linear correlation between \(Y_t\) and \(Y_{t-k}\), including indirect paths through intermediate lags.

PACF at lag \(k\)

Additional linear association after controlling for lags \(1,\ldots,k-1\).

Use as clues, not commandments. Classical ACF/PACF cutoff rules are cleanest for ideal stationary AR or MA processes. Real data, seasonality and finite samples blur them; validate candidate orders out of sample.

How to read the ACF figure

Lag 0 is always 1. The series is perfectly correlated with itself, so lag 0 is a reference rather than evidence of useful memory.
Weekly peaks matter. Positive bars at 7, 14 and 21 suggest that matching weekdays resemble one another.
The violet band is an approximation. Under a white-noise assumption, \(\pm1.96/\sqrt{T}\) is an approximate 95% range. Bars outside it suggest nonzero autocorrelation, but multiple-lag testing and nonstationarity complicate that interpretation.
Trend can create a slow decay. High correlation at many neighboring lags may reflect a changing level rather than a true high-order AR process.
Use domain lags too. In hourly bike data, inspect lag 1, lag 24 and lag 168 even if a single plot is noisy.

Feature	Hourly bike-demand meaning	Availability
\(y_{t-1}\)	Previous hour	Known for one-step forecasting
\(y_{t-24}\)	Same hour yesterday	Known
\(y_{t-168}\)	Same hour and weekday last week	Known
Rolling 24-hour mean	Recent daily level	Shift before rolling so it ends at \(t-1\)
Rolling 168-hour standard deviation	Recent weekly volatility	Also calculated only from past rows