Part 7

Explain the next value using memory and past surprises.

ARIMA combines three operations: stabilize the series by differencing, regress the stable series on its own lags, and model short-lived dependence in forecast errors.

Memory question

If today's demand remains informative after yesterday's demand is known, which component should represent that direct lag relationship?

AR: previous values

\[z_t=c+\phi_1z_{t-1}+\phi_2z_{t-2}+\cdots+\phi_pz_{t-p}+\varepsilon_t\]

An AR(\(p\)) model uses \(p\) lags of a stationary series \(z_t\). The innovation \(\varepsilon_t\) is new information not predictable from those lags.

Example: with \(\phi_1=0.7\), a positive deviation tends to persist, but its expected effect shrinks geometrically when \(|\phi_1|<1\).

Surprise question

Yesterday's unexpected station closure created a large forecast error. Could that surprise affect today's forecast even after yesterday's demand is observed?

MA: previous innovations

\[z_t=\mu+\varepsilon_t+\theta_1\varepsilon_{t-1}+\cdots+\theta_q\varepsilon_{t-q}\]

An MA(\(q\)) model uses the last \(q\) innovations. These are fitted one-step forecast errors, not a moving average of observations.

Terminology trap: “MA” in ARIMA means a linear combination of past innovations. It is different from a rolling mean such as \((y_t+y_{t-1}+y_{t-2})/3\).

Combine them: ARIMA(\(p,d,q\))

Difference the original series \(d\) times and call the result \(z_t=(1-B)^d y_t\). Then model \(z_t\) with AR and MA terms:

\[\underbrace{\phi(B)}_{\text{AR}}\underbrace{(1-B)^d y_t}_{\text{differenced series}}=c+\underbrace{\theta(B)\varepsilon_t}_{\text{current and past innovations}}\]

\[\phi(B)=1-\phi_1B-\cdots-\phi_pB^p,\qquad \theta(B)=1+\theta_1B+\cdots+\theta_qB^q\]

\(p\)

Number of nonseasonal AR lags.

\(d\)

Number of nonseasonal differences.

\(q\)

Number of nonseasonal MA terms.

Software may use an equivalent opposite sign convention for MA coefficients. Interpretation and forecasts remain consistent within that convention.

One-step calculation

A stationary demand deviation is currently \(z_t=20\). With \(c=2\) and \(\phi_1=0.6\), what does an AR(1) model forecast before any future innovation occurs?

A numerical AR(1) forecast

\[\widehat z_{t+1\mid t}=c+\phi_1z_t=2+0.6(20)=14\]

The current deviation of 20 does not persist completely. The coefficient carries 60% of it forward and the intercept contributes 2. As forecasts move farther ahead, repeated multiplication by \(\phi_1\) pulls the deviation toward the process mean when \(|\phi_1|<1\).

\[\widehat z_{t+2\mid t}=c+\phi_1\widehat z_{t+1\mid t}=2+0.6(14)=10.4\]

AR forecasts propagate predicted values. The unknown future innovation has conditional expectation zero, so it does not enter the point forecast.

What ACF and PACF suggest in ideal cases

Stationary process	Typical ACF pattern	Typical PACF pattern
AR(\(p\))	Gradual exponential or damped decay	Approximately cuts off after lag \(p\)
MA(\(q\))	Approximately cuts off after lag \(q\)	Gradual decay
ARMA	Gradual decay	Gradual decay
Seasonal dependence	Peaks at \(m,2m,\ldots\)	Seasonal spikes may remain after shorter lags are controlled

These are identification heuristics. Finite samples, trend, multiple seasonalities and mixed ARMA structure blur textbook cutoffs. Confirm choices through rolling-origin performance and residual diagnostics.

Weekly question

Where does a seven-day relationship enter an ARIMA model without adding seven ordinary lags one by one?

SARIMA adds seasonal memory

\[\operatorname{SARIMA}(p,d,q)\times(P,D,Q)_m\]

\[\phi(B)\Phi(B^m)(1-B)^d(1-B^m)^D y_t=c+\theta(B)\Theta(B^m)\varepsilon_t\]

\(P,D,Q\) are seasonal AR, differencing, and MA orders. The seasonal polynomials use \(B^m\), so they connect matching locations across seasons.

Order selection is evidence, not ritual

Plot the raw series and choose plausible transformations or differences.
Inspect ACF/PACF for candidate lag structure.
Fit a small set of sensible orders; avoid a huge blind search.
Compare rolling-origin forecasts, not only in-sample AIC.
Check that residuals retain no useful autocorrelation.

White-noise goal: residuals need not be normally distributed to be unpredictable, but their mean should be near zero and their ACF should show no systematic remaining structure.