Time Series Analysis studies data observed sequentially over time, with an emphasis on how dependence across observations shapes modeling, inference, and prediction. The course develops the probabilistic foundations needed to understand temporal structure, from basic notions of stationarity and correlation to more advanced models for persistence, seasonality, changing variance, and latent dynamics. Along the way, it connects theoretical tools with the practical goal of building models that forecast well and explain the mechanisms behind observed data.

The main themes are second-order structure, linear modeling, spectral methods, and state-space inference. The early chapters establish the language of stationary processes and linear processes, then move to ARMA modeling, estimation, prediction, and model selection. From there, the course broadens to nonstationary phenomena through differencing and ARIMA models, then deepens the theory with Wold decomposition and linear prediction. Later chapters shift between time and frequency domains, introducing spectral analysis and frequency-domain inference, before covering state space models, the Kalman filter, and conditional heteroskedasticity via GARCH.

The chapters are arranged to build progressively from core concepts to integrated applications. After the foundational material, each unit extends the previous one: ARMA models motivate estimation and forecasting; nonstationarity leads naturally to ARIMA; linear prediction theory explains why these methods work; spectral methods provide an alternative representation of dependence; state-space methods unify filtering and latent-variable modeling; and GARCH addresses time-varying volatility. The final chapter brings these strands together through an integrated workflow and case studies, showing how the theory supports end-to-end analysis of real time series problems.

# Introduction

These notes study random quantities observed in time order: daily returns, quarterly inflation, hourly demand, signal measurements, and many other data streams where dependence across time is the central feature. The course develops the probabilistic structure behind such sequences and then turns that structure into statistical models for prediction, filtering, and inference. The organizing theme is that useful time-series models separate persistent dependence, transient shocks, frequency content, and changing conditional variance.

The main object will be a discrete-time stochastic process $(X_t)_{t \in \mathbb Z}$ or $(X_t)_{t \ge 1}$ on a probability space $(\Omega, \mathcal F, \mathbb P)$. We usually observe one finite stretch $X_1,\dots,X_n$, so the theory must explain what can be learned from a single dependent trajectory. This is the first conceptual difference from independent-sample statistics: time order is information, and dependence is not a nuisance to be averaged away.

## What Makes Time Series Different from Independent Data

In independent-sample models, repeating the experiment produces a collection $Y_1,\dots,Y_n$ whose joint law is controlled by a common marginal distribution and independence. For a time series, the marginal distribution of $X_t$ is only a small part of the model: the joint behaviour of $(X_s,X_t)$, $(X_r,X_s,X_t)$, and longer blocks contains information about persistence, seasonality, cycles, volatility clustering, and predictability.

[definition: Discrete Time Series] A discrete time series is a stochastic process $(X_t)_{t \in T}$, where $T$ is either $\mathbb Z$, $\mathbb N$, or a finite subset of consecutive integers, and each $X_t$ is a random variable on a common probability space $(\Omega, \mathcal F, \mathbb P)$. [/definition]

The index $t$ is part of the structure. Reordering $X_1,\dots,X_n$ usually changes the statistical question, because lags such as $X_t$ and $X_{t-1}$ encode temporal dependence.

[example: Independent Noise and Persistent Signal] Let $(Z_t)_{t\in\mathbb Z}$ be i.i.d. with $\mathbb E[Z_t]=0$ and $\operatorname{Var}(Z_t)=\sigma^2$. For $h\ne 0$, the variables $Z_t$ and $Z_{t-h}$ are independent, so \begin{align*} \operatorname{Cov}(Z_t,Z_{t-h}) &=\mathbb E[Z_tZ_{t-h}]-\mathbb E[Z_t]\mathbb E[Z_{t-h}]\\ &=\mathbb E[Z_t]\mathbb E[Z_{t-h}]-0\cdot 0\\ &=0. \end{align*} Thus independent noise has no linear dependence across nonzero lags. Now consider the stationary causal autoregression \begin{align*} X_t=\phi X_{t-1}+Z_t,\qquad |\phi|<1, \end{align*} written as \begin{align*} X_t=\sum_{j=0}^{\infty}\phi^j Z_{t-j}. \end{align*} Since $\sum_{j=0}^{\infty}|\phi|^j<\infty$ and $\mathbb E[Z_{t-j}]=0$, \begin{align*} \mathbb E[X_t] &=\sum_{j=0}^{\infty}\phi^j\mathbb E[Z_{t-j}]\\ &=\sum_{j=0}^{\infty}\phi^j\cdot 0\\ &=0. \end{align*} For $h\ge 0$, \begin{align*} \operatorname{Cov}(X_t,X_{t-h}) &=\mathbb E[X_tX_{t-h}]\\ &=\mathbb E\left[\left(\sum_{j=0}^{\infty}\phi^j Z_{t-j}\right) \left(\sum_{k=0}^{\infty}\phi^k Z_{t-h-k}\right)\right]\\ &=\sum_{j=0}^{\infty}\sum_{k=0}^{\infty}\phi^{j+k} \mathbb E[Z_{t-j}Z_{t-h-k}]. \end{align*} The expectation $\mathbb E[Z_{t-j}Z_{t-h-k}]$ equals $\sigma^2$ when $t-j=t-h-k$, i.e. when $j=h+k$, and equals $0$ otherwise by independence and zero means. Hence \begin{align*} \operatorname{Cov}(X_t,X_{t-h}) &=\sum_{k=0}^{\infty}\phi^{(h+k)+k}\sigma^2\\ &=\sigma^2\phi^h\sum_{k=0}^{\infty}\phi^{2k}\\ &=\sigma^2\phi^h\frac{1}{1-\phi^2}. \end{align*} By symmetry of covariance, \begin{align*} \operatorname{Cov}(X_t,X_{t-h})=\frac{\sigma^2}{1-\phi^2}\phi^{|h|},\qquad h\in\mathbb Z. \end{align*} So both processes have mean zero, but their dependence structures differ: $Z_t$ has zero autocovariance at every nonzero lag, while the autoregression carries memory across lags through the factor $\phi^{|h|}$. [/example]

This example previews a recurring course pattern. We start with a probabilistic description of dependence, derive covariance or spectral consequences, and then use those consequences to build estimators and predictors.

## The Modelling Questions

A time-series model should answer several connected questions. What aspects of the distribution remain stable over time? Which transformations remove trends or changing level? How far into the future is prediction possible? Which frequencies dominate the series? How should uncertainty be updated when new observations arrive?

[definition: Forecasting Problem] Given observations $X_1,\dots,X_n$ and an integer $h\ge1$, the $h$-step forecasting problem is to construct an estimator $\widehat X_{n+h}$ of $X_{n+h}$ using the information generated by the observed data. [/definition]

The information generated by the observed data is often represented by $\mathcal F_n^X=\sigma(X_1,\dots,X_n)$. In the full probabilistic problem, the optimal square-integrable forecast is a conditional expectation. In linear time-series theory, the optimal linear forecast is an orthogonal projection in a Hilbert space.

[quotetheorem:3621]

[citeproof:3621]

This theorem gives the ideal forecast when the joint law is known. The course then asks how close practical models can get when the law must be inferred from one observed path.

[example: Forecasting an Autoregression] Suppose $X_t=\phi X_{t-1}+Z_t$, where $|\phi|<1$, $(Z_t)$ is i.i.d. with $\mathbb E[Z_t]=0$ and finite variance, and $Z_{n+1}$ is independent of $\mathcal F_n^X=\sigma(X_1,\dots,X_n)$. We compute the one-step conditional mean forecast by conditioning the recursion \begin{align*} X_{n+1}=\phi X_n+Z_{n+1}. \end{align*} Since $X_n$ is $\mathcal F_n^X$-measurable and $\phi$ is deterministic, \begin{align*} \mathbb E[X_{n+1}\mid \mathcal F_n^X] &=\mathbb E[\phi X_n+Z_{n+1}\mid \mathcal F_n^X]\\ &=\mathbb E[\phi X_n\mid \mathcal F_n^X]+\mathbb E[Z_{n+1}\mid \mathcal F_n^X]\\ &=\phi X_n+\mathbb E[Z_{n+1}\mid \mathcal F_n^X]. \end{align*} Independence of $Z_{n+1}$ from $\mathcal F_n^X$ gives \begin{align*} \mathbb E[Z_{n+1}\mid \mathcal F_n^X] &=\mathbb E[Z_{n+1}]\\ &=0. \end{align*} Therefore \begin{align*} \mathbb E[X_{n+1}\mid \mathcal F_n^X] &=\phi X_n+0\\ &=\phi X_n. \end{align*} The forecast keeps exactly the predictable autoregressive part $\phi X_n$ and discards the new innovation because its conditional mean, given the observed past, is zero. [/example]