Estimation of the exact likelihood of an ARMA model

X12/X13 implementation

We suppose that $y_t, \quad 1 \le t \le n$ follows an ARMA model.

The X12 implementation computes the exact likelihood in two main steps

Overview

We consider the transformation

$$z_t = \begin{pmatrix} z_{1t} \\ z_{2t} \end{pmatrix} = \begin{cases} y_t, & 1 \le t \le p \\ \Phi\left(B\right) y_t, & p \lt t \le n\end{cases}$$

It is obvious that $p\left(y_t\right) = p\left(z_t\right)$

We will estimate $p\left(z_t\right) = p\left(z_{2t}\right) p\left(z_{1t}\right | z_{2t} )$

Step 1: likelihood of a pure moving average process

$z_{2t}$ is a pure moving average process. Its exact likelihood is estimated as follows (see [1] for details). We list below the different steps of the algorithm.

• Compute the conditional least squares residuals by the recursion: $a_{0t} = \begin{cases} 0,& -q \lt t \leq 0 \\ z_t-\theta_1 a_{0t-1}- \cdots --\theta_q a_{0t-q},& 0 \lt t \leq n \end{cases}$

• Compute the Pi-Weights of the model. They defines the (n+q x q) matrix

$$G = \begin{pmatrix} 1 & 0 & \cdots & 0 \\ \pi_1 & 1 & \cdots & 0 \\ \pi_2 & \pi_1 & \cdots & \vdots\\ \vdots & \vdots & \vdots & \vdots \\ \pi_{n+q-1} & \pi_{n+q-2} & \cdots & \pi_{n} \end{pmatrix}$$

• Compute by recursion $G'a \quad and \quad G'G$

• Compute by Cholesky decomposition $G'G \hat z_* = G'a \quad and \quad |G'G|$

• Obtain by recursion the exact likelihood residuals $\Theta\left(B\right)\begin{pmatrix}\hat a_* \\ \hat a_t \end{pmatrix} = \begin{pmatrix}z_* \\ z_t \end{pmatrix}$

The processing defines the linear transformation $T z_t = \begin{pmatrix}\hat a_* \\ \hat a_t \end{pmatrix}$ and the searched determinant.

Step 2: conditional distribution of the initial observations

$p\left(z_{1t}\right | z_{2t} )$ is easily obtained by considering the join distribution of $\left( z_{1t}, z_{2t} \right) \sim N \left( 0, \begin{pmatrix} \Sigma_{11} && \Sigma{12} \\ \Sigma_{21} && \Sigma{22} \end{pmatrix}\right) \ $

It is distributed as $N\left( \Sigma_{12} \Sigma_{22}^{-1}v, \Sigma_{11}-\Sigma_{12} \Sigma_{22}^{-1} \Sigma_{21} \right)$

$$= N\left( \Sigma_{12} T'Tv, \Sigma_{11}-\Sigma_{12} T'T\Sigma_{21} \right)$$ $$= N\left( U'Tv, \Sigma_{11}-U'U \right)$$

where

• v is obtained by applying the auto-regressive polynomial on the observations
• T is the linear transformation defined in step 1
• $$U = T \Sigma_{21}$$

Bibliography

[1] Otto M. C., Bell W.R., Burman J.P. (1987), “An Iterative GLS Approach to Maximum Likelihood Estimation of Regression Models with Arima Errors”, Bureau of The Census, SRD Research Report CENSUS/SRD/RR_87/34.