Polynomials of the lag operator are commonly used to simplify ARMA models notation. These polynomials are called lag polynomials.
$a_{(p)} (B)$ is a lag polynomials: $$ \begin{align} a_{(p)} (B) &= 1 + \sum_{i=1}^p a_i B^i \\ &= 1 + a_1 B + a_2 B^2 + \dots + a_p B^p \end{align} $$
Using this model, the observation $y_t$ of a time series at time $t$ is defined as a linear combination of its $p$ past observations $y_{t-1}, \dots, y_{t-p}$
$$ \begin{align} AR(\color{\red}{p}): \quad\quad\quad \color{\red}{\underbrace{\varphi_{(p)} (B) y_t}_{\text{AR}}} &= c + \color{\green}{\varepsilon_t} \\ \color{\red}{ \left( 1 + \sum_{i=1}^p \varphi_i B^i \right) y_t } &= c + \color{\green}{\varepsilon_t} \\ \color{\red}{ y_t } + \color{\red}{\sum_{i=1}^p \varphi_i B^i y_t } &= c + \color{\green}{\varepsilon_t} \\ \color{\red}{ y_t } + \color{\red}{ \sum_{i=1}^p \varphi_i y_{t-i} } &= c + \color{\green}{\varepsilon_t} \\ \color{\red}{y_t} &= c + \color{\green}{\varepsilon_t} + \color{\red}{\sum_{i=1}^p \phi_i y_{t-i}} \end{align} $$with $\varphi_i = -\phi_i$.
$c$ is a constant used to ensure the series is centered to 0 ($c$ is the average of the series).
$\color{\green}{\varepsilon_t}$ is the noise component of $y_t$.
with $\varphi_i = -\phi_i$.
Combinaison linéaire des $q$ erreurs passées Using this model, the observation $y_t$ of a time series at time $t$ is defined as the output of a linear filter with transfer function $\theta_{(q)}(B)$ when the input is white noise $\varepsilon_t$.
$$ \begin{align} MA(\color{\green}{q}): \quad\quad\quad \color{\red}{y_t} &= c + \color{\green}{\underbrace{\theta_{(q)} (B) \varepsilon_t}_{MA}} \\ \color{\red}{y_t} &= c + \color{\green}{\left( 1 + \sum_{j=1}^q \theta_j B^j \right) ~ \varepsilon_t} \\ \color{\red}{y_t} &= c + \color{\green}{\varepsilon_t} + \color{\green}{\sum_{j=1}^q \theta_j B^j \varepsilon_t} \\ \color{\red}{y_t} &= c + \color{\green}{\varepsilon_t} + \color{\green}{\sum_{j=1}^q \theta_j ~ \varepsilon_{t-j}} \end{align} $$An ARMA(p,q) process of order $p$ and $q$ is a combination of an AR(p) and an MA(q) process.
$$ \begin{align} ARMA(\color{\red}{p}, \color{\green}{q}): \quad\quad\quad \color{\red}{\underbrace{\varphi_{(p)} (B) y_t}_{\text{AR}}} &= c + \color{\green}{\underbrace{\theta_{(q)} (B) \varepsilon_t}_{MA}} \\ \color{\red}{ \left( 1 + \sum_{i=1}^p \varphi_i B^i \right) y_t } &= c + \color{\green}{\left( 1 + \sum_{j=1}^q \theta_j B^j \right) ~ \varepsilon_t} \\ \color{\red}{ y_t } + \color{\red}{\sum_{i=1}^p \varphi_i B^i y_t } &= c + \color{\green}{\varepsilon_t} + \color{\green}{\sum_{j=1}^q \theta_j B^j \varepsilon_t} \\ \color{\red}{ y_t } + \color{\red}{ \sum_{i=1}^p \varphi_i y_{t-i} } &= c + \color{\green}{\varepsilon_t} + \color{\green}{\sum_{j=1}^q \theta_j ~ \varepsilon_{t-j}} \\ \color{\red}{y_t} &= c + \color{\red}{\underbrace{\sum_{i=1}^p \phi_i y_{t-i}}_{\text{AR model}}} + \color{\green}{\varepsilon_t} + \color{\green}{\underbrace{\sum_{j=1}^q \theta_j \varepsilon_{t-j}}_{\text{MA model}}} \end{align} $$with $\varphi_i = -\phi_i$
with $\varphi_i = -\phi_i$
An ARIMA process is an integrated ARMA process. It's used on non-stationary time series. An initial transformation step (the "integrated" part of the model) is applied to eliminate the non-stationarity. This transformation is usually operated by the difference operator described above.
An $ARIMA(p,d,q)$ process produce series that can be modeled as a stationary $ARMA(p,q)$ process after being differenced $d$ times.
$$ \begin{align} ARIMA(\color{\red}{p}, \color{\orange}{d}, \color{\green}{q}): \quad\quad\quad \color{\red}{\underbrace{\varphi_{(p)} (B) \color{\orange}{\nabla^d} y_t}_{\text{AR}}} &= c + \color{\green}{\underbrace{\theta_{(q)} (B) \varepsilon_t}_{MA}} \\ \color{\red}{ \left( 1 + \sum_{i=1}^p \varphi_i B^i \right)} \color{\orange}{\nabla^d} \color{\red}{ y_t } &= c + \color{\green}{\left( 1 + \sum_{j=1}^q \theta_j B^j \right) ~ \varepsilon_t} \\ \color{\orange}{\nabla^d} \color{\red}{ y_t } + \color{\red}{\sum_{i=1}^p \varphi_i B^i } \color{\orange}{\nabla^d} \color{\red}{ y_t } &= c + \color{\green}{\varepsilon_t} + \color{\green}{\sum_{j=1}^q \theta_j B^j \varepsilon_t} \\ \color{\orange}{\nabla^d} \color{\red}{ y_t } + \color{\red}{ \sum_{i=1}^p \varphi_i } \color{\orange}{\nabla^d} \color{\red}{ y_{t-i} } &= c + \color{\green}{\varepsilon_t} + \color{\green}{\sum_{j=1}^q \theta_j ~ \varepsilon_{t-j}} \\ \color{\orange}{\nabla^d} \color{\red}{y_t} &= c + \underbrace{ \color{\red}{\sum_{i=1}^p \phi_i} \color{\orange}{\nabla^d} \color{\red}{y_{t-i}} }_{\text{ARI model}} + \color{\green}{\varepsilon_t} + \underbrace{\color{\green}{\sum_{j=1}^q \theta_j \varepsilon_{t-j}}}_{\text{MA model}} \end{align} $$with
$$\nabla = (1-B)$$$$\nabla^2 = (1-B)^2$$$$\dots$$$$\nabla^d = (1-B)^d$$If $d=0$: $$ \begin{align} \nabla^0 y_t &:= (1-B)^0 y_t \\ &= y_t \end{align} $$
If $d=1$: $$ \begin{align} \nabla y_t &:= (1-B) y_t \\ &= y_t - B y_t \\ &= y_t - y_{t-1} \end{align} $$
If $d=2$: $$ \begin{align} \nabla^2 y_t &:= (1-B)^2 y_t \\ &= (1-B)(1-B) ~ y_t \\ &= (1 - 2B + B^2) y_t \\ &= y_t - 2B y_t + B^2 y_t \\ &= y_t - 2 y_{t-1} + y_{t-2} \end{align} $$
The above approach generalises to the i-th difference operator $\nabla^i y_t = (1-B)^i ~ y_t$
$x_{1,~t}, x_{2,~t}, \dots, x_{r,~t}$ is the value of the $r$ exogenous variables at time $t$, with $\beta_1, \beta_2, \dots, \beta_r$ the $r$ corresponding coefficients.
$$ ARMAX(\color{\red}{p}, \color{\green}{q}): \quad \color{\red}{y_t} = c + \color{\red}{\underbrace{\sum_{i=1}^p \varphi_i y_{t-i}}_{\text{AR model}}} + \color{\green}{\varepsilon_t} + \color{\green}{\underbrace{\sum_{j=1}^q \theta_j \varepsilon_{t-j}}_{\text{MA model}}} + \color{\purple}{\underbrace{\sum_{k=1}^r \beta_k x_{k,~t}}_{\text{Exogenous}}} $$with $\varPhi = -\Phi$ and :
Remark: most of the time, $s = s'$. In this case, the process is simply noted $ARMA(\color{\red}{P}, \color{\green}{Q})_{s}$
with $\varPhi_i = -\Phi_i$
These processes are rarely used $ARMA(p,q)+ARMA(P,Q)_{s, s'}$.
$$ \begin{align} ARMA(p,q)+ARMA(P,Q)_{s, s'}: \quad \left( \underbrace{\varphi_p \left(B\right)}_{AR} + \underbrace{\Phi_P \left(B^s\right)}_{SAR} - 1 \right) ~ y_t &= c + \left( \underbrace{\theta_q \left(B\right)}_{MA} + \underbrace{\Theta_Q \left(B^{s'}\right)}_{SMA} - 1 \right) ~ \varepsilon_t \\ \underbrace{\varphi_p (B) ~ y_t}_{AR} + \underbrace{\Phi_P \left(B^s\right) ~ y_t}_{SAR} - y_t &= c + \underbrace{\theta_q \left(B\right) ~ \varepsilon_t}_{MA} + \underbrace{\Theta_Q \left(B^{s'}\right) ~ \varepsilon_t}_{SMA} - \varepsilon_t \\ \underbrace{\left( 1 + \sum_{i=1}^p \varphi_i B^i \right) ~ y_t}_{AR} + \underbrace{\left( 1 + \sum_{k=1}^P \Phi_k B^{sk} \right) ~ y_t}_{SAR} - y_t &= c + \underbrace{\left( 1 + \sum_{j=1}^q \theta_j B^j \right) ~ \varepsilon_t}_{MA} + \underbrace{\left( 1 + \sum_{l=1}^Q \Theta_l B^{s'l} \right) ~ \varepsilon_t}_{SMA} - \varepsilon_t \\ \underbrace{y_t + \sum_{i=1}^p \varphi_i B^i y_t}_{AR} + \underbrace{y_t + \sum_{k=1}^P \Phi_k B^{sk} y_t}_{SAR} - y_t &= c + \underbrace{\varepsilon_t + \sum_{j=1}^q \theta_j B^j \varepsilon_t}_{MA} + \underbrace{\varepsilon_t + \sum_{l=1}^Q \Theta_l B^{s'l} \varepsilon_t}_{SMA} - \varepsilon_t \\ y_t + \sum_{i=1}^p \varphi_i B^i y_t + \sum_{k=1}^P \Phi_k B^{sk} y_t &= c + \varepsilon_t + \sum_{j=1}^q \theta_j B^j \varepsilon_t + \sum_{l=1}^Q \Theta_l B^{s'l} \varepsilon_t \\ y_t + \sum_{i=1}^p \varphi_i y_{t-i} + \sum_{k=1}^P \Phi_k y_{t-sk} &= c + \varepsilon_t + \sum_{j=1}^q \theta_j \varepsilon_{t-j} + \sum_{l=1}^Q \Theta_l \varepsilon_{t-s'l} \\ y_t &= c + \underbrace{\sum_{i=1}^p \phi_i y_{t-i}}_{\text{AR model}} + \underbrace{\sum_{k=1}^{P} \varPhi_k y_{t-sk}}_{\text{SAR model}} + \varepsilon_t + \underbrace{\sum_{j=1}^q \theta_j \varepsilon_{t-j}}_{\text{MA model}} + \underbrace{\sum_{l=1}^{Q} \Theta_l \varepsilon_{t-s'l}}_{\text{SMA model}} \end{align} $$with $\varPhi = -\Phi$ and $\varphi = -\phi$
Remark: most of the time, $s = s'$. In this case, the process is simply noted $ARMA(\color{\red}{p}, \color{\green}{q}) + ARMA(\color{\red}{P}, \color{\green}{Q})_{s}$
These process are more often used than previously described additive processes.
Seasonal series are characterized by a strong serial correlation at the seasonal lag (and possibly multiples thereof).
$$ \begin{align} ARMA(\color{\red}{p},\color{\green}{q}) \times ARMA(\color{\red}{P},\color{\green}{Q})_{\color{\red}{s}, \color{\green}{s'}}: \quad \color{\red}{ \underbrace{\varphi_{(p)} (B)}_{AR} ~ \underbrace{\Phi_{(P)} (B^s)}_{SAR} ~ y_t } &= c + \color{\green}{ \underbrace{\theta_{(q)} (B)}_{MA} ~ \underbrace{\Theta_{(Q)} (B^{s'})}_{SMA} ~ \varepsilon_t } \\ \color{\red}{ \underbrace{\left( 1 + \sum_{i=1}^p \varphi_i B^i \right)}_{AR} ~ \underbrace{\left( 1 + \sum_{k=1}^P \Phi_k B^{sk} \right)}_{SAR} ~ y_t } &= c + \color{\green}{ \underbrace{\left( 1 + \sum_{j=1}^q \theta_j B^j \right)}_{MA} ~ \underbrace{\left( 1 + \sum_{l=1}^Q \Theta_l B^{s'l} \right)}_{SMA} ~ \varepsilon_t } \\ \color{\red}{ \underbrace{ \left( 1 + \varphi_1 B + \varphi_2 B^2 + \dots + \varphi_p B^p \right) \left( 1 + \Phi_1 B^{s} + \Phi_2 B^{s2} + \dots + \Phi_P B^{sP} \right) y_t }_{\text{Order of the equivalent AR process} ~=~ p + Ps} } &= c + \color{\green}{ \underbrace{ \left( 1 + \theta_1 B + \theta_2 B^2 + \dots + \theta_q B^q \right) \left( 1 + \Theta_1 B^{s'} + \Theta_2 B^{s'2} + \dots + \Theta_Q B^{s'Q} \right) \varepsilon_t }_{\text{Order of the equivalent MA process} ~=~ q + Qs'} } \\ \end{align} $$with $s > p$ and $s' > q$.
$ARMA(p,q) \times ARMA(P,Q)_{s, s'}$ can be defined as an $ARMA(p + Ps, ~ q + Qs')$ with some parameters fixed to 0.
Remark: most of the time, $s = s'$. In this case, the process is simply noted $ARMA(\color{\red}{p}, \color{\green}{q}) \times ARMA(\color{\red}{P}, \color{\green}{Q})_{s}$
with
$$\nabla_{s''} = (1-B^{s''})$$$$\nabla_{s''}^2 = (1-B^{s''})^2$$$$\dots$$$$\nabla_{s''}^d = (1-B^{s''})^d$$Remark: most of the time, $s = s' = s''$. In this case, the process is simply noted $ARIMA(\color{\red}{P}, \color{\orange}{D}, \color{\green}{Q})_{s}$
Remark: $\color{\orange}{\nabla^D_{s''}} \color{\orange}{\nabla^d} y_t = \color{\orange}{\nabla^d} \color{\orange}{\nabla^D_{s''}} y_t$
As we have:
$$ \begin{align} \nabla &= (1-B) \\ \nabla_{24} &= (1-B^{24}) \\ \nabla \nabla_{24} &= (1-B)(1-B^{24}) \\ &= 1-B^{24}-B+B^{25} \\ \nabla \nabla_{24} y_t &= (1-B^{24}-B+B^{25}) y_t \\ &= y_t - B^{24} y_t - B y_t + B^{25} y_t \\ &= y_t - y_{t-24} - y_{t-1} + y_{t-25} \\ \end{align} $$Thus:
$$ \begin{align} \color{\red}{y_t} = & ~ c \\ & \color{\red}{+ \varPhi_1 \color{\orange}{\nabla_{24}} \color{\orange}{\nabla} y_{t-1} + \varPhi_2 \color{\orange}{\nabla_{24}} \color{\orange}{\nabla} y_{t-2} + \varPhi_3 \color{\orange}{\nabla_{24}} \color{\orange}{\nabla} y_{t-3}} \\ & \color{\red}{+ \phi_1 \color{\orange}{\nabla_{24}} \color{\orange}{\nabla} y_{t-24} + \varPhi_1 \phi_1 \color{\orange}{\nabla_{24}} \color{\orange}{\nabla} y_{t-25} + \varPhi_2 \phi_1 \color{\orange}{\nabla_{24}} \color{\orange}{\nabla} y_{t-26} + \varPhi_3 \phi_1 \color{\orange}{\nabla_{24}} \color{\orange}{\nabla} y_{t-27}} \\ & \color{\red}{+ \phi_2 \color{\orange}{\nabla_{24}} \color{\orange}{\nabla} y_{t-48} + \varPhi_1 \phi_2 \color{\orange}{\nabla_{24}} \color{\orange}{\nabla} y_{t-49} + \varPhi_2 \phi_2 \color{\orange}{\nabla_{24}} \color{\orange}{\nabla} y_{t-50} + \varPhi_3 \phi_2 \color{\orange}{\nabla_{24}} \color{\orange}{\nabla} y_{t-51}} \\ & \color{\green}{+ \varepsilon_t} \\ & \color{\green}{+ \theta_1 \varepsilon_{t-1} + \theta_2 \varepsilon_{t-2}} \\ & \color{\green}{+ \Theta_1 \varepsilon_{t-24} + \theta_1 \Theta_1 \varepsilon_{t-25} + \theta_2 \Theta_1 \varepsilon_{t-26}} \\ = & ... \\ \end{align} $$