In statistics and signal processing, an **autoregressive** (**AR**) **model** is a representation of a type of random process; as such, it is used to describe certain time-varying processes in nature, economics, etc. The autoregressive model specifies that the output variable depends linearly on its own previous values and on a stochastic term (an imperfectly predictable term); thus the model is in the form of a stochastic difference equation.

Together with the moving-average (MA) model, it is a special case and key component of the more general ARMA and ARIMA models of time series, which have a more complicated stochastic structure; it is also a special case of the vector autoregressive model (VAR), which consists of a system of more than one interlocking stochastic difference equation in more than one evolving random variable.

Contrary to the moving-average model, the autoregressive model is not always stationary as it may contain a unit root.

==Definition==

The notation $AR(p)$ indicates an autoregressive model of order *p*. The AR(*p*) model is defined as

:$X\_t\; =\; c\; +\; \backslash sum\_\{i=1\}^p\; \backslash varphi\_i\; X\_\{t-i\}+\; \backslash varepsilon\_t\; \backslash ,$

where $\backslash varphi\_1,\; \backslash ldots,\; \backslash varphi\_p$ are the *parameters* of the model, $c$ is a constant, and $\backslash varepsilon\_t$ is white noise. This can be equivalently written using the backshift operator *B* as

:$X\_t\; =\; c\; +\; \backslash sum\_\{i=1\}^p\; \backslash varphi\_i\; B^i\; X\_t\; +\; \backslash varepsilon\_t$

so that, moving the summation term to the left side and using polynomial notation, we have

:$\backslash phi\; [B]X\_t=\; c\; +\; \backslash varepsilon\_t\; \backslash ,\; .$

An autoregressive model can thus be viewed as the output of an all-pole infinite impulse response filter whose input is white noise.

Some parameter constraints are necessary for the model to remain wide-sense stationary. For example, processes in the AR(1) model with $|\backslash varphi\_1\; |\; \backslash geq\; 1$ are not stationary. More generally, for an AR(*p*) model to be wide-sense stationary, the roots of the polynomial $\backslash textstyle\; z^p\; -\; \backslash sum\_\{i=1\}^p\; \backslash varphi\_i\; z^\{p-i\}$ must lie inside of the unit circle, i.e., each (complex) root $z\_i$ must satisfy $|z\_i|<1$.

==Intertemporal effect of shocks==

In an AR process, a one-time shock affects values of the evolving variable infinitely far into the future. For example, consider the AR(1) model $X\_t\; =\; c\; +\; \backslash varphi\_1\; X\_\{t-1\}\; +\; \backslash varepsilon\_t$. A non-zero value for $\backslash varepsilon\_t$ at say time *t*=1 affects $X\_1$ by the amount $\backslash varepsilon\_1$. Then by the AR equation for $X\_2$ in terms of $X\_1$, this affects $X\_2$ by the amount $\backslash varphi\_1\; \backslash varepsilon\_1$. Then by the AR equation for $X\_3$ in terms of $X\_2$, this affects $X\_3$ by the amount $\backslash varphi\_1^2\; \backslash varepsilon\_1$. Continuing this process shows that the effect of $\backslash varepsilon\_1$ never ends, although if the process is stationary then the effect diminishes toward zero in the limit.

Because each shock affects *X* values infinitely far into the future from when they occur, any given value *X*_{t} is affected by shocks occurring infinitely far into the past. This can also be seen by rewriting the autoregression

:$\backslash phi\; (B)X\_t=\; \backslash varepsilon\_t\; \backslash ,$

(where the constant term has been suppressed by assuming that the variable has been measured as deviations from its mean) as

:$X\_t=\; \backslash frac\{1\}\{\backslash phi\; (B)\}\backslash varepsilon\_t\; \backslash ,\; .$

When the polynomial division on the right side is carried out, the polynomial in the backshift operator applied to $\backslash varepsilon\_t$ has an infinite order—that is, an infinite number of lagged values of $\backslash varepsilon\_t$ appear on the right side of the equation.

==Characteristic polynomial==
The autocorrelation function of an AR(*p*) process can be expressed as
:$\backslash rho(\backslash tau)\; =\; \backslash sum\_\{k=1\}^p\; a\_k\; y\_k^\{-|\backslash tau|\}\; ,$

where $y\_k$ are the roots of the polynomial : $\backslash phi(B)\; =\; 1-\; \backslash sum\_\{k=1\}^p\; \backslash varphi\_k\; B^k$

where *B* is the backshift operator, where $\backslash phi(\backslash cdot)$ is the function defining the autoregression, and where $\backslash varphi\_k$ are the coefficients in the autoregression.

The autocorrelation function of an AR(*p*) process is a sum of decaying exponentials.
* Each real root contributes a component to the autocorrelation function that decays exponentially.
* Similarly, each pair of complex conjugate roots contributes an exponentially damped oscillation.

==Graphs of AR(*p*) processes==

The simplest AR process is AR(0), which has no dependence between the terms. Only the error/innovation/noise term contributes to the output of the process, so in the figure, AR(0) corresponds to white noise.

For an AR(1) process with a positive $\backslash varphi$, only the previous term in the process and the noise term contribute to the output. If $\backslash varphi$ is close to 0, then the process still looks like white noise, but as $\backslash varphi$ approaches 1, the output gets a larger contribution from the previous term relative to the noise. This results in a "smoothing" or integration of the output, similar to a low pass filter.

For an AR(2) process, the previous two terms and the noise term contribute to the output. If both $\backslash varphi\_1$ and $\backslash varphi\_2$ are positive, the output will resemble a low pass filter, with the high frequency part of the noise decreased. If $\backslash varphi\_1$ is positive while $\backslash varphi\_2$ is negative, then the process favors changes in sign between terms of the process. The output oscillates. This can be likened to edge detection or detection of change in direction.

==Example: An AR(1) process==

An AR(1) process is given by:

:$X\_t\; =\; c\; +\; \backslash varphi\; X\_\{t-1\}+\backslash varepsilon\_t\backslash ,$

where $\backslash varepsilon\_t$ is a white noise process with zero mean and constant variance $\backslash sigma\_\backslash varepsilon^2$.
(Note: The subscript on $\backslash varphi\_1$ has been dropped.) The process is wide-sense stationary if $|\backslash varphi|<1$ since it is obtained as the output of a stable filter whose input is white noise. (If $\backslash varphi=1$ then $X\_t$ has infinite variance, and is therefore not wide sense stationary.) Assuming $|\backslash varphi|<1$, the mean $\backslash operatorname\{E\}\; (X\_t)$ is identical for all values of *t* by the very definition of wide sense stationarity. If the mean is denoted by $\backslash mu$, it follows from

:$\backslash operatorname\{E\}\; (X\_t)=\backslash operatorname\{E\}\; (c)+\backslash varphi\backslash operatorname\{E\}\; (X\_\{t-1\})+\backslash operatorname\{E\}(\backslash varepsilon\_t),$ that :$\backslash mu=c+\backslash varphi\backslash mu+0,$

and hence

:$\backslash mu=\backslash frac\{c\}\{1-\backslash varphi\}.$

In particular, if $c\; =\; 0$, then the mean is 0.

The variance is

:$\backslash textrm\{var\}(X\_t)=\backslash operatorname\{E\}(X\_t^2)-\backslash mu^2=\backslash frac\{\backslash sigma\_\backslash varepsilon^2\}\{1-\backslash varphi^2\},$ where $\backslash sigma\_\backslash varepsilon$ is the standard deviation of $\backslash varepsilon\_t$. This can be shown by noting that :$\backslash textrm\{var\}(X\_t)\; =\; \backslash varphi^2\backslash textrm\{var\}(X\_\{t-1\})\; +\; \backslash sigma\_\backslash varepsilon^2,$ and then by noticing that the quantity above is a stable fixed point of this relation.

The autocovariance is given by

:$B\_n=\backslash operatorname\{E\}(X\_\{t+n\}X\_t)-\backslash mu^2=\backslash frac\{\backslash sigma\_\backslash varepsilon^2\}\{1-\backslash varphi^2\}\backslash ,\backslash ,\backslash varphi^\{|n|\}.$

It can be seen that the autocovariance function decays with a decay time (also called time constant) of $\backslash tau=-1/\backslash ln(\backslash varphi)$ [to see this, write $B\_n=K\backslash varphi^\{|n|\}$ where $K$ is independent of $n$. Then note that $\backslash varphi^\{|n|\}=e^\{|n|\backslash ln\backslash varphi\}$ and match this to the exponential decay law $e^\{-n/\backslash tau\}$].

The spectral density function is the Fourier transform of the autocovariance function. In discrete terms this will be the discrete-time Fourier transform:

:$\backslash Phi(\backslash omega)=\; \backslash frac\{1\}\{\backslash sqrt\{2\backslash pi\}\}\backslash ,\backslash sum\_\{n=-\backslash infty\}^\backslash infty\; B\_n\; e^\{-i\backslash omega\; n\}\; =\backslash frac\{1\}\{\backslash sqrt\{2\backslash pi\}\}\backslash ,\backslash left(\backslash frac\{\backslash sigma\_\backslash varepsilon^2\}\{1+\backslash varphi^2-2\backslash varphi\backslash cos(\backslash omega)\}\backslash right).$

This expression is periodic due to the discrete nature of the $X\_j$, which is manifested as the cosine term in the denominator. If we assume that the sampling time ($\backslash Delta\; t=1$) is much smaller than the decay time ($\backslash tau$), then we can use a continuum approximation to $B\_n$:

:$B(t)\backslash approx\; \backslash frac\{\backslash sigma\_\backslash varepsilon^2\}\{1-\backslash varphi^2\}\backslash ,\backslash ,\backslash varphi^\{|t|\}$

which yields a Lorentzian profile for the spectral density:

:$\backslash Phi(\backslash omega)=\; \backslash frac\{1\}\{\backslash sqrt\{2\backslash pi\}\}\backslash ,\backslash frac\{\backslash sigma\_\backslash varepsilon^2\}\{1-\backslash varphi^2\}\backslash ,\backslash frac\{\backslash gamma\}\{\backslash pi(\backslash gamma^2+\backslash omega^2)\}$

where $\backslash gamma=1/\backslash tau$ is the angular frequency associated with the decay time $\backslash tau$.

An alternative expression for $X\_t$ can be derived by first substituting $c+\backslash varphi\; X\_\{t-2\}+\backslash varepsilon\_\{t-1\}$ for $X\_\{t-1\}$ in the defining equation. Continuing this process *N* times yields

:$X\_t=c\backslash sum\_\{k=0\}^\{N-1\}\backslash varphi^k+\backslash varphi^NX\_\{t-N\}+\backslash sum\_\{k=0\}^\{N-1\}\backslash varphi^k\backslash varepsilon\_\{t-k\}.$

For *N* approaching infinity, $\backslash varphi^N$ will approach zero and:

:$X\_t=\backslash frac\{c\}\{1-\backslash varphi\}+\backslash sum\_\{k=0\}^\backslash infty\backslash varphi^k\backslash varepsilon\_\{t-k\}.$

It is seen that $X\_t$ is white noise convolved with the $\backslash varphi^k$ kernel plus the constant mean. If the white noise $\backslash varepsilon\_t$ is a Gaussian process then $X\_t$ is also a Gaussian process. In other cases, the central limit theorem indicates that $X\_t$ will be approximately normally distributed when $\backslash varphi$ is close to one.

=== Explicit mean/difference form of AR(1) process ===

The AR(1) model is the discrete time analogy of the continuous Ornstein-Uhlenbeck process. It is therefore sometimes useful to understand the properties of the AR(1) model cast in an equivalent form. In this form, the AR(1) model is given by:

:$X\_\{t+1\}\; =\; X\_t\; +\; \backslash theta(\backslash mu\; -\; X\_t)\; +\; \backslash epsilon\_\{t+1\}\backslash ,$, where $|\backslash theta|\; <\; 1\; \backslash ,$ and $\backslash mu$ is the model mean.

By putting this in the form $X\_\{t+1\}\; =\; c\; +\; \backslash phi\; X\_t\backslash ,$, and then expanding the series for $X\_\{t+n\}$, one can show that:

:$\backslash operatorname\{E\}(X\_\{t+n\}\; |\; X\_t)\; =\; \backslash mu\backslash left[1-\backslash left(1-\backslash theta\backslash right)^n\backslash right]\; +\; X\_t(1-\backslash theta)^n$, and :$\backslash operatorname\{Var\}\; (X\_\{t+n\}\; |\; X\_t)\; =\; \backslash sigma^2\; \backslash frac\{\; 1\; -\; (1\; -\; \backslash theta)^\{2n\}\; \}\{1\; -\; (1-\backslash theta)^2\}$.

==Choosing the maximum lag==

The partial autocorrelation of an AR(p) process is zero at lag p + 1 and greater, so the appropriate maximum lag is the one beyond which the partial autocorrelations are all zero.

==Calculation of the AR parameters==

There are many ways to estimate the coefficients, such as the ordinary least squares procedure or method of moments (through Yule–Walker equations).

The AR(*p*) model is given by the equation

:$X\_t\; =\; \backslash sum\_\{i=1\}^p\; \backslash varphi\_i\; X\_\{t-i\}+\; \backslash varepsilon\_t.\backslash ,$

It is based on parameters $\backslash varphi\_i$ where *i* = 1, ..., *p*. There is a direct correspondence between these parameters and the covariance function of the process, and this correspondence can be inverted to determine the parameters from the autocorrelation function (which is itself obtained from the covariances). This is done using the Yule–Walker equations.

===Yule–Walker equations===

The Yule–Walker equations, named for Udny Yule and Gilbert Walker, are the following set of equations.

:$\backslash gamma\_m\; =\; \backslash sum\_\{k=1\}^p\; \backslash varphi\_k\; \backslash gamma\_\{m-k\}\; +\; \backslash sigma\_\backslash varepsilon^2\backslash delta\_\{m,0\},$

where , yielding equations. Here $\backslash gamma\_m$ is the autocovariance function of X_{t}, $\backslash sigma\_\backslash varepsilon$ is the standard deviation of the input noise process, and $\backslash delta\_\{m,0\}$ is the Kronecker delta function.

Because the last part of an individual equation is non-zero only if , the set of equations can be solved by representing the equations for in matrix form, thus getting the equation

:$\backslash begin\{bmatrix\}\; \backslash gamma\_1\; \backslash \backslash \; \backslash gamma\_2\; \backslash \backslash \; \backslash gamma\_3\; \backslash \backslash \; \backslash vdots\; \backslash \backslash \; \backslash gamma\_p\; \backslash \backslash \; \backslash end\{bmatrix\}$

=

\begin{bmatrix} \gamma_0 & \gamma_{-1} & \gamma_{-2} & \dots \\ \gamma_1 & \gamma_0 & \gamma_{-1} & \dots \\ \gamma_2 & \gamma_{1} & \gamma_{0} & \dots \\ \vdots & \vdots & \vdots & \ddots \\ \gamma_{p-1} & \gamma_{p-2} & \gamma_{p-3} & \dots \\ \end{bmatrix}

\begin{bmatrix} \varphi_{1} \\ \varphi_{2} \\ \varphi_{3} \\ \vdots \\ \varphi_{p} \\ \end{bmatrix}

which can be solved for all $\backslash \{\backslash varphi\_m;\; m=1,2,\; \backslash cdots\; ,p\backslash \}.$ The remaining equation for *m* = 0 is

:$\backslash gamma\_0\; =\; \backslash sum\_\{k=1\}^p\; \backslash varphi\_k\; \backslash gamma\_\{-k\}\; +\; \backslash sigma\_\backslash varepsilon^2\; ,$

which, once $\backslash \{\backslash varphi\_m\; ;\; m=1,2,\; \backslash cdots\; ,p\; \backslash \}$ are known, can be solved for $\backslash sigma\_\backslash varepsilon^2\; .$

An alternative formulation is in terms of the autocorrelation function. The AR parameters are determined by the first p+1 elements $\backslash rho(\backslash tau)$ of the autocorrelation function. The full autocorrelation function can then be derived by recursively calculating

: $\backslash rho(\backslash tau)\; =\; \backslash sum\_\{k=1\}^p\; \backslash varphi\_k\; \backslash rho(k-\backslash tau)$
Examples for some Low-order AR(*p*) processes
* p=1
** $\backslash gamma\_1\; =\; \backslash varphi\_1\; \backslash gamma\_0$
** Hence $\backslash rho\_1\; =\; \backslash gamma\_1\; /\; \backslash gamma\_0\; =\; \backslash varphi\_1$
* p=2
** The Yule–Walker equations for an AR(2) process are
**: $\backslash gamma\_1\; =\; \backslash varphi\_1\; \backslash gamma\_0\; +\; \backslash varphi\_2\; \backslash gamma\_\{-1\}$
**: $\backslash gamma\_2\; =\; \backslash varphi\_1\; \backslash gamma\_1\; +\; \backslash varphi\_2\; \backslash gamma\_0$
*** Remember that $\backslash gamma\_\{-k\}\; =\; \backslash gamma\_k$
*** Using the first equation yields $\backslash rho\_1\; =\; \backslash gamma\_1\; /\; \backslash gamma\_0\; =\; \backslash frac\{\backslash varphi\_1\}\{1-\backslash varphi\_2\}$
*** Using the recursion formula yields $\backslash rho\_2\; =\; \backslash gamma\_2\; /\; \backslash gamma\_0\; =\; \backslash frac\{\backslash varphi\_1^2\; -\; \backslash varphi\_2^2\; +\; \backslash varphi\_2\}\{1-\backslash varphi\_2\}$

===Estimation of AR parameters===

The above equations (the Yule–Walker equations) provide several routes to estimating the parameters of an AR(*p*) model, by replacing the theoretical covariances with estimated values. Some of these variants can be described as follows:

*Estimation of autocovariances or autocorrelations. Here each of these terms is estimated separately, using conventional estimates. There are different ways of doing this and the choice between these affects the properties of the estimation scheme. For example, negative estimates of the variance can be produced by some choices.
*Formulation as a least squares regression problem in which an ordinary least squares prediction problem is constructed, basing prediction of values of *X*_{t} on the *p* previous values of the same series. This can be thought of as a forward-prediction scheme. The normal equations for this problem can be seen to correspond to an approximation of the matrix form of the Yule–Walker equations in which each appearance of an autocovariance of the same lag is replaced by a slightly different estimate.
*Formulation as an extended form of ordinary least squares prediction problem. Here two sets of prediction equations are combined into a single estimation scheme and a single set of normal equations. One set is the set of forward-prediction equations and the other is a corresponding set of backward prediction equations, relating to the backward representation of the AR model:
::$X\_t\; =\; c\; +\; \backslash sum\_\{i=1\}^p\; \backslash varphi\_i\; X\_\{t-i\}+\; \backslash varepsilon^*\_t\; \backslash ,.$
:Here predicted of values of *X*_{t} would be based on the *p* future values of the same series. This way of estimating the AR parameters is due to Burg,
* MATLAB's Econometrics Toolbox and System Identification Toolbox includes autoregressive models
* Matlab and Octave: the *TSA toolbox* contains several estimation functions for uni-variate, multivariate and adaptive autoregressive models.

==Impulse response==

The impulse response of a system is the change in an evolving variable in response to a change in the value of a shock term *k* periods earlier, as a function of *k*. Since the AR model is a special case of the vector autoregressive model, the computation of the impulse response in Vector autoregression#Impulse response applies here.

==*n*-step-ahead forecasting==

Once the parameters of the autoregression

:$X\_t\; =\; c\; +\; \backslash sum\_\{i=1\}^p\; \backslash varphi\_i\; X\_\{t-i\}+\; \backslash varepsilon\_t\; \backslash ,$

have been estimated, the autoregression can be used to forecast an arbitrary number of periods into the future. First use *t* to refer to the first period for which data is not yet available; substitute the known preceding values *X*_{t-i} for *i=*1, ..., *p* into the autoregressive equation while setting the error term $\backslash varepsilon\_t$ equal to zero (because we forecast *X*_{t} to equal its expected value, and the expected value of the unobserved error term is zero). The output of the autoregressive equation is the forecast for the first unobserved period. Next, use *t* to refer to the *next* period for which data is not yet available; again the autoregressive equation is used to make the forecast, with one difference: the value of *X* one period prior to the one now being forecast is not known, so its expected value—the predicted value arising from the previous forecasting step—is used instead. Then for future periods the same procedure is used, each time using one more forecast value on the right side of the predictive equation until, after *p* predictions, all *p* right-side values are predicted values from preceding steps.

There are four sources of uncertainty regarding predictions obtained in this manner: (1) uncertainty as to whether the autoregressive model is the correct model; (2) uncertainty about the accuracy of the forecasted values that are used as lagged values in the right side of the autoregressive equation; (3) uncertainty about the true values of the autoregressive coefficients; and (4) uncertainty about the value of the error term $\backslash varepsilon\_t\; \backslash ,$ for the period being predicted. Each of the last three can be quantified and combined to give a confidence interval for the *n*-step-ahead predictions; the confidence interval will become wider as *n* increases because of the use of an increasing number of estimated values for the right-side variables.

==Evaluating the quality of forecasts==

The predictive performance of the autoregressive model can be assessed as soon as estimation has been done if cross-validation is used. In this approach, some of the initially available data was used for parameter estimation purposes, and some (from available observations later in the data set) was held back for out-of-sample testing. Alternatively, after some time has passed after the parameter estimation was conducted, more data will have become available and predictive performance can be evaluated then using the new data.

In either case, there are two aspects of predictive performance that can be evaluated: one-step-ahead and *n*-step-ahead performance. For one-step-ahead performance, the estimated parameters are used in the autoregressive equation along with observed values of *X* for all periods prior to the one being predicted, and the output of the equation is the one-step-ahead forecast; this procedure is used to obtain forecasts for each of the out-of-sample observations. To evaluate the quality of *n*-step-ahead forecasts, the forecasting procedure in the previous section is employed to obtain the predictions.

Given a set of predicted values and a corresponding set of actual values for *X* for various time periods, a common evaluation technique is to use the mean squared prediction error; other measures are also available (see Forecasting#Forecasting accuracy).

The question of how to interpret the measured forecasting accuracy arises—for example, what is a "high" (bad) or a "low" (good) value for the mean squared prediction error? There are two possible points of comparison. First, the forecasting accuracy of an alternative model, estimated under different modeling assumptions or different estimation techniques, can be used for comparison purposes. Second, the out-of-sample accuracy measure can be compared to the same measure computed for the in-sample data points (that were used for parameter estimation) for which enough prior data values are available (that is, dropping the first *p* data points, for which *p* prior data points are not available). Since the model was estimated specifically to fit the in-sample points as well as possible, it will usually be the case that the out-of-sample predictive performance will be poorer than the in-sample predictive performance. But if the predictive quality deteriorates out-of-sample by "not very much" (which is not precisely definable), then the forecaster may be satisfied with the performance.

==See also== * Moving average model * Linear difference equation * Predictive analytics * Linear predictive coding * Resonance

==Notes==

==References== * * *

==External links== *[http://paulbourke.net/miscellaneous/ar/ AutoRegression Analysis (AR)] by Paul Bourke * by Mark Thoma

Category:Autocorrelation Category:Signal processing

Texte soumis à la licence CC-BY-SA. Source : Article https://en.wikipedia.org/wiki/Autoregressive_model de Wikipédia