State Space Models - Kalman Filters#
Kalman filters solve the filtering problem for the case of linear transitions and emissions with Gaussian probabilities. In the notation of the previous lecture, these can be expressed either as equations:
Initial conditions:
where \(\boldsymbol{\mu}_0\) is a parameter and \(\mathbf{z}_0\sim N(\mathbf{0},\mathbf{V}_0)\) with \(\mathbf{V}_0\) an properly sized covariance matrix.
Transitions:
where \(\mathbf{A}\) and \(\mathbf{B}\) are matrices and \(\mathbf{z}_t\sim N(\mathbf{0},\mathbf{Q})\) with \(\mathbf{Q}\) an appropriately sized covariance matrix (known as process covariance).
Emissions:
where \(\mathbf{C}\) is a matrix and \(\mathbf{w}_t\sim N(\mathbf{0},\mathbf{R})\) with \(\mathbf{R}\) an appropriately sized covariance matrix (known as measurement covariance).
or as probabilities:
Initial conditions:
Transitions:
Emissions:
The complete details of the Kalman filter are somewhat involved. If you want to dive deeper into the technical part, I suggest you read Chapter 13.3, Bishop (2006). Another excellent resource, albeit rather extensive, is the open-source book Kalman and Bayesian Filters in Python and in particular Chapter 7.
The Kalmann filter algorithm#
The Kalman filter algorithm is a recursive algorithm that computes the posterior distribution \(p(\mathbf{x}_t|\mathbf{y}_{1:t}, \mathbf{u}_{1:t})\). It is composed of two steps:
Prediction step: This step computes the prior distribution \(p(\mathbf{x}_t|\mathbf{y}_{1:t-1}, \mathbf{u}_{1:t})\).
Update step: This step computes the posterior distribution \(p(\mathbf{x}_t|\mathbf{y}_{1:t}, \mathbf{u}_{1:t})\).
We will now go through each of these steps in detail.
Prediction step#
Suppose we have computed the posterior distribution \(p(\mathbf{x}_{t-1}|\mathbf{y}_{1:t-1}, \mathbf{u}_{1:t-1})\). It turns out that this distribution is a Gaussian distribution with mean \(\boldsymbol{\mu}_{t-1}\) and covariance matrix \(\mathbf{V}_{t-1}\). We will see how we can compute these parameters recursively in the update step. For now assume we have them. So:
We can now compute the prior distribution \(p(\mathbf{x}_t|\mathbf{y}_{1:t-1}, \mathbf{u}_{1:t})\). We can do this by marginalizing over \(\mathbf{x}_{t-1}\):
In this last step we have used properties of Gaussian distributions. We write the mean and covariance matrix of the prior distribution as:
We have used a superscript \(p\) to denote that these are computed in the prediction step.
Update step#
Now suppose we have computed the prior distribution \(p(\mathbf{x}_t|\mathbf{y}_{1:t-1}, \mathbf{u}_{1:t})\) during the prediction step. It is a Gaussian distribution with mean \(\boldsymbol{\mu}^p_t\) and covariance matrix \(\mathbf{V}^p_t\). We want to condition on a new measurement \(\mathbf{y}_t\). We apply Bayes’ rule:
Again we have a product of two Gaussian distributions. So, the resulting distribution is also Gaussian. The mean and covariance matrix of this distribution are given by:
where \(\mathbf{K}_t\) is the Kalman gain matrix:
Smoothing#
The Kalman filter algorithm computes the posterior distribution \(p(\mathbf{x}_t|\mathbf{y}_{1:t}, \mathbf{u}_{1:t})\). This is the distribution of the state at time \(t\) given all measurements and control inputs up to time \(t\). This is useful for many applications. However, sometimes we want to compute the posterior distribution \(p(\mathbf{x}_t|\mathbf{y}_{1:T}, \mathbf{u}_{1:T})\). This is the distribution of the state at time \(t\) given all measurements and control inputs up to time \(T\). This is called the smoothing problem. The difference is that we use information from the future to improve our estimate of the state at time \(t\).
The smoothing problem can be solved by first computing the filtering distribution \(p(\mathbf{x}_t|\mathbf{y}_{1:t}, \mathbf{u}_{1:t})\) for all \(t\). This can be done using the Kalman filter algorithm. Once we are done with the Kalman filter, we have the desired distribution for the last time step \(p(\mathbf{x}_T|\mathbf{y}_{1:T}, \mathbf{u}_{1:T})\). It is a Gaussian distribution with mean \(\boldsymbol{\mu}_T\) and covariance matrix \(\mathbf{V}_T\). How do we get the distribution for the previous time step \(p(\mathbf{x}_{T-1}|\mathbf{y}_{1:T}, \mathbf{u}_{1:T})\)?
The joint distribution of \(\mathbf{x}_{T-1}\) and \(\mathbf{x}_T\) is given by:
We have the right-most term. Let’s just worry about the first term on the right-hand side. Think about it. To go from step \(T-1\) to \(T\) you need to know the control \(u_T\). All other controls do not matter. Also, since we condition on the state at time \(T\), the measurements at time \(T\) do not matter either. We have:
Okay. Now we need to make the transition probability appear. Let’s use Bayes’ rule:
This is starting to look good. The denominator is the prediction step of the Kalman filter. It is:
The second term in the numerator is the filtering distribution at time \(T-1\):
For the first term in the numerator, we use the Markov property. The state at time \(T\) only depends on the state at time \(T-1\) and the control at time \(T\). So:
Let’s put everything together:
From this point on, it is just boring algebra. You have to show that the joint is a Gaussian and then you need to integrate out \(\mathbf{x}_T\) to get the distribution of \(\mathbf{x}_{T-1}\). The result is a Gaussian with mean \(\boldsymbol{\mu}_{T-1|T}\) and covariance matrix \(\mathbf{V}_{T-1|T}\):
where \(\mathbf{J}_T\) is the smoother gain matrix:
And you keep going back in time until you have the distribution for all time steps.