Random Vectors#

Take \(N\) random variables \(X_1,\dots,X_N\) and put them in a vector:

\[ \mathbf{X} = (X_1,\dots,X_N). \]

We say that \(\mathbf{X}\) is a random vector. Random vectors are used to model uncertain stuff that requires multiple numbers to be described. For example:

  • The state of a multi-body system can be described by the random vector of coordinates and velocities.

  • An unknown function could be modeled by the random vector of its function values at \(N\) test points.

  • An image could be modeled by the random vector of its pixel values.

Probability density function of a random vector#

The PDF of the random vector is the joint PDF of the components. We write:

\[ p(\mathbf{x}) = p(x_1,\dots,x_N). \]

This function is the joint PDF of \(N\) variables. You can marginalize it to get the PDF of any subset of the variables. For example, the PDF of the first component is:

\[ p(x_1) = \int_{-\infty}^{\infty}\dots\int_{-\infty}^{\infty} p(x_1,\dots,x_N)dx_2\dots dx_N. \]

But you can integrate any other subset of variables. For example, the PDF of the first and second components is:

\[ p(x_1,x_2) = \int_{-\infty}^{\infty}\dots\int_{-\infty}^{\infty} p(x_1,\dots,x_N)dx_3\dots dx_N. \]

And so on.

Expectation of a random vector#

The expectation of a random vector is the vector of expectations of each component:

\[\begin{split} \mathbb{E}[\mathbf{X}] = \begin{pmatrix} \mathbb{E}[X_1]\\ \vdots\\ \mathbb{E}[X_N] \end{pmatrix} \end{split}\]

This satisfies properties similar to the expectation of scalar random variables. For example, for any real number \(\lambda\), we have that:

\[ \mathbb{E}[\lambda \mathbf{X}] = \lambda\mathbb{E}[X]. \]

Also, if \(\mathbf{Y}\) is another \(N\)-dimensional random vector, we have:

\[ \mathbb{E}[\mathbf{X}+\mathbf{Y}] = \mathbb{E}[\mathbf{X}] + \mathbb{E}[\mathbf{Y}]. \]

Covariance matrix of two random vectors#

Let \(\mathbf{X}\) and \(\mathbf{Y}\) be \(N\)- and \(M\)-dimensional random vectors, respectively. The covariance of \(\mathbb{X}\) and \(\mathbf{Y}\) is the \(N\times M\) matrix consisting of all covariances between the components of \(\mathbf{X}\) and \(\mathbf{Y}\), i.e.,

\[ \mathbb{C}[\mathbf{X}, \mathbf{Y}] = (\mathbb{C}[X_i, Y_j]). \]

It can also be rewritten as the expectation of a matrix:

\[ \mathbb{C}[\mathbf{X}, \mathbf{Y}] = \mathbb{E}\left[\left(\mathbf{X}-\mathbb{E}[\mathbf{X}]\right)\left(\mathbf{Y}-\mathbb{E}[\mathbf{Y}]\right)^T\right]. \]

Here we assumed that the expectation operator is applied to each one of the matrix components.

It is easy to show that each argument’s covariance is a linear function.

The \(N\times N\) matrix \(\mathbb{C}[\mathbf{X},\mathbf{X}]\) is the self covariance matrix (or just covariance matrix) of \(\mathbf{X}\). The diagonal of the covariance matrix of \(\mathbf{X}\) contains the variances of each component of \(\mathbf{X}\).