Cross-covariance matrix

Not to be confused with Covariance matrix.

Short description: Type of matrix in probability theory and statistics

In probability theory and statistics, a cross-covariance matrix is a matrix whose element in the i, j position is the covariance between the i-th element of a random vector and j-th element of another random vector. A random vector is a random variable with multiple dimensions. Each element of the vector is a scalar random variable. Each element has either a finite number of observed empirical values or a finite or infinite number of potential values. The potential values are specified by a theoretical joint probability distribution. Intuitively, the cross-covariance matrix generalizes the notion of covariance to multiple dimensions.

The cross-covariance matrix of two random vectors $\mathbf{𝐗}$ and $\mathbf{𝐘}$ is typically denoted by ${\mathrm{K}}_{\mathbf{𝐗}\mathbf{𝐘}}$ or ${\mathrm{\Sigma }}_{\mathbf{𝐗}\mathbf{𝐘}}$.

Definition

For random vectors $\mathbf{𝐗}$ and $\mathbf{𝐘}$, each containing random elements whose expected value and variance exist, the cross-covariance matrix of $\mathbf{𝐗}$ and $\mathbf{𝐘}$ is defined by

${\mathrm{K}}_{\mathbf{𝐗}\mathbf{𝐘}}=\mathrm{cov}(\mathbf{𝐗},\mathbf{𝐘})\stackrel{\mathrm{d}\mathrm{e}\mathrm{f}}{=}\mathrm{E}[(\mathbf{𝐗}-{\mathit{\mu }}_{\mathbf{𝐗}})(\mathbf{𝐘}-{\mathit{\mu }}_{\mathbf{𝐘}}{)}^{\mathrm{T}}]$

(Eq.1)

where ${\mathit{\mu }}_{\mathbf{𝐗}}=\mathrm{E}[\mathbf{𝐗}]$ and ${\mathit{\mu }}_{\mathbf{𝐘}}=\mathrm{E}[\mathbf{𝐘}]$ are vectors containing the expected values of $\mathbf{𝐗}$ and $\mathbf{𝐘}$. The vectors $\mathbf{𝐗}$ and $\mathbf{𝐘}$ need not have the same dimension, and either might be a scalar value.

The cross-covariance matrix is the matrix whose $(i,j)$ entry is the covariance

${\mathrm{K}}_{X_i{Y}_j}=\mathrm{cov}[X_i,Y_j]=\mathrm{E}[(X_i-\mathrm{E}[X_i])(Y_j-\mathrm{E}[Y_j])]$

between the i-th element of $\mathbf{𝐗}$ and the j-th element of $\mathbf{𝐘}$. This gives the following component-wise definition of the cross-covariance matrix.

${\mathrm{K}}_{\mathbf{𝐗}\mathbf{𝐘}}=\left[\begin{array}{cccc}\mathrm{E}[(X_1-\mathrm{E}[X_1])(Y_1-\mathrm{E}[Y_1])]& \mathrm{E}[(X_1-\mathrm{E}[X_1])(Y_2-\mathrm{E}[Y_2])]& \cdots & \mathrm{E}[(X_1-\mathrm{E}[X_1])(Y_n-\mathrm{E}[Y_n])]\\ \\ \mathrm{E}[(X_2-\mathrm{E}[X_2])(Y_1-\mathrm{E}[Y_1])]& \mathrm{E}[(X_2-\mathrm{E}[X_2])(Y_2-\mathrm{E}[Y_2])]& \cdots & \mathrm{E}[(X_2-\mathrm{E}[X_2])(Y_n-\mathrm{E}[Y_n])]\\ \\ ⋮& ⋮& \ddots & ⋮\\ \\ \mathrm{E}[(X_m-\mathrm{E}[X_m])(Y_1-\mathrm{E}[Y_1])]& \mathrm{E}[(X_m-\mathrm{E}[X_m])(Y_2-\mathrm{E}[Y_2])]& \cdots & \mathrm{E}[(X_m-\mathrm{E}[X_m])(Y_n-\mathrm{E}[Y_n])]\end{array}\right]$

Example

For example, if $\mathbf{𝐗}={(X_1,X_2,X_3)}^{\mathrm{T}}$ and $\mathbf{𝐘}={(Y_1,Y_2)}^{\mathrm{T}}$ are random vectors, then $\mathrm{cov}(\mathbf{𝐗},\mathbf{𝐘})$ is a $3\times 2$ matrix whose $(i,j)$-th entry is $\mathrm{cov}(X_i,Y_j)$.

Properties

For the cross-covariance matrix, the following basic properties apply:

1. $\mathrm{cov}(\mathbf{𝐗},\mathbf{𝐘})=\mathrm{E}[\mathbf{𝐗}{\mathbf{𝐘}}^{\mathrm{T}}]-{\mathit{\mu }}_{\mathbf{𝐗}}{{\mathit{\mu }}_{\mathbf{𝐘}}}^{\mathrm{T}}$
2. $\mathrm{cov}(\mathbf{𝐗},\mathbf{𝐘})=\mathrm{cov}(\mathbf{𝐘},\mathbf{𝐗}{)}^{\mathrm{T}}$
3. $\mathrm{cov}({\mathbf{X}}_{\mathbf{𝟏}}+{\mathbf{X}}_{\mathbf{𝟐}},\mathbf{𝐘})=\mathrm{cov}({\mathbf{X}}_{\mathbf{𝟏}},\mathbf{𝐘})+\mathrm{cov}({\mathbf{X}}_{\mathbf{𝟐}},\mathbf{𝐘})$
4. $\mathrm{cov}(A\mathbf{𝐗}+\mathbf{𝐚},B^{\mathrm{T}}\mathbf{𝐘}+\mathbf{𝐛})=A\mathrm{cov}(\mathbf{𝐗},\mathbf{𝐘})B$
5. If $\mathbf{𝐗}$ and $\mathbf{𝐘}$ are independent (or somewhat less restrictedly, if every random variable in $\mathbf{𝐗}$ is uncorrelated with every random variable in $\mathbf{𝐘}$), then $\mathrm{cov}(\mathbf{𝐗},\mathbf{𝐘})=0_{p\times q}$

where $\mathbf{𝐗}$, ${\mathbf{X}}_{\mathbf{𝟏}}$ and ${\mathbf{X}}_{\mathbf{𝟐}}$ are random $p\times 1$ vectors, $\mathbf{𝐘}$ is a random $q\times 1$ vector, $\mathbf{𝐚}$ is a $q\times 1$ vector, $\mathbf{𝐛}$ is a $p\times 1$ vector, $A$ and $B$ are $q\times p$ matrices of constants, and $0_{p\times q}$ is a $p\times q$ matrix of zeroes.

Definition for complex random vectors

If $\mathbf{𝐙}$ and $\mathbf{𝐖}$ are complex random vectors, the definition of the cross-covariance matrix is slightly changed. Transposition is replaced by Hermitian transposition:

${\mathrm{K}}_{\mathbf{𝐙}\mathbf{𝐖}}=\mathrm{cov}(\mathbf{𝐙},\mathbf{𝐖})\stackrel{\mathrm{d}\mathrm{e}\mathrm{f}}{=}\mathrm{E}[(\mathbf{𝐙}-{\mathit{\mu }}_{\mathbf{𝐙}})(\mathbf{𝐖}-{\mathit{\mu }}_{\mathbf{𝐖}}{)}^{\mathrm{H}}]$

For complex random vectors, another matrix called the pseudo-cross-covariance matrix is defined as follows:

${\mathrm{J}}_{\mathbf{𝐙}\mathbf{𝐖}}=\mathrm{cov}(\mathbf{𝐙},\overline{\mathbf{𝐖}})\stackrel{\mathrm{d}\mathrm{e}\mathrm{f}}{=}\mathrm{E}[(\mathbf{𝐙}-{\mathit{\mu }}_{\mathbf{𝐙}})(\mathbf{𝐖}-{\mathit{\mu }}_{\mathbf{𝐖}}{)}^{\mathrm{T}}]$

Uncorrelatedness

Two random vectors $\mathbf{𝐗}$ and $\mathbf{𝐘}$ are called uncorrelated if their cross-covariance matrix ${\mathrm{K}}_{\mathbf{𝐗}\mathbf{𝐘}}$ matrix is a zero matrix.

Complex random vectors $\mathbf{𝐙}$ and $\mathbf{𝐖}$ are called uncorrelated if their covariance matrix and pseudo-covariance matrix is zero, i.e. if ${\mathrm{K}}_{\mathbf{𝐙}\mathbf{𝐖}}={\mathrm{J}}_{\mathbf{𝐙}\mathbf{𝐖}}=0$.