Pseudolikelihood

In statistical theory, a pseudolikelihood is an approximation to the joint probability distribution of a collection of random variables. The practical use of this is that it can provide an approximation to the likelihood function of a set of observed data which may either provide a computationally simpler problem for estimation, or may provide a way of obtaining explicit estimates of model parameters.

The pseudolikelihood approach was introduced by Julian Besag in the context of analysing data having spatial dependence.

Definition

Given a set of random variables $X=X_1,X_2,\dots ,X_n$ the pseudolikelihood of $X=x=(x_1,x_2,\dots ,x_n)$ is

$L(\theta ):=\prod _i{\mathrm{P}\mathrm{r}}_{\theta }(X_i=x_i\mid X_j=x_j\text{ for }j\ne i)=\prod _i{\mathrm{P}\mathrm{r}}_{\theta }(X_i=x_i\mid X_{-i}=x_{-i})$

in discrete case and

$L(\theta ):=\prod _i{p}_{\theta }(x_i\mid x_j\text{ for }j\ne i)=\prod _i{p}_{\theta }(x_i\mid x_{-i})=\prod _i{p}_{\theta }(x_i\mid x_1,\dots ,{\hat{x}}_i,\dots ,x_n)$

in continuous one. Here $X$ is a vector of variables, $x$ is a vector of values, $p_{\theta }(\cdot \mid \cdot )$ is conditional density and $\theta =({\theta }_1,\dots ,{\theta }_p)$ is the vector of parameters we are to estimate. The expression $X=x$ above means that each variable $X_i$ in the vector $X$ has a corresponding value $x_i$ in the vector $x$ and $x_{-i}=(x_1,\dots ,{\hat{x}}_i,\dots ,x_n)$ means that the coordinate $x_i$ has been omitted. The expression ${\mathrm{P}\mathrm{r}}_{\theta }(X=x)$ is the probability that the vector of variables $X$ has values equal to the vector $x$. This probability of course depends on the unknown parameter $\theta$. Because situations can often be described using state variables ranging over a set of possible values, the expression ${\mathrm{P}\mathrm{r}}_{\theta }(X=x)$ can therefore represent the probability of a certain state among all possible states allowed by the state variables.

The pseudo-log-likelihood is a similar measure derived from the above expression, namely (in discrete case)

$l(\theta ):=\log L(\theta )=\sum _i\log {\mathrm{P}\mathrm{r}}_{\theta }(X_i=x_i\mid X_j=x_j\text{ for }j\ne i).$

One use of the pseudolikelihood measure is as an approximation for inference about a Markov or Bayesian network, as the pseudolikelihood of an assignment to $X_i$ may often be computed more efficiently than the likelihood, particularly when the latter may require marginalization over a large number of variables.

Properties

Use of the pseudolikelihood in place of the true likelihood function in a maximum likelihood analysis can lead to good estimates, but a straightforward application of the usual likelihood techniques to derive information about estimation uncertainty, or for significance testing, would in general be incorrect.