[Athreya] 6.3. Kolmogorov's consistency theorem

$\def\NN{\mathbb{N}}$$\def\RR{\mathbb{R}}$$\def\eps{\epsilon}$$\def\calR{\mathcal{R}}$$\def\calC{\mathcal{C}}$$\def\calF{\mathcal{F}}$$\def\calB{\mathcal{B}}$$\def\calS{\mathcal{S}}$$\def\calA{\mathcal{A}}$$\newcommand{\inner}[2]{\langle#1, #2\rangle}$$\newcommand{\abs}[1]{\left\vert#1\right\vert}$$\newcommand{\norm}[1]{\left\Vert#1\right\Vert}$$\newcommand{\paren}[1]{\left(#1\right)}$$\newcommand{\sqbracket}[1]{\left[#1\right]}$$\def\var{\text{Var}}$$\def\cov{\text{Cov}}$$\newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}}$$\newcommand{\doublepd}[3]{\frac{\partial^2 #1}{\partial #2 \partial #3}}$

 

Definition. A (real-valued) stochastic process with an index set $A$ is a family $\{X_\alpha\}_{\alpha \in A}$ of random variables defined on a probability space $(\Omega, \calF, P)$.

 

Definition. Let $\{X_\alpha\}_{\alpha \in A}$ be a stochastic process. For $(\alpha_1, \ldots, \alpha_k) \in A^k$, let $\mu_{(\alpha_1, \ldots, \alpha_k)}$ denote the probability distribution of $(X_{\alpha_1}, \ldots, X_{\alpha_k})$ on $(\RR^k, \calB(\RR^k))$. Then the family $\{\mu_{(\alpha_1, \ldots, \alpha_k)}\}_{(\alpha_1, \ldots, \alpha_k) \in A^k, 1 \le k < \infty}$ is called the family of finite-dimensional distributions associated with $\{X_\alpha\}_{\alpha \in A}$.

 

One can verify that the family of finite-dimensional distributions satisfies the following consistency conditions: for any $k \ge 2, (\alpha_1, \ldots, \alpha_k) \in A^k, B_1, \ldots, B_k \in \calB(\RR)$,

(C1) $\mu_{(\alpha_1, \ldots, \alpha_k)}(B_1 \times \ldots \times B_{k-1} \times \RR) = \mu_{(\alpha_1, \ldots, \alpha_{k-1})}(B_1 \times \ldots \times B_{k-1})$;

(C2) For any permutation $(i_1, \ldots, i_k)$ of $(1, \ldots, k)$,

$$\mu_{(\alpha_{i_1}, \ldots, \alpha_{i_k})}(B_{i_1} \times \ldots \times B_{i_k}) = \mu_{(\alpha_1, \ldots, \alpha_k)} (B_1 \times \ldots \times B_k).$$

 

Theorem. (Kolmogorov's consistency theorem.) Let $A$ be a nonempty set and

$$Q_A := \{\nu_{(\alpha_1, \ldots, \alpha_k)}\}_{(\alpha_1, \ldots, \alpha_k) \in A^k, 1 \le k < \infty}$$

be a family of probability distributions such that

(a) $\nu_{(\alpha_1, \ldots, \alpha_k)}$ is a probability distribution on $(\RR^k, \calB(\RR^k))$;

(b) C1 and C2 hold, i.e., for any $k \ge 2, (\alpha_1, \ldots, \alpha_k) \in A^k, B_1, \ldots, B_k \in \calB(\RR)$,

(C1) $\nu_{(\alpha_1, \ldots, \alpha_k)}(B_1 \times \ldots \times B_{k-1} \times \RR) = \nu_{(\alpha_1, \ldots, \alpha_{k-1})}(B_1 \times \ldots \times B_{k-1})$;

(C2) For any permutation $(i_1, \ldots, i_k)$ of $(1, \ldots, k)$,

$$\nu_{(\alpha_{i_1}, \ldots, \alpha_{i_k})}(B_{i_1} \times \ldots \times B_{i_k}) = \nu_{(\alpha_1, \ldots, \alpha_k)} (B_1 \times \ldots \times B_k).$$

Then there exists a stochastic process $X_A := \{X_\alpha\}_{\alpha \in A}$ on a probability space $(\Omega, \calF, P)$ such that $Q_A$ is the family of finite-dimensional distributions associated with $X_A$.

 

Remark. The above theorem says that given such $Q_A$, there exists a real-valued function $A \times \Omega$ such that for each $(\alpha_1, \ldots, \alpha_k) \in A^k$, the function

$$\omega \mapsto (f(\alpha_1, \omega), \ldots, f(\alpha_k, \omega))$$

is a random vector with the probability distribution $\nu_{(\alpha_1, \ldots, \alpha_k)}$.

 

Definition. For a nonempty set $A$, let $\RR^A$ denote the set of all real-valued functions on $A$.

 

Definition. Let $A$ be a nonempty set. A subset $C \subseteq \RR^A$ is called a finite-dimensional cylinder set if there exists a finite subset $\{\alpha_1, \ldots, \alpha_k\} \subseteq A$ and $B \in \calB(\RR^k)$ such that

$$C = \{f \in \RR^A : (f(\alpha_1), \ldots, f(\alpha_k)) \in B\}.$$

The set $B$ is called a base for $C$.

 

Proposition. Let $A$ be a nonempty set, the collection $\calC$ of all finite-dimensional cylinder sets in $\RR^A$ is an algebra; that is, (a) $\RR^A \in C$; (b) $C$ is closed under complements and pairwise unions.

 

Definition. For a nonempty set $A$, the product $\sigma$-algebra on $\RR^A$, denoted $\calR^A, is the $\sigma$-algebra generated by $\calC$.

 

Remark. If $A$ is a finite nonempty set, then $\calC = \calR^A$. If $A = \NN$, then the product $\sigma$-algebra on $\RR^A$ coincides with the Borel $\sigma$-algebra on $\RR^\infty$ under the metric

$$d(x, y) = \sum_{j=1}^\infty \frac{1}{2^j} \paren{\frac{\abs{x_j - y_j}}{1 + \abs{x_j - y_j}}}.$$

 

Definition. For any subset $A_1 \subseteq A$, the projection map $\pi_{A_1} : \RR^A \to \RR^{A_1}$ is defined by

$$f \mapsto (f(\alpha))_{\alpha \in A_1}.$$

In particular, for $\alpha \in A$, the coordinate map $\pi_\alpha : \RR^A \to \RR$ is defined by $f \mapsto f(\alpha)$.

 

Remark. The finite-dimensional cylinder set

$$C = \{f \in \RR^A : (f(\alpha_1), \ldots, f(\alpha_k)) \in B\}$$

is equal to

$$\pi_{(\alpha_1, \ldots, \alpha_k)}^{-1}(B).$$

 

Sketch of Proof of K.C.T. Let $\Omega := \RR^A, \calF := \calR^A$. For any finite-dimensional cylinder set of the form

$$C = \{f \in \RR^A : (f(\alpha_1), \ldots, f(\alpha_k)) \in B\} \in \calC,$$

define

$$P(C) := \nu_{(\alpha_1, \ldots, \alpha_k)} (B).$$

Our claim is that:

(i) $P$ is well-defined on $\calC$;

(ii) $P$ is countably additive on $\calC$.

If we assume the claim, the Caratheodory extension theorem says that there uniquely exists the extension of $P$ to $\calF$ such that $(\Omega, \calF, P)$ is a probability space. Now for each $\alpha \in A$, define $X_\alpha : \Omega \to \RR : \omega \to \pi_\alpha(\omega)$, then the stochastic process $\{X_\alpha\}_{\alpha \in A}$ has $Q_A$ as the family of finite-dimensional distributions. Refer to the textbook for verifying the claim. $\square$

 

Corollary. Let $\{\mu_n\}_{n \ge 1}$ be a sequence of probability measures such that

(a) for each $n \ge 1$, $\mu_n$ is a probability measure on $(\RR^n, \calB(\RR^n))$;

(b) for each $n \ge 1$ and $B \in \calB(\RR^n)$, $\mu_{n+1}(B \times \RR) = \mu_n(B)$.

Then there exists a stochastic process $\{X_n\}_{n \ge 1}$ on a probability space $(\Omega, \calF, P)$ with $\Omega = \RR^\infty, \calF = \calB(\RR^\infty)$ such that for each $n \ge 1$, $\mu_n$ is the probability distribution of $(X_1, \ldots, X_n)$.

 

In fact, we can get a more precise formulation for $\calF$: 

 

Proposition. $\calF$ coincides with the collection of all $G \subseteq \RR^A$ such that $G = \pi_{A_1}^{-1}(B)$ for some countable subset $A_1 \subseteq A$ and $B \in \calB(\RR^\infty)$.

Proof. The RHS is a $\sigma$-algebra containing $\calC$ and contained in $\calF$. $\square$