$\def\NN{\mathbb{N}}$$\def\RR{\mathbb{R}}$$\def\eps{\epsilon}$$\def\calP{\mathcal{P}}$$\def\calL{\mathcal{L}}$$\def\calR{\mathcal{R}}$$\def\calC{\mathcal{C}}$$\def\calF{\mathcal{F}}$$\def\calB{\mathcal{B}}$$\def\calS{\mathcal{S}}$$\def\calA{\mathcal{A}}$$\def\calG{\mathcal{G}}$$\newcommand{\inner}[2]{\langle#1, #2\rangle}$$\newcommand{\abs}[1]{\left\vert#1\right\vert}$$\newcommand{\norm}[1]{\left\Vert#1\right\Vert}$$\newcommand{\paren}[1]{\left(#1\right)}$$\newcommand{\sqbracket}[1]{\left[#1\right]}$$\def\var{\text{Var}}$$\def\cov{\text{Cov}}$$\newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}}$$\newcommand{\doublepd}[3]{\frac{\partial^2 #1}{\partial #2 \partial #3}}$
Definition. Let $(\Omega, \calF, P)$ be a probability space and $\{B_1, \ldots, B_n\}$ be a finite collection of events. $B_1, \ldots, B_n$ are called independent if
$$P \paren{\bigcap_{j=1}^k B_{i_j}} = \prod_{j=1}^k B_{i_j}$$
for any subset $\{i_1, \ldots, i_k\} \subseteq \{1, \ldots, n\}$. Generally, a collection of events $\{B_\alpha\}_{\alpha \in A}$ is called independent if every finite subcollection of $\{B_\alpha\}_{\alpha \in A}$ is independent. $\{B_\alpha\}_{\alpha \in A}$ is called pairwise independent if $P(B_\alpha \cap B_\beta) = P(B_\alpha)P(B_\beta)$ for all $\alpha \ne \beta$.
Note that pairwise independence does not imply independence.
Definition. A collection of events $\{B_\alpha\}_{\alpha \in A}$ is called independent if every finite subcollection of $\{B_\alpha\}_{\alpha \in A}$ is independent.
Definition. Let $A$ be a nonempty index set. For each $\alpha \in A$, let $\calG_\alpha \subseteq \calF$ be a collection of events. The family $\{\calG_\alpha\}_{\alpha \in A}$ is called independent if for every choice $B_\alpha \in \calG_\alpha$, the collection of events $\{B_\alpha\}_{\alpha \in A}$ is independent.
Definition. A collection $\{X_\alpha\}_{\alpha \in A}$ of random variables is independent if the family $\{\sigma(X_\alpha)\}_{\alpha \in A}$ of $\sigma$-algebras is independent. Here, $\sigma(X)$ is the $\sigma$-algebra generated by $X$, i.e.,
$$\sigma(X) = \{X^{-1}(B) \in \calF : B \in \calB(\RR)\}.$$
$\{X_\alpha\}_{\alpha \in A}$ is independent if and only if for any finite subset $\{\alpha_1, \ldots, \alpha_k\} \subseteq A$ and $B_i \in \calB(\RR) \ (i = 1, \ldots, k)$,
$$P(X_{\alpha_i} \in B_{i}, \ i = 1, \ldots, k) = \prod_{i=1}^k P(X_{\alpha_i} \in B_i).$$
Proposition. For each $\alpha \in A$, let $\calG_\alpha \subseteq \calF$ be a collection of events which is a $\pi$-system. Suppose $\{\calG_\alpha\}_{\alpha \in A}$ is independent. Then $\sigma(\calG_\alpha)\}_{\alpha \in A}$ is also independent.
Proof. Fix $\{\alpha_1, \ldots, \alpha_k\} \subseteq A$ and $B_i \in \sigma(\calG_{\alpha_i}) \ (i = 1, \ldots, k)$. Let
$$\calL := \left\{B \in \sigma(\calG_{\alpha_k}) : P(B_1 \cap \ldots \cap B_{k-1} \cap B) = P(B_1 \cap \ldots \cap B_{k-1}) P (B)\right\}$$.
Then one can easily check that $\calL$ is a $\lambda$-system that contains $\calG_{\alpha}$. By the $\pi$-$\lambda$ theorem, $\calL$ contains $\sigma(\calG_{\alpha_k})$ so $B_k \in \calL$. Repeating this $(k-1)$-times, we have that
$$P(B_1 \cap \ldots \cap B_k) = \prod_{i=1}^k P(B_i).$$
$\square$
Corollary. A collection $\{X_\alpha\}_{\alpha \in A}$ of random variables is independent if and only if for every finite subset $\{\alpha_1, \ldots, \alpha_k\} \subseteq A$, the joint cdf $F_{(\alpha_1, \ldots, \alpha_k)}$ is a product of marginal cdf $F_{\alpha_i}$'s, i.e.,
$$F_{(\alpha_1, \ldots, \alpha_k)}(x_1, \ldots, x_k) = \prod_{i=1}^k F_{\alpha_i}(x_i)$$
for all $x_1, \ldots, x_k \in \RR$.
Proof. $(\Rightarrow)$ is easy. For each $\alpha \in A$, let $\calG_\alpha := \{X_\alpha^{-1}((\infty, x]) : x \in \RR\}$.
Remark. Suppose that the probability distribution of $(X_1, \ldots, X_k)$ is absolutely continuous w.r.t. the Lebesgue measure on $\RR^k$. Then $\{X_1, \ldots, X_k\}$ is independent if and only if the joint density $f_{1, \ldots, k}$ of $(X_1, \ldots, X_k)$ is a product of marginal density $f_i$'s, i.e.,
$$f_{1, \ldots, k}(x_1, \ldots, x_k) = \prod_{i=1}^k f_{i}(x_i)$$
for all $x_1, \ldots, x_k \in \RR$.
Proposition. Let $X_1, \ldots, X_k$ be random variables.
(a) $\{X_1, \ldots, X_k\}$ is independent if and only if
$$E \prod_{i=1}^k h_i(X_i) = \prod_{i=1}^k Eh_i(X_i)$$
for all bounded Borel functions $h_1, \ldots, h_k : \RR \to \RR$.
(b) Suppose $X_1$ and $X_2$ are independent and $E\abs{X_1}, E\abs{X_2} < \infty$. Then $E\abs{X_1X_2} < \infty$ and $EX_1X_2 = EX_1EX_2$.
Proof.
(a) $(\Rightarrow)$ can be shown by setting $h_i$ to be arbitrary Borel simple functions. Now suppose $\{X_1, \ldots, X_k\}$ is independent. Then the desired equality holds for simple function $h_i$'s; so for all Borel functions, by DCT.
(b) By Fubini's theorem. $\square$
Definition. A collection $\calP$ of subsets of a $\Omega$ is called a $\pi$-system if for any $A, B \in \calP$, $A \cap B \in \calP$ holds.
Definition. A collection $\calL$ of subsets of a $\Omega$ is called a $\lambda$-system if (a) $\Omega \in \calL$; (b) for any $A \in \calL$, $A^C \in \calL$; (c) for any disjoint sequence of $(A_n)_{n=1}^\infty$ in $\calL$, $\bigcup_{n} A_n \in \calL$.
Remark. For a $\lambda$-system $\calL$ and $A, B \in \calL$,
$$A \setminus B = A \setminus (A \cap B) = (A^C \cup (A \cap B))^C \in \calL.$$
Note that intersection of $\lambda$-systems is also a $\lambda$-system. Therefore, for any collection $\calC$ of subsets of $\Omega$, there exists the smallest $\lambda$-system containing $\calC$, denoted $\lambda(\calC)$, which is defined as the intersection of all $\lambda$-systems containing $\calC$. $\lambda(\calC)$ is called the $\lambda$-system generated by $\calC$.
Proposition. A collection $\calC$ of subsets of $\Omega$ is a $\sigma$-algebra if and only if it is both $\pi$-system and $\lambda$-system.
Proof. $(\Rightarrow)$ is obvious. Now let $\calC$ be both $\pi$-system and $\lambda$-system. Let $(B_n)_{n=1}^\infty$ be a sequence in $\calC$. For each $n$, let $A_n := B_n \setminus B_{n-1}$, where $B_0 = \varnothing$. Now $\bigcup_n A_n = \bigcup_n B_n$, so it suffices to show $\bigcup_n A_n \in \calC$. By assumption, $A_n = B_n \cap B_{n-1}^C$ is an element of $\calC$ and $A_n$'s are disjoint. Therefore, $\bigcup_n A_n \in \calC$. $\square$
Lemma. Let $\calL$ be a $\lambda$-system in $\Omega$. Fix $A \in \calL$ Then the collection
$$\calL_A := \{B \subseteq \Omega : A \cap B \in \calL\}$$
is also a $\lambda$-system.
Proof.
(a) $A \cap \Omega = A \in \calL$ so $\Omega \in \calL_A$.
(b) Let $B \in \calL_A$. Then $A \cap B^C = A \setminus B = A \setminus (A \cap B) \in \calL$ so $B^C \in \calL_A$.
(c) Let $(A_n)_{n=1}^\infty$ be a disjoint sequence in $\calL$. Then $A \cap \bigcup_n A_n = \bigcup_n (A \cap A_n) \in \calL$ so $\bigcup_n A_n \in \calL_A$.
Proposition. For a $\pi$-system $\calP$ in $\Omega$, $\lambda(\calP)$ is a $\sigma$-algebra.
Proof. The proof consists of three steps.
Step 1. We claim that $\lambda(\calP) \subseteq \lambda(\calP)_A$ for each $A \in \calP$. It suffices to show that $\calP \subseteq \lambda(\calP)_A$. For any $B \in \calP$, $A \cap B \in \calP \subseteq \lambda(\calP)$ so $\calP \subseteq \lambda(\calP)_A$. This implies that $\calP \subseteq \lambda(\calP)_A$.
Step 2. We claim that $\lambda(\calP) \subseteq \lambda(\calP)_A$ for each $A \in \lambda(\calP)$. It suffices to show $\calP \subseteq \lambda(\calP)_A$. For any $B \in \calP$, $A \in \lambda(\calP) \subseteq \lambda(\calP)_B$ by Step 1. Thus $A \cap B \in \lambda(\calP)$, so $B \in \lambda(\calP)_A$. This implies that $\calP \subseteq \lambda(\calP)_A$.
Step 3. Now for any $A, B \in \lambda(\calP)$, $B \in \lambda(\calP)_A$ by Step 2 so $A \cap B \in\lambda(\calP)$. This implies that $\lambda(\calP)$ is a $\pi$-system. Therefore, $\lambda(\calP)$ is a $\sigma$-algebra.
Corollary. ($\pi$-$\lambda$ theorem.) For a $\pi$-system $\calP$ in $\Omega$, $\lambda(\calP) = \sigma(\calP)$.
'수학 > 확률론' 카테고리의 다른 글
[Athreya] 8.1. Weak Laws of Large Numbers (0) | 2022.08.16 |
---|---|
[Athreya] 7.2. Borel-Cantelli Lemmas, Tail $\sigma$-algebras, and Kolmogorov's 0-1 Law (0) | 2022.08.10 |
[Athreya] 6.3. Kolmogorov's consistency theorem (0) | 2022.08.06 |
[Athreya] 6.2. Random Variables and Random Vectors (0) | 2022.08.04 |