条件期望函数及其性质

条件期望函数

设 $Y$ 和 $X$ 是两个随机变量，对于任意一个实数 $x \in supp (X)$ 都仅存在一个实数 $E [Y ∣ X = x]$ 与之对应，因此可以构造随机变量 $X$ 的映射 $X \to E [Y ∣ X]$ ，称 $E [Y ∣ X]$ 为条件期望函数 (CEF)。

Theorem

E_{X} [E_{Y} (Y ∣ X)] = E [Y]

Proof
离散型随机变量

\begin{aligned} E_{X} [E_{Y} (Y ∣ X)] & = \sum_{x} E_{Y} (Y ∣ X = x) P (X = x) \\ = \sum_{x} \sum_{y} y \cdot P (Y = y ∣ X = x) P (X = x) \\ = \sum_{x} \sum_{y} y \cdot P (X = x ∣ Y = y) P (Y = y) \\ = \sum_{x} P (X = x ∣ Y = y) \sum_{y} y \cdot P (Y = y) \\ = \sum_{y} y \cdot P (Y = y) \\ = E [Y] \end{aligned}

连续型随机变量

\begin{aligned} E_{X} [E_{Y} (Y ∣ X)] & = \int_{x} E_{Y} (Y ∣ X) f_{X} (x) d x \\ = \int_{x} \int_{y} y \cdot f_{Y ∣ X} (y ∣ x) d y f_{X} (x) d x \\ = \int_{x} \int_{y} y \cdot f_{X ∣ Y} (x ∣ y) f_{Y} (y) d y d x \\ = \int_{x} f_{X ∣ Y} (x ∣ y) d x \int_{y} y \cdot f_{Y} (y) d y \\ = \int_{y} y \cdot f_{Y} (y) d y \\ = E [Y] \end{aligned}

推论：将 $E (Y | X)$ 整体视为一个随机变量

E (Y | X) = E_{Z} [E_{Y} (Y | X, Z) ∣ X]

Theorem Let $g (X)$ be any function of $X$ , then

E [Y | X] = \underset{g}{\arg min} E [(Y - g (X))^{2}]

Proof

\begin{aligned} E [(Y - g (X))^{2}] = & E [(Y - E [Y | X] + E [Y | X] - g (X))^{2}] \\ = & E [(Y - E [Y | X])^{2}] + E [(E [Y | X] - g (X))^{2}] + \\ + 2 E [(Y - E [Y | X]) (E [Y | X] - g (X))] \end{aligned}

the first term

E [(Y - E [Y | X])^{2}] = E {(Y - E [Y | X])^{2} | X} = E {V a r [Y | X]} \geq 0

the last term

\begin{aligned} 2 E [(Y - E [Y | X]) (E [Y | X] - g (X))] \\ = & 2 E [E {(Y - E [Y | X]) (E [Y | X] - g (X)) | X}] \\ = & 2 E [(E [Y | X] - g (X)) E {Y - E [Y | X] | X}] \\ = & 2 E [(E [Y | X] - g (X)) {E [Y | X] - E [Y | X]}] \\ = & 0 \end{aligned}

therefore, take

g (X) = E [Y | X]

因此，条件期望函数 $E [Y ∣ X]$ 是给定 $X$ 对 $Y$ 做出的最小平方预测。