Chapter 4

Dependent Random Variables

Conditioning

One of the central concepts in probability theory is the notion of conditional probabiliy and conditional expectation. Suppose that we have a probaility space (Ω, F, P) consisting of a space Ω, a σ-field F of subsets of Ω and a probability measure on the σ-field F. If we have a set A ∈ F of positive measure then conditioning with respect to A means we restrict ourselves to the set A. Ω gets replaced by A. The σ-field F by the σ-field F_{A} of subsets of A that are in F. For B ⊂ A we define

P_{A}(B) = \frac{P (B)}{P (A)}

We could achieve the same thing by defining for arbitrary B ∈ F

P_{A}(B) = \frac{P (A ∩ B)}{P (A)}

in which case P_{B}(·) is a measure defined on F as well but one that is concentrated on A and assigning 0 probability to A^{c}. The definition of conditional probability is

P (B|A) = \frac{P (A ∩ B)}{P (A)}

Similarly the definition of conditional expectation of an integrable function f(ω) given a set A ∈ F of positive probability is defined to be

E\{f|A\} = \frac{∫_{A} f(ω)dP}{P (A)}

In particular if we take f = χ_{B} for some B ∈ F we recover the definition of conditional probability. In general if we know P (B|A) and P (A) we can recover P (A∩B) = P (A)P (B|A) but we cannot recover P (B). But if we know P (B|A) as well as P (B|A^{c}) along with P (A) and P (A^{c}) = 1 − P (A) then

P (B) = P (A ∩ B) + P (A^{c} ∩ B) = P (A)P (B|A) + P (A^{c})P (B|A^{c})

More generally if P is a partition of Ω into a finite or even a countable number of disjoint measurable sets A_{1}, ⋯, A_{j}, ⋯

P (B) = ∑_{j} P (A_{j})P (B|A_{j})

If ξ is a random variable taking distinct values {a_{j}} on {A_{j}} then

P (B|ξ = a_{j}) = P (B|A_{j})

or more generally

P (B|ξ = a) = \frac{P (B ∩ ξ = a)}{P (ξ = a)}

provided P (ξ = a) > 0. One of our goals is to seek a definition that makes sense when P (ξ = a) = 0. This involves dividing 0 by 0 and should involve differentiation of some kind. In the countable case we may think of P (B|ξ = a_{j}) as a function f_{B}(ξ) which is equal to P (B|A_{j}) on ξ = a_{j}. We can rewrite our definition of

f_{B}(a_{j}) = P (B|ξ = a_{j})

as

∫_{ξ=a_{j}} f_{B}(ξ)dP = P (B ∩ ξ = a_{j}) for each j

or summing over any arbitrary collection of j’s

∫_{ξ∈E} f_{B}(ξ)dP = P (B ∩ {ξ ∈ E})

Sets of the form ξ ∈ E form a sub σ-field Σ ⊂ F and we can rewrite the definition as

∫_{A} f_{B}(ξ)dP = P (B ∩ A)

for all A ∈ Σ. Of course in this case A ∈ Σ if and only if A is a union of the atoms ξ = a of the partition over a finite or countable subcollection of the possible values of a. Similar considerations apply to the conditional expectation of a random variable G given ξ. The equation becomes

∫_{A} g(ξ)dP = ∫_{A} G(ω)dP

or we can rewrite this as

∫_{A} g(ω)dP = ∫_{A} G(ω)dP

for all A ∈ Σ and instead of demanding that g be a function of ξ we demand that g be Σ measurable which is the same thing. Now the random variable ξ is out of the picture and rightly so. What is important is the information we have if we know ξ and that is the same if we replace ξ by a one-to-one function of itself. The σ-field Σ abstracts that information nicely. So it turns out that the proper notion of conditioning involves a sub σ-field Σ ⊂ F. If G is an integrable function and Σ ⊂ F is given we will seek another integrable function g that is Σ measurable and satisfies

∫_{A} g(ω)dP = ∫_{A} G(ω)dP

for all A ∈ Σ. We will prove existence and uniqueness of such a g and call it the conditional expectaion of G given Σ and denote it by g = E[G|Σ].

The way to prove the above result will take us on a detour. A signed measure on a measurable space (Ω, F) is a set function λ(.) defined for A ∈ F which is countably additive but not necessarily nonnegative. Countable addivity is again in any of the folowing two equivalent senses.

λ(∪A_{n}) = ∑ λ(A_{n})

for any countable collection of disjoint sets in F, or

lim_{n→∞} λ(A_{n}) = λ(A)

whenver A_{n} ↓ A or A_{n} ↑ A.

Examples of such λ can be constructed by taking the difference μ_{1} − μ_{2} of two nonnegative measures μ_{1} and μ_{2}.

Definition 4.1. A set A ∈ F is totally positive (totally negative) for λ if for every subset B ∈ F with B ⊂ A λ(B) ≥ 0. (≤ 0)

Remark: A measurable subset of a totally positive set is totally positive. Any countable union of totally positive subsets is again totally positive. Lemma 4.7. If λ is a countably additive signed measure on (Ω, F),

sup_{A∈F} |λ(A)| < ∞

Proof: The key idea in the proof is that, since λ(Ω) is a finite number, if λ(A) is large so is λ(A^{c}) with an opposite sign. In fact, it is not hard to see that ||λ(A)| − |λ(A^{c})|| ≤ |λ(Ω)| for all A ∈ F. Another fact is that if sup_{B⊂A} |λ(B)| and sup_{B⊂A^{c}} |λ(B)| are finite, so is sup_{B} |λ(B)|. Now let us complete the proof. Given a subset A ∈ F with sup_{B⊂A} |λ(B)| = ∞, and any positive number N, there is a subset A_{1} ∈ F with A_{1} ⊂ A such that |λ(A_{1})| ≥ N and sup_{B⊂A_{1}} |λ(A)| = ∞. This is obvious because if we pick a set E ⊂ A with |λ(E)| very large so will λ(E^{c}) be. At least one of the two sets E, E^{c} will have the second property and we can call it A_{1}. If we proceed by induction we have a sequence A_{n} that is ↓ and |λ(A_{n})| → ∞ that contradicts countable additivity. Lemma 4.8. Given a subset A ∈ F with λ(A) = ℓ > 0 there is a subset Ā ⊂ A that is totally positive with λ(Ā) ≥ ℓ. Proof: Let us define m = inf_{B⊂A} λ(B). Since the empty set is included, m ≤ 0. If m = 0 then A is totally positive and and we are done. So let us assume that m < 0. By the previous lemma m > −∞. Let us find B_{1} ⊂ A such that λ(B_{1}) ≤ \frac{m}{2}. Then for A_{1} = A − B_{1} we have A_{1} ⊂ A, λ(A_{1}) ≥ ℓ and inf_{B⊂A_{1}} λ(B) ≥ \frac{m}{2}. By induction we can find A_{k} with A ⊃ A_{1} ⊃ ⋯ ⊃ A_{k} ⋯, λ(A_{k}) ≥ ℓ for every k and inf_{B⊂A_{k}} λ(A_{k}) ≥ \frac{m}{2^{k}}. Clearly if we define Ā = ∩A_{k} which is the decreasing limit, Ā works.

Theorem 4.9.(Hahn-Jordan Decomposition). Given a countably additive signed measure λ on (Ω, F) it can be written always as λ = μ^{+} − μ^{−} the difference of two nonnegative measures. Moreover μ^{+} and μ^{−} may be chosen to be orthogonal i.e, there are disjoint sets Ω_{+}, Ω_{−} ∈ F such that μ^{+}(Ω_{−}) = μ^{−}(Ω_{+}) = 0. In fact Ω_{+} and Ω_{−} can be taken to be subsets of Ω that are respectively totally positive and totally negative for λ. μ^{±} then become just the restrictions of λ to Ω_{±}.

Proof: Totally positive sets are closed under countable unions, disjoint or not. Let us define m^{+} = sup_{A} λ(A). If m^{+} = 0 then λ(A) ≤ 0 for all A and we can take Ω_{+} = Φ and Ω_{−} = Ω which works. Assume that m^{+} > 0. There exist sets A_{n} with λ(A) ≥ m^{+} − \frac{1}{n} and therefore totally positive subsets Ā_{n} of A_{n} with λ(Ā_{n}) ≥ m^{+} − \frac{1}{n}. Clearly Ω_{+} = ∪_{n}Ā_{n} is totally positive and λ(Ω_{+}) = m^{+}. It is easy to see that Ω_{−} = Ω − Ω_{+} is totally negative. μ^{±} can be taken to be the restriction of λ to Ω_{±}.

Remark: If λ = μ^{+} − μ^{−} with μ^{+} and μ^{−} orthogonal to each other, then they have to be the restrictions of λ to the totally positive and totally negative sets for λ and such a representation for λ is unique. It is clear that in general the representation is not unique because we can add a common μ to both μ^{+} and μ^{−} and the μ will cancel when we compute λ = μ^{+} − μ^{−}.

Remark: If μ is a nonnegative measure and we define λ by

λ(A) = ∫_{A} f(ω) dμ = ∫ χ_{A}(ω)f(ω) dμ

where f is an integrable function, then λ is a countably additive signed measure and Ω_{+} = {ω : f(ω) > 0} and Ω_{−} = {ω : f(ω) < 0}. If we define f^{±}(ω) as the positive and negative parts of f, then

μ^{±}(A) = ∫_{A} f^{±}(ω) dμ

The signed measure λ that was constructed in the preceeding remark enjoys a very special relationship to μ. For any set A with μ(A) = 0, λ(A) = 0 because the integrand χ_{A}(ω)f(ω) is 0 for μ-almost all ω and for all practical purposes is a function that vanishes identically.

Definition 4.2. A signed measure λ is said to be absolutely continuous with respect to a nonnegative measure μ, λ << μ in symbols, if whenever μ(A) is zero for a set A ∈ F it is also true that λ(A) = 0.

Theorem 4.10. (Radon-Nikodym). If λ << μ then there is an integrable function f(ω) such that

λ(A) = ∫_{A} f(ω) dμ  (4.1)

for all A ∈ F. The function f is uniquely determined almost everywhere and is called the Radon-Nikodym derivative of λ with respect to μ. It is denoted by

f(ω) = \frac{dλ}{dμ}

Proof: The proof depends on the decomposition theorem. We saw that if the relation 4.1 holds, then Ω_{+} = {ω : f(ω) > 0}. If we define λ_{a} = λ − aμ, then λ_{a} is a signed measure for every real number a. Let us define Ω(a) to be the totally positive subset of λ_{a}. These sets are only defined up to sets of measure zero, and we can only handle a countable number of sets of measure 0 at one time. So it is prudent to restrict a to the set Q of rational numbers. Roughly speaking Ω(a) will be the sets f(ω) > a and we will try to construct f from the sets Ω(a) by the definition

f(ω) = [sup a ∈ Q : ω ∈ Ω(a)]

The plan is to check that the function f(ω) defined above works. Since λ_{a} is getting more negative as a increases, Ω(a) is ↓ as a ↑. There is trouble with sets of measure 0 for every comparison between two rationals a_{1} and a_{2}. Collect all such troublesome sets (only a countable number and throw them away). In other words we may assume without loss of generality that Ω(a_{1}) ⊂ Ω(a_{2}) whenever a_{1} > a_{2}. Clearly

{ω : f(ω) > x} = {ω : ω ∈ Ω(y) for some rational y > x}
= ∪_{{y>x \atop y∈Q}} Ω(y)

and this makes f measurable. If A ⊂ ∩_{a}Ω(a) then λ(A) − aμ(A) ≥ 0 for all A. If μ(A) > 0, λ(A) has to be infinite which is not possible. Therefore μ(A) has to be zero and by absolute continuity λ(A) = 0 as well. On the other hand if A∩Ω(a) = Φ for all a, then λ(A)−aμ(A) ≤ 0 for all a and again if μ(A) > 0, λ(A) = −∞ which is not possible either. Therefore μ(A), and by absolute continuity, λ(A) are zero. This proves that f(ω) is finite almost everywhere with rsepect to both λ and μ. Let us take two real numbers a < b and consider E_{a,b} = {ω : a ≤ f(ω) ≤ b}. It is clear that the set E_{a,b} is in Ω(a′) and Ω^{c}(b′) for any a′ < a and b′ > b. Therefore for any set A ⊂ E_{a,b} by letting a′ and b′ tend to a and b

a μ(A) ≤ λ(A) ≤ b μ(A)

Now we are essentially done. Let us take a grid {n h} and consider E_{n} = {ω : nh ≤ f(ω) < (n + 1)h} for −∞ < n < ∞. Then for any A ∈ F and each n,

λ(A ∩ E_{n}) − hμ(A ∩ E_{n}) ≤ n h μ(A ∩ E_{n}) ≤ ∫_{A∩E_{n}} f(ω) dμ
≤ (n + 1) hμ(A ∩ E_{n}) ≤ λ(A ∩ E_{n}) + h μ(A ∩ E_{n})

Summing over n we have

λ(A) − h μ(A) ≤ ∫_{A} f(ω) dμ ≤ λ(A) + h μ(A)

proving the integrability of f and if we let h → 0 establishing

λ(A) = ∫_{A} f(ω) dμ

for all A ∈ F.

Remark: (Uniqueness). If we have two choices of f say f_{1} and f_{2} their difference g = f_{1} − f_{2} satisfies

∫_{A} g(ω) dμ = 0

for all A ∈ F. If we take A_{ϵ} = {ω : g(ω) ≥ ϵ}, then 0 ≥ ϵμ(A_{ϵ}) and this implies μ(A_{ϵ}) = 0 for all ϵ > 0 or g(ω) ≤ 0, almost everywhere with respect to μ. A similar argument establishes g(ω) ≥ 0 almost everywhere with respect to μ. Therefore g = 0 a.e. μ proving uniqueness.

Exercise 4.1. If λ(A) ≥ 0 for all A ∈ F, prove that f(ω) ≥ 0 almost everywhere.

Exercise 4.2. If Ω is a countable set and μ({ω}) > 0 for each single point set prove that any measure λ is absolutely continuous with respect to λ and calculate the Radon-Nikodym derivative.

Exercise 4.3. Let F (x) be a distribution function on the line with F (0) = 0 and F (1) = 1 so that the probability measure α corresponding to it lives on the interval [0, 1]. If F (x) satisfies a Lipschitz condition

|F (x) − F (y)| ≤ A|x − y|

then prove that α << m where m is the Lebesgue measure on [0, 1]. Show also that 0 ≤ \frac{dα}{dm} ≤ A almost surely.

Exercise 4.4. If ν, λ, μ are three nonnegative measures such that ν << λ and λ << μ then show that ν << μ and

\frac{dν}{dμ} = \frac{dν}{dλ} \frac{dλ}{dμ}

a.e.

Exercise 4.5. If λ, μ are nonnegative measures with λ << μ and \frac{dλ}{dμ} = f, then show that g is integrable with respect to λ if and only if g f is integrable with respect to μ and

∫ g(ω) dλ = ∫ g(ω) f(ω) dμ

Exercise 4.6. Given two nonnegative measures λ and μ, λ is said to be uniformly absolutely continuous with respect to μ on F if for any ϵ > 0 there exists a δ > 0 such that for any A ∈ F with μ(A) < δ it is true that λ(A) < ϵ. Use the Radon-Nikodym theorem to show that absolute continuity on a σ-field F implies uniform absolute continuity. If F_{0} is a field that generates the σ-field F show by an example that absolute continuity on F_{0} does not imply absolute continuity on F. Show however that uniform absolute continuity on F_{0} implies uniform absolute continuity and therefore absolute continuity on F.

Exercise 4.7. If F is a distribution function on the line show that it is absolutely continuous with respect to Lebesgue measure on the line, if and only if for any ϵ > 0, there exists a δ > 0 such that for arbitrary finite collection of disjoint intervals I_{j} = [a_{j}, b_{j}] with ∑_{j} |b_{j} − a_{j}| < δ it follows that ∑_{j}[F (b_{j}) − F (a_{j})] ≤ ϵ.

Conditional Probability

In the Radon-Nikodym theorem, if λ << μ are two probability distributions on (Ω, F), we defined the Radon-Nikodym derivative f(ω) = \frac{dλ}{dμ} as an F measurable function such that

λ(A) = ∫_{A} f(ω) dμ for all A ∈ F

If Σ ⊂ F is a sub σ-field, the absolute continuity of λ with respect to μ on Σ is clearly implied by the absolute continuity of λ with respect to μ on F. We can therefore apply the Radon-Nikodym theorem on the measurable space (Ω, Σ), and we will obtain a new Radon-Nikodym derivative

g(ω) = \frac{dλ}{dμ} = \frac{dλ}{dμ}|_{Σ}

such that

λ(A) = ∫_{A} g(ω) dμ for all A ∈ Σ

and g is Σ measurable. Since the old function f(ω) was only F measurable, in general, it cannot be used as the Radon-Nikodym derivative for the sub σ-field Σ. Now if f is an integrable function on (Ω, F, μ) and Σ ⊂ F is a sub σ-field we can define λ on F by

λ(A) = ∫_{A} f(ω) dμ for all A ∈ F

and recalculate the Radon-Nikodym derivative g for Σ and g will be a Σ measurable, integrable function such that

λ(A) = ∫_{A} g(ω) dμ for all A ∈ Σ

In other words g is the perfect candidate for the conditional expectation

g(ω) = E{f(·)|Σ}

We have therefore proved the existence of the conditional expectation. Theorem 4.11. The conditional expectation as mapping of f → g has the following properties. (i). If g = E{f|Σ} then E[g] = E[f]. E[1|Σ] = 1 a.e. (ii). If f is nonnegative then g = E{f|Σ} is almost surely nonnegative. (iii). The map is linear. If a_{1}, a_{2} are constants

E{a_{1}f_{1} + a_{2}f_{2}|Σ} = a_{1}E{f_{1}|Σ} + a_{2}E{f_{2}|Σ} a.e.

(iv). If g = E{f|Σ}, then

∫ |g(ω)| dμ ≤ ∫ |f(ω)| dμ

(v). If h is a bounded Σ measurable function then

E{f h|Σ} = h E{f|Σ} a.e.

(vi). If Σ_{2} ⊂ Σ_{1} ⊂ F, then

E{f|Σ_{2}} = E{E{f|Σ_{1}}|Σ_{2}}

(vii). If ϕ(x) is a convex function of x, and g = E{f|Σ} then

E{ϕ(f(ω))|Σ} ≥ ϕ(g(ω)) a.e.

and if we take expectations

E[ϕ(f)] ≥ E[ϕ(g)].

Proof: (i), (ii) and (iii) are obvious. For (iv) we note that if dλ = f dμ

∫ |f| dμ = sup_{A∈F} λ(A) − inf_{A∈F} λ(A)

and if we replace F by a sub σ-field Σ the right hand side is decreased.

(v) is ovious if h is the indicator function of a set A in Σ. To go from indicator functions to simple functions to bounded measurable functions is routine. (vi) is an easy consequence of the definition. To see (vii) we note that if f_{1} ≥ f_{2} then E{f_{1}|Σ} ≥ E{f_{2}|Σ} a.e. and consequently E{ max(f_{1}, f_{2})|Σ} ≥ max(g_{1}, g_{2}) a.e. where g_{i} = E{f_{i}|Σ} for i = 1, 2. Since we can represent any convex function ϕ as ϕ(x) = sup_{a}[ax−ψ(a)], limiting ourselves to rational a, we have only a countable set of functions to deal with, and

E{ϕ(f)|Σ} = E{ sup_{a}[af − ψ(a)]|Σ}
≥ sup_{a} [a E{f|Σ} − ψ(a)]
= sup_{a} [a g − ψ(a)]
= ϕ(g)

a.e. and after taking expectations

E[ϕ(f)] ≥ E[ϕ(g)]

In a somewhat more familiar context if μ = λ_{1} × λ_{2} is a product measure on (Ω, F) = (Ω_{1} ×Ω_{2}, F_{1} ×F_{2}) and we take Σ = {A×Ω_{2} : A ∈ F_{1}} then for any function f(ω) = f(ω_{1}, ω_{2}), E[f(·)|Σ] = g(ω) where g(ω) = g(ω_{1}) is given by

g(ω_{1}) = ∫_{Ω_{2}} f(ω_{1}, ω_{2}) dλ_{2}

so that the conditional expectation is just integrating the unwanted variable ω_{2}. We can go one step more. If ϕ(x, y) is the joint density on R^{2} of two random variables X, Y (with respect to the Lebesgue measure on R^{2}), and ψ(x) is the marginal density of X given by

ψ(x) = ∫_{−∞}^{∞} ϕ(x, y) dy

then for any integrable function f(x, y)

E[f(X, Y)|X] = E[f(·, ·)|Σ] = \frac{∫_{−∞}^{∞} f(x, y) ϕ(x, y) dy}{ψ(x)}

where Σ is the σ-field of vertical strips A×(−∞, ∞) with a measurable horizontal base A.

Exercise 4.8. If f is already Σ measurable then E[f|Σ] = f. This suggests that the map f → g = E[f|Σ] is some sort of a projection. In fact if we consider the Hilbert space H = L_{2}[Ω, F, μ] of all F measurable square integrable functions with an inner product

< f, g >_{μ}= ∫ f g dμ

then

H_{0} = L_{2}[Ω, Σ, μ] ⊂ H = L_{2}[Ω, F, μ]

and f → E[f|Σ] is seen to be the same as the orthogonal projection from H onto H_{0}. Prove it.

Conditional expectation is then the best nonlinear predictor if the loss function is the expected (mean) square error.

We now turn our attention to conditional probability. If we take f = χ_{B}(ω) then E[f|Σ] = P (ω, B) is called the conditional probability of B given Σ. It is characterized by the property that it is Σ measurable as a function of ω and for any A ∈ Σ

μ(A ∩ B) = ∫_{A} P (ω, B) dμ

Theorem 4.12. P (·, ·) has the following properties. (i). P (ω, Ω) = 1, P (ω, Φ) = 0 a.e. (ii). For any B ∈ F, 0 ≤ P (ω, B) ≤ 1 a.e. (iii). For any countable collection {B_{j}} of disjoint sets in F,

P (ω, ∪_{j}B_{j}) = ∑_{j} P (ω, B_{j}) a.e.

(iv). If B ∈ Σ, P (ω, B) = χ_{B}(ω) a.e.

Proof: All are easy consequences of properties of conditional expectations. Property (iii) perhaps needs an explanation. If E[|f_{n}−f|] → 0 by the properties of conditional expectation E[|E{f_{n}|Σ} − E{f|Σ}| → 0. Property (iii) is an easy consequence of this.

The problem with the above theorem is that every property is valid only almost everywhere. There are exceptional sets of measure zero for each case. While each null set or a countable number of them can be ignored we have an uncountable number of null sets and we would like a single null set outside which all the properties hold. This means constructing a good version of the conditional probability. It may not be always possible. If possible, such a version is called a regular conditional probability. The existence of such a regular version depends on the space (Ω, F) and the sub σ-field Σ being nice. If Ω is a complete separable metric space and F are its Borel stes, and if Σ is any countably generated sub σ-field of F, then it is nice enough. We will prove it in the special case when Ω = [0, 1] is the unit interval and F are the Borel subsets of [0, 1]. Σ can be any countably generated sub σ-field of F.

The trick is not to be too ambitious in the first place and try to construct the conditional expectations

Q(ω, B) = E{χ_{B}(ω)|Σ}

only for sets B given by B = (−∞, x) for rational x. We call our conditional expectation, which is in fact a conditional probability, by F (ω, x). By the properties of conditional expectations for any pair of rationals x < y, there is a null set E_{x,y}, such that for ω ∉ E_{x,y}

F (ω, x) ≤ F (ω, y)

Moreover for any rational x < 0, there is a null set N_{x} outside which F (ω, x) = 0 and similar null sets N_{x} for x > 1, ouside which F (ω, x) = 1. If we collect all these null sets, of which there are only countably many, and take their union, we get a null set N ∈ Σ such that for ω ∉ N, we have have a family F (ω, x) defined for rational x that satisfies

F (ω, x) ≤ F (ω, y) if x < y are rational
F (ω, x) = 0 for rational x < 0 and F (ω, x)
= 1 for rational x > 1
P (A ∩ [0, x]) = ∫_{A} F (ω, x) dP for all A ∈ Σ

For ω ∉ N and real y we can define

G(ω, y) = lim_{x↓y ; x rational} F (ω, x)

For ω ∉ N, G is a right continuous nondecreasing function (distribution function) with G(y) = 0 for y < 0 and G(y) = 1 for y ≥ 1. There is then a probability measure Q̂(ω, B) on the Borel subsets of [0, 1] such that Q̂(ω, [0, y]) = G(ω, y) for all y. Q̂ is our candidate for regular conditional probability. Clearly Q̂(ω, I) is Σ measurable for all intervals I and by standard arguments will continue to be Σ measurable for all Borel sets B ∈ F. If we check that

P (A ∩ [0, x]) = ∫_{A} G(ω, x) dP for all A ∈ Σ

for all 0 ≤ x ≤ 1 then

P (A ∩ I) = ∫_{A} Q̂(ω, I) dP for all A ∈ Σ

for all intervals I and by standard arguments this will extend to finite disjoint unions of half open intervals that constitute a field and finally to the σ-field F generated by that field. To verify that for all real y,

P (A ∩ [0, y]) = ∫_{A} G(ω, y) dP for all A ∈ Σ

we start from

P (A ∩ [0, x]) = ∫_{A} F (ω, x) dP for all A ∈ Σ

valid for rational x and let x ↓ y through rationals. From the countable additivity of P the left hand side converges to P (A ∩ [0, y]) and by the bounded convergence theorem, the right hand side converges to ∫_{A} G(ω, y) dP and we are done. Finally from the uniqueness of the conditional expectation if A ∈ Σ

Q̂(ω, A) = χ_{A}(ω)

provided ω ∉ N_{A}, which is a null set that depends on A. We can take a countable set Σ_{0} of generators A that forms a field and get a single null set N such that if ω ∉ N

Q̂(ω, A) = χ_{A}(ω)

for all A ∈ Σ_{0}. Since both side are countably additive measures in A and as they agree on Σ_{0} they have to agree on Σ as well.

Exercise 4.9. Disintegration Theorem. Let μ be a probability measure on the plane R^{2} with a marginal distribution α for the first coordinate. In other words if we denote by (x, y) a point in R^{2}, then

∫_{R^{2}} f(x) dμ = ∫_{R} f(x) dα

Show that there exist measures β_{x} depending measurably on x such that β_{x}[{x} × R] = 1 i.e. β_{x} is supported on the vertial line through (x, y) : y ∈ R and μ = ∫_{R} β_{x} dα. The converse is of course easier. Given α and β_{x} we can construct a unique μ such that μ disintegrates as expected.