Probability and Conditional Expectation Fundamentals for the Empirical Sciences

Probability and Conditional Expectations bridges the gap between books on probability theory and statistics by providing the probabilistic concepts estimated and tested in analysis of variance, regression analysis, factor analysis, structural equation modeling, hierarchical linear models and analysis of qualitative data. The authors emphasize the theory of conditional expectations that is also fundamental to conditional independence and conditional distributions.

Probability and Conditional Expectations

• Presents a rigorous and detailed mathematical treatment of probability theory focusing on concepts that are fundamental to understand what we are estimating in applied statistics.
• Explores the basics of random variables along with extensive coverage of measurable functions and integration.
• Extensively treats conditional expectations also with respect to a conditional probability measure and the concept of conditional effect functions, which are crucial in the analysis of causal effects.
• Is illustrated throughout with simple examples, numerous exercises and detailed solutions.
• Provides website links to further resources including videos of courses delivered by the authors as well as R code exercises to help illustrate the theory presented throughout the book.

Rolf Steyer,
Institute of Psychology, University of Jena, Germany

Werner Nagel,
Institute of Mathematics, University of Jena, Germany

Part I Measure-Theoretical Foundations of Probability Theory

1 Measure 3

1.1 Introductory Examples 3

1.2 σ-Algebra and Measurable Space 4

1.2.1 σ-Algebra Generated by a Set System 9

1.2.2 σ-Algebra of Borel Sets on Rn 12

1.2.3 σ-Algebra on a Cartesian Product 13

1.2.4 ∩-Stable Set Systems That Generate a σ-Algebra 15

1.3 Measure and Measure Space 16

1.3.1 σ-Additivity and Related Properties 17

1.3.2 Other Properties 18

1.4 Specific Measures 20

1.4.1 Dirac Measure and Counting Measure 21

1.4.2 Lebesgue Measure 22

1.4.3 Other Examples of a Measure 23

1.4.4 Finite and σ-Finite Measures 23

1.4.5 Product Measure 24

1.5 Continuity of a Measure 25

1.6 Specifying a Measure via a Generating System 27

1.7 σ-Algebra That is Trivial With Respect to a Measure 28

1.8 Proofs 28

1.9 Exercises 31

2 Measurable Mapping 41

2.1 Image and Inverse Image 41

2.2 Introductory Examples 42

2.2.1 Example 1: Rectangles 42

2.2.2 Example 2: Flipping two Coins 44

2.3 Measurable Mapping 46

2.3.1 Measurable Mapping 46

2.3.2 σ-Algebra Generated by a Mapping 51

2.3.3 Final σ-Algebra 54

2.3.4 Multivariate Mapping 54

2.3.5 Projection Mapping 56

2.3.6 Measurability With Respect to a Mapping 56

2.4 Theorems on Measurable Mappings 58

2.4.1 Measurability of a Composition 59

2.4.2 Theorems on Measurable Functions 61

2.5 Equivalence of Two Mappings With Respect to a Measure 64

2.6 Image Measure 67

2.7 Proofs 70

2.8 Exercises 75

3 Integral 83

3.1 Definition 83

3.1.1 Integral of a Nonnegative Step Function 83

3.1.2 Integral of a Nonnegative Measurable Function 88

3.1.3 Integral of a Measurable Function 93

3.2 Properties 96

3.2.1 Integral of μ-Equivalent Functions 98

3.2.2 Integral With Respect to a Weighted Sum of Measures 100

3.2.3 Integral With Respect to an Image Measure 102

3.2.4 Convergence Theorems 103

3.3 Lebesgue and Riemann Integral 104

3.4 Density 106

3.5 Absolute Continuity and the Radon-Nikodym Theorem 108

3.6 Integral With Respect to a Product Measure 110

3.7 Proofs 111

3.8 Exercises 120

Part II Probability, Random Variable and its Distribution

4 Probability Measure 127

4.1 Probability Measure and Probability Space 127

4.1.1 Definition 127

4.1.2 Formal and Substantive Meaning of Probabilistic Terms 128

4.1.3 Properties of a Probability Measure 128

4.1.4 Examples 130

4.2 Conditional Probability 132

4.2.1 Definition 132

4.2.2 Filtration and Time Order Between Events and Sets of Events 133

4.2.3 Multiplication Rule 135

4.2.4 Examples 136

4.2.5 Theorem of Total Probability 137

4.2.6 Bayes’ Theorem 138

4.2.7 Conditional-Probability Measure 139

4.3 Independence 143

4.3.1 Independence of Events 143

4.3.2 Independence of Set Systems 144

4.4 Conditional Independence Given an Event 145

4.4.1 Conditional Independence of Events Given an Event 145

4.4.2 Conditional Independence of Set Systems Given an Event 146

4.5 Proofs 148

4.6 Exercises 150

5 Random Variable, Distribution, Density, and Distribution Function 155

5.1 Random Variable and its Distribution 155

5.2 Equivalence of Two Random Variables With Respect to a Probability Measure 161

5.2.1 Identical and P-Equivalent Random Variables 161

5.2.2 P-Equivalence, PB-Equivalence, and Absolute Continuity 164

5.3 Multivariate Random Variable 167

5.4 Independence of Random Variables 169

5.5 Probability Function of a Discrete Random Variable 175

5.6 Probability Density With Respect to a Measure 178

5.6.1 General Concepts and Properties 178

5.6.2 Density of a Discrete Random Variable 180

5.6.3 Density of a Bivariate Random Variable 180

5.7 Uni- or Multivariate Real-Valued Random Variable 182

5.7.1 Distribution Function of a Univariate Real-Valued Random Variable 182

5.7.2 Distribution Function of a Multivariate Real-Valued Random Variable 184

5.7.3 Density of a Continuous Univariate Real-Valued Random Variable 185

5.7.4 Density of a Continuous Multivariate Real-Valued Random Variable 187

5.8 Proofs 188

5.9 Exercises 196

6 Expectation, Variance, and Other Moments 199

6.1 Expectation 199

6.1.1 Definition 199

6.1.2 Expectation of a Discrete Random Variable 200

6.1.3 Computing the Expectation Using a Density 202

6.1.4 Transformation Theorem 203

6.1.5 Rules of Computation 206

6.2 Moments, Variance, and Standard Deviation 207

6.3 Proofs 212

6.4 Exercises 213

7 Linear Quasi-Regression, Covariance, and Correlation 217

7.1 Linear Quasi-Regression 217

7.2 Covariance 220

7.3 Correlation 224

7.4 Expectation Vector and Covariance Matrix 227

7.4.1 Random Vector and Random Matrix 227

7.4.2 Expectation of a Random Vector and a Random Matrix 228

7.4.3 Covariance Matrix of two Multivariate Random Variables 229

7.5 Multiple Linear Quasi-Regression 231

7.6 Proofs 233

7.7 Exercises 237

8 Some Distributions 245

8.1 Some Distributions of Discrete Random Variables 245

8.1.1 Discrete Uniform Distribution 245

8.1.2 Bernoulli Distribution 246

8.1.3 Binomial Distribution 247

8.1.4 Poisson Distribution 250

8.1.5 Geometric Distribution 252

8.2 Some Distributions of Continuous Random Variables 254

8.2.1 Continuous Uniform Distribution 254

8.2.2 Normal Distribution 256

8.2.3 Multivariate Normal Distribution 259

8.2.4 Central χ2-Distribution 262

8.2.5 Central t -Distribution 264

8.2.6 Central F-Distribution 266

8.3 Proofs 267

8.4 Exercises 271

Part III Conditional Expectation and Regression

9 Conditional Expectation Value and Discrete Conditional Expectation 277

9.1 Conditional Expectation Value 277

9.2 Transformation Theorem 280

9.3 Other Properties 282

9.4 Discrete Conditional Expectation 283

9.5 Discrete Regression 285

9.6 Examples 287

9.7 Proofs 291

9.8 Exercises 291

10 Conditional Expectation 295

10.1 Assumptions and Definitions 295

10.2 Existence and Uniqueness 297

10.2.1 Uniqueness With Respect to a Probability Measure 298

10.2.2 A Necessary and Sufficient Condition of Uniqueness 299

10.2.3 Examples 300

10.3 Rules of Computation and Other Properties 301

10.3.1 Rules of Computation 301

10.3.2 Monotonicity 302

10.3.3 Convergence Theorems 302

10.4 Factorization, Regression, and Conditional Expectation Value 306

10.4.1 Existence of a Factorization 306

10.4.2 Conditional Expectation and Mean-Squared Error 307

10.4.3 Uniqueness of a Factorization 308

10.4.4 Conditional Expectation Value 309

10.5 Characterizing a Conditional Expectation by the Joint Distribution 312

10.6 Conditional Mean Independence 313

10.7 Proofs 318

10.8 Exercises 321

11 Residual, Conditional Variance, and Conditional Covariance 329

11.1 Residual With Respect to a Conditional Expectation 329

11.2 Coefficient of Determination and Multiple Correlation 333

11.3 Conditional Variance and Covariance Given a σ-Algebra 338

11.4 Conditional Variance and Covariance Given a Value of a Random Variable 339

11.5 Properties of Conditional Variances and Covariances 342

11.6 Partial Correlation 345

11.7 Proofs 347

11.8 Exercises 348

12 Linear Regression 357

12.1 Basic Ideas 357

12.2 Assumptions and Definitions 359

12.3 Examples 361

12.4 Linear Quasi-Regression 366

12.5 Uniqueness and Identification of Regression Coefficients 367

12.6 Linear Regression 369

12.7 Parametrizations of a Discrete Conditional Expectation 370

12.8 Invariance of Regression Coefficients 374

12.9 Proofs 375

12.10Exercises 377

13 Linear Logistic Regression 381

13.1 Logit Transformation of a Conditional Probability 381

13.2 Linear Logistic Parametrization 383

13.3 A Parametrization of a Discrete Conditional Probability 385

13.4 Identification of Coefficients of a Linear Logistic Parametrization 387

13.5 Linear Logistic Regression and Linear Logit Regression 388

13.6 Proofs 394

13.7 Exercises 396

14 Conditional Expectation With Respect to a Conditional-Probability Measure 399

14.1 Introductory Examples 399

14.2 Assumptions and Definitions 404

14.3 Properties 410

14.4 Partial Conditional Expectation 412

14.5 Factorization 413

14.5.1 Conditional Expectation Value With Respect to PB 414

14.5.2 Uniqueness of Factorizations 415

14.6 Uniqueness 415

14.6.1 A Necessary and Sufficient Condition of Uniqueness 415

14.6.2 Uniqueness w.r.t. P and Other Probability Measures 417

14.6.3 Necessary and Sufficient Conditions of P-Uniqueness 418

14.6.4 Properties Related to P-Uniqueness 420

14.7 Conditional Mean Independence With Respect to PZ=z 424

14.8 Proofs 426

14.9 Exercises 431

15 Conditional Effect Functions of a Discrete Regressor 437

15.1 Assumptions and Definitions 437

15.2 Conditional Intercept Function and Effect Functions 438

15.3 Implications of Independence of X and Z for Regression Coefficients 441

15.4 Adjusted Conditional Effect Functions 443

15.5 Conditional Logit Effect Functions 447

15.6 Implications of Independence of X and Z for the Logit Regression Coefficients 450

15.7 Proofs 452

15.8 Exercises 454

Part IV Conditional Independence and Conditional Distribution

16 Conditional Independence 459

16.1 Assumptions and Definitions 459

16.1.1 Two Events 459

16.1.2 Two Sets of Events 461

16.1.3 Two Random Variables 462

16.2 Properties 463

16.3 Conditional Independence and Conditional Mean Independence 470

16.4 Families of Events 473

16.5 Families of Set Systems 473

16.6 Families of Random Variables 475

16.7 Proofs 478

16.8 Exercises 486

17 Conditional Distribution 491

17.1 Conditional Distribution Given a σ-Algebra or a Random Variable 491

17.2 Conditional Distribution Given a Value of a Random Variable 494

17.3 Existence and Uniqueness 497

17.3.1 Existence 497

17.3.2 Uniqueness of the Functions PY |C ( ·, A′) 498

17.3.3 Common Null Set (CNS) Uniqueness of a Conditional Distribution 499

17.4 Conditional-Probability Measure Given a Value of a Random Variable 502

17.5 Decomposing the Joint Distribution of Random Variables 504

17.6 Conditional Independence and Conditional Distributions 506

17.7 Expectations With Respect to a Conditional Distribution 511

17.8 Conditional Distribution Function and Probability Density 513

17.9 Conditional Distribution and Radon-Nikodym Density 516

17.10Proofs 520

17.11Exercises 536

References 541

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.