Notebook for SI
Preface
1
Probability Theory
1.1
Set Theory
1.2
Basics of Probability Theory
1.2.1
Axiomatic（公理的） Foundations
1.2.2
The Calculus of Probabilities
1.2.3
Counting
1.2.4
Enumerating Outcomes
1.3
Conditional Probability and Independence
1.4
Random Variables
1.5
Distribution Functions
1.6
Density and Mass Functions
1.7
Miscellanea
2
Transformations and Expectations
2.1
Distributions of Functions of a Random Variable
2.2
Expected Values
2.3
Moments and Moment Generating Functions
2.4
Differentiating Under an Integral Sign
2.5
Miscellanea
3
Common Families of Distributions
3.1
Introduction
3.2
Discrete Distributions
3.3
Continuous Distributions
3.4
Exponential Families
3.5
Location and Scale Families
3.6
Inequalities and Identities
3.6.1
Probability Inequalities
3.6.2
Identities
3.7
Miscellanea
4
Multiple Random Variables
4.1
Joint and Marginal Distributions
4.2
Conditional Distributions and Independence
4.3
Bivariate Transformations
4.4
Hierarchical Models and Mixtrue Distributions
4.5
Covariance and Correlation
4.6
Multivariate Distributions
4.7
Inequalities
4.7.1
Numerical Inequalities
4.7.2
Functional Inequalities
4.8
Miscellanea
5
Properties of a Random Sample
5.1
Basic Concepts of Random Samples
5.2
Sums of Random Variables from a Random Sample
5.3
Sampling from the Normal Distribution
5.3.1
Properties of the Sample Mean and Variance
5.3.2
The Derived Distributions: Student’s t and Snedecor’s F
5.4
Order Statistics
5.5
Convergence Concepts
5.5.1
Convergence in Probability
5.5.2
Almost Sure Convergence
5.5.3
Convergence in Distribution
5.5.4
The Delta Method
5.6
Generating a Random Sample
5.6.1
Direct Methods
5.6.2
Indirect Methods
5.6.3
The Accept/Reject Algorithm
5.7
Miscellance
6
Principles of Data Reduction
6.1
Introduction
6.2
The Sufficiency Principle
6.2.1
Sufficient Statistics
6.2.2
Minimal Sufficient Statistics
6.2.3
Ancillary Statistics
6.2.4
Sufficient, Ancillary, and Complete Statistics
6.3
The Likelihood Principle
6.3.1
The Likelihood Function
6.3.2
The Formal Likelihood Principe
6.4
The Equivariance Principle
6.5
Miscelllanea
7
Point Estimation
7.1
Introduction
7.2
Methods of Finding Estimators
7.2.1
Method of Moments
7.2.2
Maximum Likelihood Estimators
7.2.3
Bayes Estimators
7.2.4
The EM Algorithm
7.3
Methods of Evaluating Estimators
7.3.1
Mean Squared Error
7.3.2
Best Unbiased Estimators
7.3.3
Sufficiency and Unbiasedness
7.3.4
Loss Function Optimality
7.4
Miscellanea
8
Hypothesis Testing
8.1
Introduction
8.2
Methods of Finding Tests
8.2.1
Likelihood Ratio Tests
8.2.2
Bayesian Tests
8.2.3
Union-Intersection and Intersection-Union Tests
8.3
Methods of Evaluating Tests
8.3.1
Error Probabilities and the Power Function
8.3.2
Most Powerful Tests
8.3.3
Sizes of Union-Intersection and Intersection-Union Tests
8.3.4
p-Values
8.3.5
Loss Function Optimality
8.4
Miscellanea
9
Interval Estimation
9.1
Introduction
9.2
Methods of Finding Interval Estimators
9.2.1
Inverting a Test Statistic
9.2.2
Pivotal Quantities
9.2.3
Pivoting the CDF
9.2.4
Bayesian Intervals
9.3
Methods of Evaluating Interval Estimators
9.3.1
Size and Coverage Probability
9.3.2
Test-Related Optimality
9.3.3
Bayesian Optimality
9.3.4
Loss Function Optimality
9.4
Miscllanea
10
Asymptotic Evaluations
10.1
Point Estimation
10.1.1
Consistency
10.1.2
Efficiency
10.1.3
Calculations and Comparisons
10.1.4
Bootstrap Standard Errors
10.2
Robustness
10.2.1
The Mean and the Median
10.2.2
M-Estimators
10.3
Hypothesis Testing
10.3.1
Asymptotic Distribution of LRTs
10.3.2
Other Large-Sample Tests
10.4
Interval Estimation
10.4.1
Approximate Maximum Likelihood Intervals
10.4.2
Other Large-Sample Intervals
10.5
Miscellanea
11
Analysis of Variance and Regression
11.1
Introduction
11.2
Oneway Analysis of Variance
11.2.1
Model and Distribution Assumptions
11.2.2
The Classical ANOVA Hypothesis
11.2.3
Inferences Regarding Linear Combinations of Means
11.2.4
The ANOVA F Test
11.2.5
Simultaneous Estimation of Contrasts
11.2.6
Partitioning Sums of Squares
11.3
Simple Linear Regression
11.3.1
Least Squares: A Mathematical Solution
11.3.2
Best Linear Unbiased Estimators: A Statistical Solution
11.3.3
Models and Distribution Assumptions
11.3.4
Estimation and Testing with Normal Erros
11.3.5
Estimation and Prediction at a Specified x = x0
11.3.6
Simultaneous Estimation and Confidence Bands
11.4
Miscellanea
12
Regression Models
12.1
Introduction
12.2
Regression with Errors in Variables
12.2.1
Functional and Structural Relationships
12.2.2
A Least Squares Solution
12.2.3
Maximum Likelihood Estimation
12.2.4
Confidence Sets
12.3
Logistic Regression
12.3.1
The Model
12.3.2
Estimation
12.4
Robust Regression
12.5
Miscellanea
Published with
bookdown
by
zjuwhw
12
Regression Models
12.1
Introduction
12.2
Regression with Errors in Variables
12.2.1
Functional and Structural Relationships
12.2.2
A Least Squares Solution
12.2.3
Maximum Likelihood Estimation
12.2.4
Confidence Sets
12.3
Logistic Regression
12.3.1
The Model
12.3.2
Estimation
12.4
Robust Regression
12.5
Miscellanea