本书内容分为三部分：(1)?线性回归分析所需要的矩阵理论、多元正态分布；(2)?线性回归的基本理论和方法，包括线性估计的一般小样本理论、关于线性假设的?F-检验方法、基于线性模型的方差分析理论、变量选择问题的讨论、共线性问题、异常值问题以及?Box-Cox?模型等与线性回归相关的内容；(3)?用于分类响应变量的Logist回归模型的基本理论和方法。
　　本书要求读者具有高等代数(或者线性代数)和概率论与数理统计的良好基础。本书的特点之一是在尽可能少的基础知识要求下讲清线性回归分析的理论问题，同时，本书也附带了一些SAS代码，这将有助于实际应用中的数据处理。
　　本书可供统计学专业、数学专业或者其他相关专业作为本科生回归分析课程教材使用，也可作为非统计学专业的研究人员学习回归分析基础理论的参考书使用。

　本书内容能够分为三部分：（1）线性回归分析所需要的矩阵代数理论、多元正态分布；（2）线性回归的基本利率和方法，包括线性估计的一般小样本理论、关于线性线性假设的F-检验方法、基于线性模型的方差分析理论、变量选择问题的讨论、共线性问题、异常值问题以及Box-Cox模型等与线性回归相关的内容；（3）用于分类响应变量的Logist回归模型的基本理论和方法。本书要求读者具有高等代数（或者线性代数）和概率论与数理统计的良好基础。本书的特点之一是在尽可能少的基础知识要求下讲清回归分析的理论问题，同事，本书也附带了一些SAS代码，这将有助于实际应用中的数据处理。本书可供统计学专业、数学专业或者其他相关专业作为本科生回归分析课程教材使用，也可作为非统计学专业的研究人员学习回归分析基础理论的参考使用。

吴贤毅，华东师范大学金融与统计学院教授，博士生导师，研究领域包括随机调度，概率统计，非寿险精算，在随机调度，概率统计以及非寿险精算的国际主流杂志发表过数十篇学术研究论文，在随机调度方面的研究获得过三次国家自然科学基金资助，其在随机调度方面的研究成果发表于Operations Research,European Journal of Operations Research,Journal of Scheduling 等。

Contents
Chapter 1 Preliminaries: Matrix Algebra and Random Vectors ........ 1
1.1 Preliminary matrix algebra ............................................................ 1
1.1.1 Trace and eigenvalues.......................................................... 1
1.1.2 Symmetric matrices............................................................. 3
1.1.3 Idempotent matrices and orthogonal projection..................... 6
1.1.4 Singular value decomposition ............................................... 9
1.1.5 Vector di.erentiation and generalized inverse .......................10
1.1.6 Exercises ...........................................................................10
1.2 Expectation and covariance...........................................................11
1.2.1 Basic properties .................................................................11
1.2.2 Mean and variance of quadratic forms .................................12
1.2.3 Exercises ...........................................................................14
1.3 Moment generating functions and independence .............................16
1.3.1 Exercises ...........................................................................17
Chapter 2 Multivariate Normal Distributions.....................................18
2.1 De.nitions and fundamental results ...............................................18
2.2 Distribution of quadratic forms .....................................................25
2.3 Exercises......................................................................................27
Chapter 3 Linear Regression Models ...................................................29
3.1 Introduction.................................................................................29
3.2 Regression interpreted as conditional mean ....................................31
3.3 Linear regression interpreted as linear prediction ............................33
3.4 Some nonlinear regressions............................................................34
3.5 Typical data structure of linear regression models ..........................35
3.6 Exercises......................................................................................36
Chapter 4 Estimation and Distribution Theory ..................................38
4.1 Least squares estimation (LSE) .....................................................38
4.1.1 Motivation: why is LS reasonable........................................38
Regression Analysis
4.1.2 The LS solution .................................................................40
4.1.3 Exercises ...........................................................................48
4.2 Properties of LSE .........................................................................50
4.2.1 Small sample distribution-free properties .............................51
4.2.2 Properties under normally distributed errors........................55
4.2.3 Asymptotic properties ........................................................57
4.2.4 Exercises ...........................................................................60
4.3 Estimation under linear restrictions ...............................................60
4.4 Introducing further explanatory variables and related topics ...........67
4.4.1 Introducing further explanatory variables ............................67
4.4.2 Centering and scaling the explanatory variables ...................71
4.4.3 Estimation in terms of linear prediction...............................72
4.4.4 Exercises ...........................................................................73
4.5 Design matrices of less than full rank.............................................74
4.5.1 An example .......................................................................74
4.5.2 Estimability.......................................................................74
4.5.3 Identi.ability under Constraints..........................................76
4.5.4 Dropping variables to change the model ..............................77
4.5.5 Exercises ...........................................................................77
4.6 Generalized least squares ..............................................................78
4.6.1 Basic theory ......................................................................78
4.6.2 Incorrect speci.cation of variance matrix.............................80
4.6.3 Exercises ...........................................................................83
4.7 Bayesian estimation......................................................................83
4.7.1 The basic idea....................................................................83
4.7.2 Normal-noninformative structure ........................................84
4.7.3 Conjugate priors ................................................................85
4.8 Numerical examples......................................................................86
4.9 Exercises......................................................................................90
Chapter 5 Testing Linear Hypotheses ..................................................92
5.1 Linear hypotheses.........................................................................92
5.2 F -Test .........................................................................................93
5.2.1 F -test................................................................................94
Contents VII
5.2.2 What are actually tested ....................................................95
5.2.3 Examples...........................................................................96
5.3 Con.dence ellipse ....................................................................... 101
5.4 Prediction and calibration...........................................................103
5.5 Multiple correlation coe.cient .................................................... 105
5.5.1 Variable selection ............................................................. 105
5.5.2 Multiple correlation coe.cient: straight line ...................... 106
5.5.3 Multiple correlation coe.cient: multiple regression ............ 108
5.5.4 Partial correlation coe.cient ............................................ 110
5.5.5 Adjusted multiple correlation coe.cient ............................ 111
5.6 Testing linearity: goodness-of-.t test ........................................... 112
5.7 Multiple comparisons..................................................................114
5.7.1 Simultaneous inference ..................................................... 114
5.7.2 Some classical methods for simultaneous intervals .............. 116
5.8 Univariate analysis of variance .................................................... 120
5.8.1 ANOVA model................................................................. 120
5.8.2 ANCOVA model .............................................................. 126
5.8.3 SAS procedures for ANOVA ............................................. 127
5.9 Exercises.................................................................................... 129
Chapter 6 Variable Selection............................................................... 133
......