Word Count : 596

---

# Derivation + manual worked example (no Python) We start from the simple linear regression model $$ Y_i=\alpha+\beta X_i+u_i,\qquad i=1,\dots,n, $$ with the usual OLS assumptions (in particular $E[u_i\mid X]=0$). ## 1) OLS estimators (derivation) OLS minimizes the sum of squared residuals $S(\alpha,\beta)=\sum_{i=1}^n (Y_i-\alpha-\beta X_i)^2$. Set partial derivatives to zero (normal equations): $$ \begin{cases} \frac{\partial S}{\partial \alpha}=-2\sum_{i}(Y_i-\alpha-\beta X_i)=0\\[6pt] \frac{\partial S}{\partial \beta}=-2\sum_{i}X_i(Y_i-\alpha-\beta X_i)=0 \end{cases} $$ From these we get $$ \sum_i Y_i = n\hat\alpha + \hat\beta\sum_i X_i,\qquad \sum_i X_iY_i = \hat\alpha\sum_i X_i + \hat\beta\sum_i X_i^2. $$ Solving yields the familiar formulas (written in mean-deviation form): $$ \boxed{\;\hat\beta=\dfrac{\sum_{i=1}^n (X_i-\bar X)(Y_i-\bar Y)}{\sum_{i=1}^n (X_i-\bar X)^2}\;},\qquad \boxed{\;\hat\alpha=\bar Y-\hat\beta\,\bar X\;}. $$ ## 2) Residuals and variance estimator Predicted values $\hat Y_i=\hat\alpha+\hat\beta X_i$. Residuals $\hat u_i = Y_i-\hat Y_i$. Sum of squared residuals $SSR=\sum_{i=1}^n \hat u_i^2.$ Unbiased estimator of the error variance: $$ \boxed{\;\hat\sigma^2=\dfrac{SSR}{\,n-2\,}\;}. $$ ## 3) Sampling variances (under homoskedasticity) $$ \operatorname{Var}(\hat\beta)=\dfrac{\sigma^2}{\sum_{i=1}^n (X_i-\bar X)^2},\qquad \operatorname{Var}(\hat\alpha)=\sigma^2\Big(\frac{1}{n}+\frac{\bar _____ _______ ______ ______ _____ ____ ________ _______ __________ _______ _________.
____ ______ ___ ________ _________.
__________ _______ _______ ___ _________ _______ ______ ___ ____ __________ ___.
__________ ___ _______ _______ ___ _______ _________.
__________ ______ ____ ______ _______ ________ _____.
__________ __________ _______ _________ ___.
___ _______ ___ _____ ______ ____ __________.
___ ____ _______ ____ ____ _________ ________ ___.
___ ________ __________ ______ _________.
___ _____ ____ __________ ___ _______.
__________ _____ ______ ______ _______ ___ _______ __________ ______.
__________ ________ ____ __________ _________ ________ ________.
________ ______ ___ ________ ___ ____ _________ ______.
__________ __________ ____ _____ ________ ___ _________ ______.
_______ _____ _____ ____ _______ ____.
_____ ________ ______ __________ _________ ___ _____ _____.
_____ ______ ___ ______ ________.
____ _______ ____ _______ __________.
___ __________ _____ _____ _______ ____ _____.
_________ ____ ____ ____ __________ ______ _______ _______ _________.
_________ ____ _______ ________ _____.
________ ___ ____ __________ ____.
________ _______ _______ _________ __________ _____ _______ __________ __________.
_____ ___ _______ _________ ____ ____.
___ ______ _________ ___ _______ ____ __________ _______ _________ __________.
___ __________ ______ __________ ______.
______ ______ ________ _______ _________ ____ ______ _____ _________ ________ ________ __________.
__________ ____ ____ ______ ____ ________ ___.
______ ______ ____ ________ ________ ____ _______ ____ __________ ___.
_______ _________ ____ ______ _______ ____ ___ _______ ___ ________ ______ ________.
_________ ________ ___ ________ _____ _________ ______ ______.
__________ __________ ________ ________ _____ ____.
__________ ___ ________ _________ ___ ____ _________ _______.
______ ______ _____ _____ ___.
___ ___ _____ ____ __________ _____ _______ ___ ___ ______ ________ _____.
______ __________ __________ __________ _____ __________ ________ ______.
______ _____ _____ ___ _________ ________ ______.
____ _________ _________ _________ _______ _________ ______.
______ _______ ______ ________ ________ ________ _____ ________ ___.
_________ __________ _____ __________ ____ ___ ___ ____ __________ ___ ___ ____.
______ ________ _________ ____ ___ _____ ________ _____ ____ ______.
_________ __________ ___ __________ ________ __________ ________ ____.
________ ______ __________ ___ ____ ___ ________ _____ __________ __________ __________ ______.
___ _________ _______ ________ ___.
____ _____ _______ ____ ______ ________.
______ ___ ______ _______ ______ ______ _______ __________ __________ ____.
__________ ____ _________ _________ ___.
_____ _________ __________ ________ __________ ________ ____.
__________ ___ ______ __________ ________.
___ ______ ___ _______ ________ _____ _________ ______.
_______ _____ _____ _____ ____.
__________ ____ ___ _______ ______ _________ ______ _______ ______.
__________ __________ ____ _____ _____ ________.
___ _______ _______ _______ _________ _________ ____ ____ _______ _________ ____.
_________ ________ ____ ______ ____ ___ ______ _________ _____ __________.
__________ ___ ________ ___ _____ __________ ________ ________ ______.
______ __________ ___ _____ ________ ________ ___.
___ _______ ______ ______ ________ ____.
__________.
Get Full Answer on WhatsApp