Word Count : 596

---

# Derivation + manual worked example (no Python) We start from the simple linear regression model $$ Y_i=\alpha+\beta X_i+u_i,\qquad i=1,\dots,n, $$ with the usual OLS assumptions (in particular $E[u_i\mid X]=0$). ## 1) OLS estimators (derivation) OLS minimizes the sum of squared residuals $S(\alpha,\beta)=\sum_{i=1}^n (Y_i-\alpha-\beta X_i)^2$. Set partial derivatives to zero (normal equations): $$ \begin{cases} \frac{\partial S}{\partial \alpha}=-2\sum_{i}(Y_i-\alpha-\beta X_i)=0\\[6pt] \frac{\partial S}{\partial \beta}=-2\sum_{i}X_i(Y_i-\alpha-\beta X_i)=0 \end{cases} $$ From these we get $$ \sum_i Y_i = n\hat\alpha + \hat\beta\sum_i X_i,\qquad \sum_i X_iY_i = \hat\alpha\sum_i X_i + \hat\beta\sum_i X_i^2. $$ Solving yields the familiar formulas (written in mean-deviation form): $$ \boxed{\;\hat\beta=\dfrac{\sum_{i=1}^n (X_i-\bar X)(Y_i-\bar Y)}{\sum_{i=1}^n (X_i-\bar X)^2}\;},\qquad \boxed{\;\hat\alpha=\bar Y-\hat\beta\,\bar X\;}. $$ ## 2) Residuals and variance estimator Predicted values $\hat Y_i=\hat\alpha+\hat\beta X_i$. Residuals $\hat u_i ______ __________ ____ _______ ___ ______ _______ __________ _______ ______ _____.
_____ ________ ______ ___ _______ __________ _______ ________ __________ ___ ________.
________ _________ __________ ________ ________ ______ _____ _________ __________.
________ _______ ________ ____ _________ _______ _________ __________ _______ _______ ___.
_________ ________ ______ ____ ______ __________ ___ __________ _______ _________ ____.
___ _____ ______ ___ ___ ___ ______ ________ _____ ______ _______ __________.
________ _____ ____ ____ ____ __________.
_____ __________ _____ _____ _________ _____ ____ _____ ___ _______ _______ _________.
_________ __________ _______ __________ __________ _______ ___ _______ ______ ______ __________.
____ _________ __________ ___ __________ _____ __________ ______ ________ ______ ______ _____.
_____ ______ ______ _____ ________ _______ __________ ____ ________ _______.
_______ ________ ______ ________ ______.
______ _______ _______ _________ _______ ____ ____ __________ _____ _______ _____.
_______ ______ __________ _______ _______.
__________ _____ _____ ___ _________ ____ ______ _________ ____ __________ ____ _________.
____ _________ ________ ______ ________ _________ _______ ___ _____ ____ ___ _______.
___ _____ ______ _______ ______ ___ _____ _______ ___ ______ __________.
__________ __________ ____ ____ _______ ___ _______ ______ ________.
_____ ___ _________ ______ _____ ________ ______ ___ __________.
_______ ___ _______ ______ ____.
______ ___ ________ ______ _________ _________ _______ ____ ________ ________ ______ __________.
__________ ___ ______ __________ ________ _______ __________ ______ _________ __________ _______.
_______ _________ ____ _________ ________ _______ __________ ____ ___ ________.
______ ___ _______ ___ ____ _____ ___.
___ ______ ______ _________ ________ ____ _______ _____ _________.
________ ___ ____ _______ _____ ____ ________ __________ ________ ___.
_________ _________ _______ ______ _________ ________ ___ ______ ________ ______ _____ ______.
_____ ______ ___ ___ __________ ______ _______ _____.
____ ___ ________ ______ __________ _______ ________ ____ _____ __________.
_______ ____ ___ __________ ______ ____ ____ ___ ___.
_______ _______ ______ _______ __________ __________.
_____ ___ ___ ___ __________ ______ ____ __________ __________ _____ ______.
________ _______ ________ ____ ______ _______ _________ __________ ______ __________ __________.
___ _________ ________ _______ ___.
_________ _______ ______ __________ ___ ____ _____ _____ _________ _________ ________ ___.
________ ___ ________ ___ ______ ___ ____.
______ ___ _________ _________ _____ ______ __________ _____ ____ ____.
_______ _______ _______ ___ _________ ___ ___ ________ _______ __________.
__________ _____ _______ ________ ______ _________.
_________ _____ _____ ______ ________ ________.
_______ _____ _____ ___ _______ _______ ______ ___.
_________ ________ ____ _____ _______.
________ ______ _______ _______ ____.
____ ________ ____ _______ _______ _______ _________ ______ _______ _______.
_______ _________ ___ _____ __________ ______ _____ _____ _______ _______ __________.
____ ________ _________ _____ ______ ______.
____ ________ __________ _______ _________ _______ ___.
__________ ____ ___ _____ _______ _____ ____ ___ _____ ___ ___.
_____ ______ ______ _____ ________ ___.
_____ __________ _________ _______ _________.
____ ____ ___ ___ ___ __________.
_______ ________ _________ ________ __________ _______.
____ ____ ___ __________ ______ _________ _______ _____ ________.
__________ ________ ___ _____ ____.
Get Full Answer on WhatsApp