How to Fit a Regression Line by Least Squares

Fits the line y = ax + b that best matches the points by least squares. The slope is the covariance ÷ the variance of x, and the intercept is ȳ − a·x̄. R² measures how well the line fits.

Explanation

A regression line is the straight line $y = ax + b$ that best fits a cloud of points. "Best" means that the sum of the squared vertical gaps between the points and the line is as small as possible — the method of least squares.

a = \dfrac{s_{xy}}{\sigma_x^2} \qquad b = \bar{y} - a\bar{x}

The slope $a$ is the covariance divided by the variance of $x$ . The intercept $b$ follows from the fact that the line always passes through the point $(\bar{x}, \bar{y})$ .

Example

With $x$ = 1, 2, 3, 4, 5 and $y$ = 2, 4, 5, 4, 5, we have $\bar{x} = 3$ , $\bar{y} = 4$ , a covariance of 1.2 and a variance of $x$ equal to 2.

a = \dfrac{1.2}{2} = 0.6 \qquad b = 4 - 0.6 \times 3 = 2.2

The regression line is $y = 0.6x + 2.2$ , so an $x$ of 6 predicts a $y$ of 5.8.

R squared

The coefficient of determination $R^2$ is the square of the correlation coefficient. It is the share of the variation in $y$ that the line accounts for.

Here $r = 0.7746$ , so $R^2 = 0.6$ : the line explains 60% of the variation in $y$ .

Watch out

Do not extend the line beyond the range of your data. Predicting $y$ at $x = 100$ from data covering $x$ from 1 to 5 has no basis
A single outlier can swing the slope dramatically
The line that predicts $x$ from $y$ is a different line. Swapping the roles does not simply invert the equation