How to Fit a Multiple Regression with Two Predictors

Fits y = b₀ + b₁x₁ + b₂x₂ to predict y from two explanatory variables. Each coefficient shows the effect of its own variable once the other has been accounted for. The adjusted coefficient of determination is also given.

This builds a formula predicting $y$ from two explanatory variables, choosing the three coefficients that make the sum of squared errors as small as possible.

y = b_0 + b_1x_1 + b_2x_2

Unlike simple regression, $b_1$ gives the effect of $x_1$ with $x_2$ held constant: the slope once the other variable has been accounted for.

The coefficient of determination

$R^2$ is the share of the variation in $y$ that the formula explains. Adding an explanatory variable always raises it, though, so on its own it rewards piling variables in. The adjusted $R^2$ penalises the count of variables, and is the one to compare on.

Example

Six observations give $y = 1.71 + 2.375x_1 + 0.375x_2$ , with $R^2$ of 0.997 and an adjusted figure of 0.996. Putting in $x_1 = 7$ and $x_2 = 7$ predicts 20.96.

Predictors that move together cannot be separated

If $x_1$ and $x_2$ are exactly proportional, there is no telling which one is doing the work and the coefficients are not determined. Short of exact, a strong relationship still leaves them unstable, sometimes even flipping their signs. This is multicollinearity.

Notes

Three coefficients need at least four observations.

The sizes of the coefficients cannot be compared directly. Different units give different sizes.