Correlation & Regression • CR8

Multiple Regression Foundation

When one variable is predicted from two independent variables, the regression line becomes a regression plane. Here the main focus is meaning, normal equations and interpretation.

What Is Multiple Regression?

In simple regression, one independent variable is used to predict one dependent variable. In multiple regression, two or more independent variables are used to predict one dependent variable. For two independent variables, we write: \[ \hat{Y}=A+BX_1+CX_2 \]

Dependent Variable

\(Y\) is the value we want to predict.

Examples: expenditure, productivity, salary, marks, cost.

First Independent Variable

\(X_1\) is the first variable used to predict \(Y\).

Its coefficient is \(B\).

Second Independent Variable

\(X_2\) is the second variable used to predict \(Y\).

Its coefficient is \(C\).

Meaning of the Regression Plane

\[ \hat{Y}=A+BX_1+CX_2 \] \(A\) is the intercept.
\(B\) tells how much predicted \(Y\) changes when \(X_1\) increases by one unit, keeping \(X_2\) constant.
\(C\) tells how much predicted \(Y\) changes when \(X_2\) increases by one unit, keeping \(X_1\) constant.
Very important: In multiple regression, never say \(B\) is the total effect of \(X_1\) without condition. Say it properly: \(B\) is the change in predicted \(Y\) for one unit increase in \(X_1\), keeping \(X_2\) constant.

Normal Equations for Multiple Regression

Step 1: Assume the regression plane

\[ \hat{Y}=A+BX_1+CX_2 \]

We find \(A\), \(B\), and \(C\) using normal equations.

First Normal Equation

Taking summation on both sides:

\[ \sum Y=\sum(A+BX_1+CX_2) \]

\[ \sum Y=nA+B\sum X_1+C\sum X_2 \]

\[ \boxed{\sum Y=nA+B\sum X_1+C\sum X_2} \]

Second Normal Equation

Multiply the assumed equation by \(X_1\):

\[ X_1Y=AX_1+BX_1^2+CX_1X_2 \]

Taking summation:

\[ \sum X_1Y=A\sum X_1+B\sum X_1^2+C\sum X_1X_2 \]

\[ \boxed{\sum X_1Y=A\sum X_1+B\sum X_1^2+C\sum X_1X_2} \]

Third Normal Equation

Multiply the assumed equation by \(X_2\):

\[ X_2Y=AX_2+BX_1X_2+CX_2^2 \]

Taking summation:

\[ \sum X_2Y=A\sum X_2+B\sum X_1X_2+C\sum X_2^2 \]

\[ \boxed{\sum X_2Y=A\sum X_2+B\sum X_1X_2+C\sum X_2^2} \]

Short form to remember: \[ \sum Y=nA+B\sum X_1+C\sum X_2 \] \[ \sum X_1Y=A\sum X_1+B\sum X_1^2+C\sum X_1X_2 \] \[ \sum X_2Y=A\sum X_2+B\sum X_1X_2+C\sum X_2^2 \]

Calculator Setup for Multiple Regression

Most calculators used by students allow only two-variable statistics mode at a time. So, we cannot enter \(X_1\), \(X_2\), and \(Y\) together in one single calculator table.

The calculator may show the columns only as \(X\) and \(Y\), but those are just calculator labels. We must carefully decide what those two columns mean in each entry.

Smart way: Enter two variables at a time, collect the sums, then rename the calculator results properly before forming the normal equations.
Entry Put in Calculator Column 1 Put in Calculator Column 2 Write the Sums Like This
1 \(X_1\) \(X_2\) \(\sum X_1,\ \sum X_2,\ \sum X_1^2,\ \sum X_2^2,\ \sum X_1X_2\)
2 \(Y\) \(X_2\) \(\sum Y,\ \sum X_2,\ \sum YX_2\)
3 \(Y\) \(X_1\) \(\sum Y,\ \sum X_1,\ \sum YX_1\)
Careful: If you enter \(Y\) and \(X_2\), then the calculator's \(\sum XY\) means \(\sum YX_2\). If you enter \(Y\) and \(X_1\), then the calculator's \(\sum XY\) means \(\sum YX_1\). The calculator does not know the original names used in the question.
After collecting the sums, write the three normal equations: \[ \sum Y=nA+B\sum X_1+C\sum X_2 \] \[ \sum X_1Y=A\sum X_1+B\sum X_1^2+C\sum X_1X_2 \] \[ \sum X_2Y=A\sum X_2+B\sum X_1X_2+C\sum X_2^2 \] Then solve these three equations using the calculator and write: \[ A=\cdots,\qquad B=\cdots,\qquad C=\cdots \] Finally, write the regression plane: \[ \hat{Y}=A+BX_1+CX_2 \]
Now let us check whether the idea is clear. Try the interpretation first. If you are not able to complete it, open the solution or watch the video. Do not forget to support SJMathTube.

Quick Interpretation Practice

Multiple Regression • Coefficient Meaning

Suppose a multiple regression equation is: \[ \hat{Y}=10+2.5X_1+4X_2 \] Answer the following interpretation questions.

✓
Enter your answer first.
Correct answer: \(10\)
✓
Enter your answer first.
Correct answer: \(2.5\)
✓
Enter your answer first.
Correct answer: \(4\)
✓
Select your answer first.
Correct answer: Predicted \(Y\) increases by \(2.5\) when \(X_1\) increases by 1 unit, keeping \(X_2\) constant.
✓
Select your answer first.
Correct answer: Predicted \(Y\) increases by \(4\) when \(X_2\) increases by 1 unit, keeping \(X_1\) constant.
✓
Enter your answer first.
Correct answer: \(32\)

Try once before opening the solution

First identify \(A\), \(B\), and \(C\). Then write the meaning of \(B\) and \(C\) carefully using “keeping the other variable constant”.

Detailed Explanation

The given regression equation is:

\[ \hat{Y}=10+2.5X_1+4X_2 \]

Compare it with:

\[ \hat{Y}=A+BX_1+CX_2 \]

Therefore,

\[ A=10,\qquad B=2.5,\qquad C=4 \]

Meaning of \(B=2.5\):

When \(X_1\) increases by one unit, the predicted value of \(Y\) increases by \(2.5\), keeping \(X_2\) constant.

Meaning of \(C=4\):

When \(X_2\) increases by one unit, the predicted value of \(Y\) increases by \(4\), keeping \(X_1\) constant.

For \(X_1=4\) and \(X_2=3\),

\[ \hat{Y}=10+2.5(4)+4(3) \]

\[ \hat{Y}=10+10+12 \]

\[ \hat{Y}=32 \]

Final Interpretation

\(A=10,\ B=2.5,\ C=4\) \(B=2.5\): Predicted \(Y\) increases by \(2.5\) when \(X_1\) increases by one unit, keeping \(X_2\) constant. \(C=4\): Predicted \(Y\) increases by \(4\) when \(X_2\) increases by one unit, keeping \(X_1\) constant. For \(X_1=4,\ X_2=3\), predicted value \(\hat{Y}=32\).

Video Explanation

In this video, we explain the multiple regression equation, the three normal equations and the correct meaning of each coefficient.
Formulae

Available Formula Sheets

⌂ Home
Latest
Latest Content Auto
Home