Correlation & Regression • CR13

Important Correlation and Regression Theory Questions

Exam-oriented theory answers for IOE Probability and Statistics. Read the question, prepare the points, then click Check Answer.

Theory Revision

Write theory answers in clear points. Use the terms degree, direction, explained variation, predicted value and keeping the other variable constant.
1

Correlation Coefficient and Coefficient of Determination

Define the correlation coefficient and mention its important properties. What does the coefficient of determination measure?

Answer

Correlation coefficient is a numerical measure of the degree and direction of linear relationship between two variables.

\[-1\leq r\leq1\]

  • If \(r>0\), the variables move in the same direction.
  • If \(r<0\), the variables move in opposite directions.
  • If \(r=0\), there is no linear correlation.
  • The value of \(r\) is independent of change of origin and scale.
  • The value of \(r\) has no unit.
  • The closer \(r\) is to \(1\) or \(-1\), the stronger the relationship.

Coefficient of determination is \(r^2 imes100\%\). It measures the percentage of variation in the dependent variable explained by the independent variable according to the fitted model.

2

Coefficient of Determination and Properties

Explain coefficient of determination and its interpretation. Write properties of correlation coefficients and regression coefficients.

Answer

The coefficient of determination is:

\[r^2 imes100\%\]

It gives the percentage of variation in the dependent variable explained by the independent variable according to the fitted model.

Properties of correlation coefficient:

  • \(-1\leq r\leq1\)
  • It has no unit.
  • It shows strength and direction of linear relationship.
  • It is independent of change of origin and scale.

Properties of regression coefficients:

  • Both regression coefficients have the same sign.
  • The product of two regression coefficients is \(r^2\).
  • Regression coefficients are independent of change of origin but not of scale.
  • If one regression coefficient is greater than 1, the other must be less than 1.
3

Correlation Coefficient and Regression Coefficient

Distinguish between correlation coefficient and regression coefficient and write its importance in engineering.

Answer

Correlation CoefficientRegression Coefficient
It measures degree and direction of linear relationship.It measures change in predicted dependent variable for one unit change in independent variable.
It is denoted by \(r\).It is usually denoted by \(b\), \(B\), or regression coefficient.
It always lies between \(-1\) and \(1\).It may take any real value.
It has no unit.It depends on the units of variables.

Importance in engineering: These tools help in studying relationships, estimating values, predicting performance, studying quality and making data-based decisions.

4

Simple Correlation and Simple Regression

Define simple correlation, simple regression and their use for an engineer.

Answer

Simple correlation studies the degree and direction of relationship between two variables.

Simple regression gives an approximate mathematical equation to predict one dependent variable from one independent variable.

\[\hat{Y}=A+BX\]

Use in engineering: prediction of strength, cost, traffic, load, deflection, performance, production, power consumption and other measurable quantities.

5

Properties of Correlation Coefficient and Application of \(r^2\)

Mention important properties of correlation coefficient. What is the application of coefficient of determination \(r^2\)?

Answer

  • \(r\) lies between \(-1\) and \(1\).
  • \(r\) is positive for same-direction movement and negative for opposite-direction movement.
  • \(r=0\) means no linear correlation.
  • \(r\) has no unit.
  • \(r\) is independent of change of origin and scale.

The application of \(r^2\) is to measure the proportion of variation in the dependent variable explained by the independent variable according to the fitted model.

6

Correlation and Regression

Distinguish between correlation and regression.

Answer

CorrelationRegression
Correlation measures degree and direction of relationship between two variables.Regression gives an equation for predicting one variable from another variable.
It does not identify dependent and independent variables.It identifies dependent and independent variables.
It is mainly used to study relationship.It is mainly used for estimation and prediction.
The correlation coefficient has no unit.The regression coefficient depends on the units of variables.
The value of \(r\) lies between \(-1\) and \(1\).The regression coefficient may take any real value.
7

Karl Pearson’s Coefficient of Correlation

Define Karl Pearson’s coefficient of correlation.

Answer

Karl Pearson’s coefficient of correlation is a numerical measure of the strength and direction of linear relationship between two variables.

\[ r= rac{n\sum XY-\sum X\sum Y}{\sqrt{\left[n\sum X^2-(\sum X)^2 ight]\left[n\sum Y^2-(\sum Y)^2 ight]}} \]

Its value lies between \(-1\) and \(1\). Positive value indicates same-direction movement and negative value indicates opposite-direction movement.

8

Regression Coefficients

What are the two regression coefficients and what do they represent?

Answer

The two regression coefficients are:

  • Regression coefficient of \(Y\) on \(X\), denoted by \(b_{yx}\).
  • Regression coefficient of \(X\) on \(Y\), denoted by \(b_{xy}\).

\(b_{yx}\) represents the change in predicted \(Y\) for one unit change in \(X\). \(b_{xy}\) represents the change in predicted \(X\) for one unit change in \(Y\).

9

Properties of Regression Coefficients

Write the properties of regression coefficients.

Answer

  • Both regression coefficients have the same sign.
  • If one regression coefficient is positive, the other is also positive.
  • If one regression coefficient is negative, the other is also negative.
  • The product of two regression coefficients is \(r^2\).

\[b_{xy}b_{yx}=r^2\]

  • Regression coefficients are independent of change of origin but not of scale.
  • If one regression coefficient is greater than 1, the other must be less than 1.
  • Correlation coefficient is the geometric mean of the two regression coefficients with proper sign.
10

Regression Lines

What are regression lines?

Answer

Regression lines are lines of best fit used to estimate one variable from another.

There are two regression lines in simple regression:

  • Regression line of \(Y\) on \(X\), used to estimate \(Y\) from \(X\).
  • Regression line of \(X\) on \(Y\), used to estimate \(X\) from \(Y\).

If correlation is perfect, the two regression lines coincide. If correlation is not perfect, the two regression lines are different.

11

Multiple Regression

What is multiple regression?

Answer

Multiple regression is a statistical method used to predict one dependent variable from two or more independent variables.

For two independent variables, the regression plane is:

\[\hat{Y}=A+BX_1+CX_2\]

Here \(B\) is interpreted by keeping \(X_2\) constant, and \(C\) is interpreted by keeping \(X_1\) constant.

12

Interpretation in Multiple Regression

How should regression coefficients be interpreted in multiple regression?

Answer

In multiple regression, coefficients must be interpreted by keeping the other independent variables constant.

If:

\[\hat{Y}=A+BX_1+CX_2\]

  • \(B\) means predicted \(Y\) changes by \(B\) units when \(X_1\) increases by one unit, keeping \(X_2\) constant.
  • \(C\) means predicted \(Y\) changes by \(C\) units when \(X_2\) increases by one unit, keeping \(X_1\) constant.
13

Predicted Value and Observed Value

Why may a predicted value not match the observed value exactly?

Answer

A regression equation gives an estimated value from the fitted trend. It is based on all observations, not only one observation.

Because real data contain variation, the predicted value \(\hat{Y}\) may not be exactly equal to the observed value \(Y\).

Regression is used for estimation. It does not guarantee exact prediction for every observation.
14

Common Mistakes in Correlation and Regression

Write common mistakes in correlation and regression.

Answer

  • Writing correlation as causation.
  • Writing coefficient of determination as cause instead of explained variation.
  • Using \(Y\) instead of \(\hat{Y}\) for predicted value.
  • Forgetting “keeping the other variable constant” in multiple regression.
  • Interpreting a negative intercept as a real practical value without checking the context.
  • Writing regression coefficient as if it is always between \(-1\) and \(1\).
  • Confusing correlation coefficient with regression coefficient.
Formulae

Available Formula Sheets

⌂ Home
Latest
Latest Content Auto
Home