Correlation and Regression: Meaning, History and Visual Understanding
Before using formulae, understand what correlation and regression actually mean. Correlation measures relationship. Regression gives a model for prediction.
Video Explanation
Watch this video for a quick introduction to correlation, regression, dependent variable, independent variable and coefficient of determination.
1. Meaning of Correlation
Correlation shows how two variables are related. It tells the strength and direction of the relation between them.
Correlation means the degree and direction of relationship between two variables.
Degree means strength. Direction means whether the variables move in the same direction or in opposite directions.
Degree
Strength of Relationship
Degree shows how close the relation is.
The relation may be weak, mild, strong or very strong.
Direction
Same or Opposite Movement
Positive correlation means the variables move in the same direction.
Negative correlation means the variables move in opposite directions.
In positive correlation, both variables may increase together or both may decrease together.
The main idea is same-direction movement, not only increase.
In negative correlation, one variable tends to increase while the other tends to decrease.
This is opposite-direction movement.
Exam answer:
Correlation is a statistical measure that shows the degree and direction of relationship between two variables.
Positive correlation means same-direction movement, while negative correlation means opposite-direction movement.
2. Five Scatter Diagram Possibilities
A scatter diagram gives a visual idea of the relationship between two variables.
The points may show positive relation, negative relation, no clear relation, or perfect relation.
Perfect Positive
Positive
No Correlation
Negative
Perfect Negative
3. Range and Interpretation of \(r\)
The correlation coefficient \(r\) always lies between \(-1\) and \(+1\).
The sign gives direction and the numerical size gives strength.
Range of Correlation Coefficient
\[
-1\le r\le 1
\]
\(-1\)
\(-0.9\)
\(-0.7\)
\(-0.4\)
\(0\)
\(0.4\)
\(0.7\)
\(0.9\)
\(1\)
Perfect negative
Very strong negative
Strong negative
Weak / mild negative
No linear correlation
Weak / mild positive
Strong positive
Very strong positive
Perfect positive
Negative correlation means opposite-direction movement.
Positive correlation means same-direction movement.
Writing habit:
Do not write only the value of \(r\). Also write the direction and strength of the relationship.
4. Dependent and Independent Variables
In regression, one variable is predicted using another variable.
So we must clearly identify the dependent and independent variables.
Dependent Variable
Variable to be Predicted
The dependent variable is what we need to predict in an experiment.
It is usually denoted by \(Y\).
Independent Variable
Variable Used for Prediction
The independent variable is used to predict or explain the dependent variable.
It is usually denoted by \(X\).
Example: If age is used to predict blood pressure, then age is the independent variable and blood pressure is the dependent variable.
Exam answer:
The dependent variable is the variable whose value is to be predicted or estimated.
The independent variable is the variable used to make that prediction.
5. Coefficient of Determination
Correlation gives strength and direction. Coefficient of determination gives the percentage of variation explained by the model.
Coefficient of Determination
\[
COD=r^2\times100\%
\]
If the coefficient of determination between age and blood pressure is \(82\%\), then age explains about \(82\%\) of the variation in blood pressure according to the fitted model.
It does not mean age is the only cause of blood pressure.
Exam answer:
The coefficient of determination is \(r^2\times100\%\). It gives the percentage of variation in the dependent variable explained by the independent variable through the regression model.
6. Meaning of Regression
Correlation tells whether variables are related. Regression gives an approximate equation for prediction.
History note:
The word regression became famous through Francis Galton’s biological study of heredity.
He observed that extreme measurements tended to move back toward the average in the next generation.
This idea became known as regression toward the mean.
Regression is an approximate mathematical model used to predict the dependent variable from one or more independent variables.
Simple Regression Form
\[
\hat{Y}=a+bX
\]
Here, \(\hat{Y}\) means the predicted value of \(Y\).
A regression equation is not a perfect rule. It is a best-fitting approximate relation based on the given data.
Exam answer:
Regression is a statistical method used to estimate or predict the value of a dependent variable from one or more independent variables.
In simple linear regression, the predicted value is written as \(\hat{Y}=a+bX\), where \(\hat{Y}\) is the predicted value.
Final Summary
Remember These Lines
Correlation gives the degree and direction of relationship.
Positive correlation means same-direction movement. Negative correlation means opposite-direction movement.
The dependent variable is what we need to predict in an experiment.
Regression gives an approximate model for prediction.