OLS (Ordinary Least Square) Assumptions - Deepstash

Bite-sized knowledge

to upgrade

your career

Ideas from books, articles & podcasts.

created 13 ideas

These are some of the Assumptions to be pondered while Applying Ordinary Least Square Method and Performing Regression Analysis.

OLS (Ordinary Least Square) Assumptions

📘 Learning Linear Regression

STASHED IN:

81 reads

Ordinary least squares, or linear least squares, estimates the parameters in a regression model by minimizing the sum of the squared residuals.

OLS Assumptions are the Conditions that we need to consider them before performing Regression Analysis.

Some OLS Assumptions are:  

  1. Linearity
  2. No Endogeneity
  3. Normality
  4. Zero Mean of Error Terms
  5. Homoscedasticity

1. Linearity

  • A Linear Regression is Linear because the Equation is Linear.
  • To verify the Linearity between Independent and Dependent Variable apply Scatter plot.
  • If the Data points form a pattern that looks like a Straight Line then Linear Regression is Suitable.

If the pattern doesn't looks like a Straight Line, then we need to apply

  • A Non Linear Regression or
  • Exponential Transformation or
  • Log Transformation.

  • Technically, Endogeneity occurs when a predictor variable (x) in a regression model is correlated with the error term (e) in the model.
  • This usually occurs due to the Omitted Variable Bias (when we forget to include a releva...

3. Normality of Error Terms

  • We need to consider that our Error Term is normally distributed.
  • We only need Normal Distribution while making Statistical Inferences.
  • T-tests and F-tests work because we have assumed normality.
  • If Error Term isn't normally distribute...

  • If the Mean of the Error Terms is not expected to be zero then the line is not the Best fitting one.
  • Having an Intercept Solves the problem. 

5. Homoscedasticity

  • Homoscedasticity means to have “Equal Variance”. The error terms should have equal variance with one another.
  • When the error terms don’t have “Equal Variance”, then Heteroscedasticity happens.

In order to prevent Heteroscedasticity, we need to

  • Look for Omitted Variable Bias.
  • Look for Outliers.
  • Apply log Transformation.

6. No Autocorrelation

  • Autocorrelation is a mathematical representation of the degree of similarity between a given time series and a lagged version of itself over successive time intervals.
  • It is similar to the Correlation between two different Time Series but Autocorrela...

Autocorrelation Detection

To detect autocorrelation

  • Plot all points and check for patterns or
  • Use Durbin - Watson test.

There is no remedy for Autocorrelation. Instead of linear regression, we can use

  • Autoregressive Models.
  • Moving Average Models.

7. No Multicollinearity

  • We observe Multicollinearity when two or more Independent Variables in a model are “Highly Correlated”.
  • Correlation could be Positive (or) Negative.
  • This will lead result in less reliable predictions.
  • Multicollineari...

To fix Multicollinearity: 

  • If the Dataset is small, we can drop one independent variable.
  • If the Dataset is large, we will use Ridge and Lasso Regression.

1 Reaction

Comment

It's time to

READ

LIKE

A PRO!

Jump-start your

reading habits

, gather your

knowledge

,

remember what you read

and stay ahead of the crowd!

Takes just 5 minutes a day.


TRY THE DEEPSTASH APP

+2M Installs

4.7 App Score