Ordinary least squares, or linear least squares, estimates the parameters in a regression model by minimizing the sum of the squared residuals.

What is Ordinary Least Square Method?

OLS Assumptions are the Conditions that we need to consider them before performing Regression Analysis.Some OLS Assumptions are:  <ol><li>Linearity</li><li>No Endogeneity</li><li>Normality</li><li>Zero Mean of Error Terms</li><li>Homoscedasticity</li><li>No Autocorrelation</li><li>No Multicollinearity</li></ol>

Assumptions for OLS

<ul><li>A Linear Regression is Linear because the Equation is Linear.</li><li>To verify the Linearity between Independent and Dependent Variable apply Scatter plot.</li><li>If the Data points form a pattern that looks like a Straight Line then Linear Regression is Suitable.</li></ul>

1. Linearity

If the pattern doesn't looks like a Straight Line, then we need to apply<ul><li class="ql-align-justify">A Non Linear Regression or</li><li class="ql-align-justify">Exponential Transformation or</li><li class="ql-align-justify">Log Transformation.</li></ul>

Non-Linear Pattern

<ul><li>Technically, Endogeneity occurs when a predictor variable (x) in a regression model is correlated with the error term (e) in the model.</li><li>This usually occurs due to the Omitted Variable Bias (when we forget to include a relevant variable in the Model).</li><li>As the thing that we don’t explain with our model goes into the error, the error would be correlated to the variable that we have omitted.</li><li>Omitted Variable Bias is hard to find. When in doubt, include the Variable in the model and try it yourself. </li></ul>

2. No Endogeneity

<ul><li>We need to consider that our Error Term is normally distributed.</li><li>We only need Normal Distribution while making Statistical Inferences.</li><li>T-tests and F-tests work because we have assumed normality.</li><li>If Error Term isn't normally distributed then we can use Central Limit Theorem.</li></ul>

3. Normality of Error Terms

<ul><li>If the Mean of the Error Terms is not expected to be zero then the line is not the Best fitting one.</li><li>Having an Intercept Solves the problem. </li></ul>

4. Zero Mean of Error Terms

<ul><li>Homoscedasticity means to have “Equal Variance”. The error terms should have equal variance with one another.</li><li>When the error terms don’t have “Equal Variance”, then Heteroscedasticity happens.</li></ul>

5. Homoscedasticity

In order to prevent Heteroscedasticity, we need to<ul><li>Look for Omitted Variable Bias.</li><li>Look for Outliers.</li><li>Apply log Transformation.</li></ul>

Heteroscedasticity 

<ul><li>Autocorrelation is a mathematical representation of the degree of similarity between a given time series and a lagged version of itself over successive time intervals.</li><li>It is similar to the Correlation between two different Time Series but Autocorrelation uses same time series twice.</li><li>One is Original form and other one is Lagged version of Original form.</li></ul>

6. No Autocorrelation 

To detect autocorrelation<ul><li>Plot all points and check for patterns or</li><li>Use Durbin - Watson test.</li></ul>There is no remedy for Autocorrelation. Instead of linear regression, we can use<ul><li>Autoregressive Models.</li><li>Moving Average Models.</li></ul>

Autocorrelation Detection

<ul><li>We observe Multicollinearity when two or more Independent Variables in a model are “Highly Correlated”.</li><li>Correlation could be Positive (or) Negative.</li><li>This will lead result in less reliable predictions.</li><li>Multicollinearity can be noticed easily by finding the correlation between each two pairs of Independent Variables.</li></ul>

7. No Multicollinearity

To fix Multicollinearity: <ul><li>If the Dataset is small, we can drop one independent variable.</li><li>If the Dataset is large, we will use Ridge and Lasso Regression.</li></ul>

Fixing Multicollinearity

These are some of the Assumptions to be pondered while Applying Ordinary Least Square Method and Performing Regression Analysis.

Education

OLS (Ordinary Least Square) Assumptions

Machine Learning is the process of letting your machine use the data to learn the relationship between predictor variables and the target variable. It is one of the first steps toward becoming a data scientist.
There are two kinds of Machine Learning: supervised, and unsupervised learning. In supervised learning, there are two types: Regression and Classification.

Machine Learning Explained

<ul>
<li>Nominal: used for labelling variables(m- male and f- female)</li>
<li>Ordinal: used for measuring non-numeric with an order of the values(1-unhappy, 2-ok, 3- happy)</li>
<li>Data Cleaning: In this data set, there are 2051 rows with 80 columns. So, there were a lot of missing and null values to be clean.</li>
<li>Exploratory Data Analysis: The highest correlation between the sale price and features was the overall quality of the house.</li>
</ul>

Types Of Data

In order to predict the outcome variable (y variable) you need to turn all the object type columns into numeric in order to predict the Sale Price. So first, I converted all the ordinal columns into numeric by assigning them by numbers, and for all the other features I created dummy variables.

Feature Engineering

After creating all the features to predict the sale prices, using sklearn, we train-test-split the training data. In Regression, there are a lot of models you can choose from to get the best performing model. In this project, I ran Linear Regression, Ridge, and Lasso models.

Modelling

What is Machine Learning? Machine Learning is the process of letting your machine use the data to learn the relationship between predictor variables and the target variable. It is one of the first…

Bite-sized knowledge
to upgrade
your career

#regressionanalysis

IDEAS ABOUT #regressionanalysis

OLS (Ordinary Least Square) Assumptions

Machine Learning by Using Regression Model

It's time to

READ

LIKE

A PRO!

Bite-sized knowledgeto upgradeyour career

#regressionanalysis

IDEAS ABOUT #regressionanalysis

OLS (Ordinary Least Square) Assumptions

Machine Learning by Using Regression Model

It's time to

READ

LIKE

A PRO!

Bite-sized knowledge
to upgrade
your career