Ordinary least squares, or linear least squares, estimates the parameters in a regression model by minimizing the sum of the squared residuals.

What is Ordinary Least Square Method?

OLS Assumptions are the Conditions that we need to consider them before performing Regression Analysis.Some OLS Assumptions are:  <ol><li>Linearity</li><li>No Endogeneity</li><li>Normality</li><li>Zero Mean of Error Terms</li><li>Homoscedasticity</li><li>No Autocorrelation</li><li>No Multicollinearity</li></ol>

Assumptions for OLS

<ul><li>A Linear Regression is Linear because the Equation is Linear.</li><li>To verify the Linearity between Independent and Dependent Variable apply Scatter plot.</li><li>If the Data points form a pattern that looks like a Straight Line then Linear Regression is Suitable.</li></ul>

1. Linearity

If the pattern doesn't looks like a Straight Line, then we need to apply<ul><li class="ql-align-justify">A Non Linear Regression or</li><li class="ql-align-justify">Exponential Transformation or</li><li class="ql-align-justify">Log Transformation.</li></ul>

Non-Linear Pattern

<ul><li>Technically, Endogeneity occurs when a predictor variable (x) in a regression model is correlated with the error term (e) in the model.</li><li>This usually occurs due to the Omitted Variable Bias (when we forget to include a relevant variable in the Model).</li><li>As the thing that we don’t explain with our model goes into the error, the error would be correlated to the variable that we have omitted.</li><li>Omitted Variable Bias is hard to find. When in doubt, include the Variable in the model and try it yourself. </li></ul>

2. No Endogeneity

<ul><li>We need to consider that our Error Term is normally distributed.</li><li>We only need Normal Distribution while making Statistical Inferences.</li><li>T-tests and F-tests work because we have assumed normality.</li><li>If Error Term isn't normally distributed then we can use Central Limit Theorem.</li></ul>

3. Normality of Error Terms

<ul><li>If the Mean of the Error Terms is not expected to be zero then the line is not the Best fitting one.</li><li>Having an Intercept Solves the problem. </li></ul>

4. Zero Mean of Error Terms

<ul><li>Homoscedasticity means to have “Equal Variance”. The error terms should have equal variance with one another.</li><li>When the error terms don’t have “Equal Variance”, then Heteroscedasticity happens.</li></ul>

5. Homoscedasticity

In order to prevent Heteroscedasticity, we need to<ul><li>Look for Omitted Variable Bias.</li><li>Look for Outliers.</li><li>Apply log Transformation.</li></ul>

Heteroscedasticity 

<ul><li>Autocorrelation is a mathematical representation of the degree of similarity between a given time series and a lagged version of itself over successive time intervals.</li><li>It is similar to the Correlation between two different Time Series but Autocorrelation uses same time series twice.</li><li>One is Original form and other one is Lagged version of Original form.</li></ul>

6. No Autocorrelation 

To detect autocorrelation<ul><li>Plot all points and check for patterns or</li><li>Use Durbin - Watson test.</li></ul>There is no remedy for Autocorrelation. Instead of linear regression, we can use<ul><li>Autoregressive Models.</li><li>Moving Average Models.</li></ul>

Autocorrelation Detection

<ul><li>We observe Multicollinearity when two or more Independent Variables in a model are “Highly Correlated”.</li><li>Correlation could be Positive (or) Negative.</li><li>This will lead result in less reliable predictions.</li><li>Multicollinearity can be noticed easily by finding the correlation between each two pairs of Independent Variables.</li></ul>

7. No Multicollinearity

To fix Multicollinearity: <ul><li>If the Dataset is small, we can drop one independent variable.</li><li>If the Dataset is large, we will use Ridge and Lasso Regression.</li></ul>

Fixing Multicollinearity

These are some of the Assumptions to be pondered while Applying Ordinary Least Square Method and Performing Regression Analysis.

Education

OLS (Ordinary Least Square) Assumptions

Averages, relationships, trends and graphs are not what they seem. Statistics can make an important fact look as if it is nothing noteworthy and a trivial fact like there may be more in it. Writers need statistical methods and statistical terms when reporting data of social and economic trends, but they need to use the words honestly, and readers need an understanding of what they mean. Otherwise, it may result in semantic absurdity.

The abused statistic

“The secret language of statistics, so appealing in a fact-minded culture, is employed to sensationalize, inflate, confuse, and oversimplify.”

For any statistic to be worth it, it should come from a representative sample of the group it is describing. But a representative sample is near impossible because of lack of time, resources, and other practical reasons. As a result, you are more likely to see a subset based on biases from the method of sampling rather than the representative group itself. When you see the statistic that, for example, represent the entire nation, ask yourself, “Which part of the nation?”

The representative sample

There are three common kinds of average: 
<ol>
<li>Mean. Add up all the values and divide by the quantity of the values.</li>
<li>Median. The value in the middle of the sample.</li>
<li>Mode. The most common value. </li>
</ol>
When a statistic cites the “average”, it could be the mean, median or mode, which can be drastically different. The unspecified “average” is used to prove a point while leaving out any numbers that might question their predetermined point.

The kind of average

If you want to use a test to measure something or somebody, it is equally important to know the limitations of the test. What can it not measure?  For example, an intelligence test does not measure leadership, creative imagination, diligence, and emotional balance. The test may still be useful for admission to a university or a job, but you should also understand its limitations.

Consider what it really measures

“As long as the errors remain one-sided, it is not easy to attribute them to bungling or accident.” Statistics are often used to support a cause or agenda a person believes in, rather than to disprove them. When people err in their use of statistics, it is always in the direction of their beliefs or agenda. So, instead of getting an unbiased view of the world, we get a biased view that other people want us to see.

Statistics are always biased

“‘Does it make sense?’ will often cut a statistic down to size when the whole rigmarole is based on an unproved assumption.” Consider what they are saying and why they are saying it. If a statistic doesn’t seem right, or someone arrives at a conclusion that seems questionable, ask “Does it make sense”.

The common-sense test

It describes using one thing as a way to claim proof of something else, even if there exists no correlation between the two.Meaningless statements:<ul><li>“32% of doctors think Mercedes vehicles are safe."</li><li>“27 % of a large sample of eminent physicians smoke Throaties— more than any other brand.”</li></ul>Expressing the data to suit your view. For instance, express exactly the same fact by calling it a one % return on sales, a fifteen % return on investment, a ten-million-dollar profit, or a decrease of sixty % from last year.

The semidetached figure

If B follows A, then A has caused B.For example:<ul><li>Assuming that since smoking and low grades go together, smoking causes low grades.</li><li>If you attend college, you will earn more money than if you decide to use the next four years in another way. The unwarranted assumption is that since college-trained people make more money, they make it because they went to college. However, we don’t know if these people would have made more money had they not gone to college.</li></ul>

Correlation vs. Causation

<ol><li>Who says so? What is the agenda? Who will benefit from presenting the data that way? Does it tell the whole story or only the good or bad part? Does it misuse average? Does any bias come into play?</li><li>How do they know? Where did the data come from? Is it representative? Can it be biased? Is the sample big enough?</li><li>What’s missing? </li><li>Did someone change the subject between the raw data and the conclusion? Is it a real statistic, or what someone reported?</li><li>Does it make sense? Pay attention if it fails the common sense test. Is something complex reduced to a single number?</li></ol>

How to avoid blind belief in statistics

This book lays the groundwork for the many ways statistics are used to mislead people. Honest people must learn the tricks of statistics so that they can interpret data intelligently.

Over Half a Million Copies Sold--an Honest-to-Goodness Bestseller Darrell Huff runs the gamut of every popularly used type of statistic, probes such things as the sample study, the tabulation method, the interview technique, or the way the results are derived from the figures, and points up the countless number of dodges which are used to full rather than to inform.

Stashed ideas

How to Lie with Statistics

Becoming a Data Scientists is an exciting path, but you cannot learn data science within one year or six months—instead, it’s a lifetime process that you have to follow with proper dedication and hard work. 
To guide your journey, the skills outlined here are the first you must acquire to become a data scientist.

Becoming A Data Scientist

Python is the most preferred coding language and is adopted by most Data Scientists. It is easy to understand, versatile, and supports various in-built libraries such as Numpy, Pandas, MatplotLib, Seaborn, Scipy, and many more.<ul><li>FreeCodeCamp’s Python Tutorial (Recommended)</li><li>Kaggle’s Python Course</li><li>Krish Naik’s Python Tutorial (Recommended)</li><li>Udemy’s Python for Data Science and Machine Learning Bootcamp</li><li>Coursera Python Course</li></ul>While learning Python, one should know essential Python variables, data types, OOPs concepts, Numpy, Pandas, Matplotlib, and Seaborn.

Choose A Programming Language (Python / R)

For becoming a Data Scientist, having knowledge of statistics and probability is as essential as having salt in food. Knowing them will help the data scientists interpret large data sets, get insights from them, and analyze them better.<ul><li>Krish Naik’s Statistics Playlist (Recommended)</li><li>Coursera Statistics Course</li><li>Khan Academy Statistics And Probability Course</li><li>FreeCodeCamp Statistics Course (Recommended)</li></ul>

Knowledge Of Statistics And Probability

Structured Query Language (SQL) is used for extracting and communicating with large databases. One should focus on understanding the different types of normalization, writing nested queries, using co-related questions, group-by, performing join operations, etc., on the data and extract in raw format. This data will then further be cleaned either in Microsoft Excel or by using Python libraries.<ul><li>Freecodecamp SQL (Recommended)</li><li>Intro To SQL By Kaggle (Recommended)</li><li>Advanced SQL By kaggle</li><li>Edureka’s SQL Playlist</li></ul>

Learn SQL

When a Data Scientist is given a project, the majority of the time goes into cleaning the data set, removing unwanted values, handling missing values. It can be achieved by using some inbuilt python libraries like Pandas and Numpy.One should also know how to manipulate data using Microsoft Excel.<ul><li>Blog — Cleaning Data Using Python (Recommended)</li><li>Edureka’s Microsoft Excel Course</li><li>Learning Pandas By Kaggle (Recommended)</li></ul>

Data Cleaning

Exploratory data analysis is the essential part when talking about data science. The data scientist has many tasks, including finding data patterns, analyzing data, finding the appropriate trends in the data and obtaining valuable insights, etc., from them with the help of various graphical and statistical methods, including:A) Data Analysis using Pandas and NumpyB) Data ManipulationC) Data Visualization<ul><li>Intro To EDA By Code Heroku’s (Recommended)</li><li>Blog — Performing EDA on Iris Data Set (Recommended)</li><li>Coursera Course On EDA, Statistics, Probability</li></ul>

Exploratory Data Analysis

According to Google, “Machine learning is a method of data analysis that automates analytical model building. It is a branch of artificial intelligence based on the idea that systems can learn from data, identify patterns and make decisions with minimal human intervention.”It is the most crucial step in a life cycle of a data scientist where one has to build various models using machine learning algorithms and should be able to predict and come with the most optimum solution to solve any problem.<ul><li>Machine Learning By Andrew NG (Recommended)</li><li>Deep Learning By Krish Naik</li></ul>

Learn Machine Learning Algorithms

After acquiring the basics of Data Science, now it’s time to get hands-on experience in its part. There are many online platforms, like Kaggle and Analytics Vidhya, that can provide you with hands-on experience with both beginner and advanced level data sets. They can help you to understand various machine learning algorithms, different analyzing techniques, etc.

Practice On Analytics Vidhya and Kaggle

Becoming a Data Scientists is an exciting path, but you cannot learn data science within one year or six months—instead, it’s a lifetime process that you have to follow with proper dedication and hard work. To guide your journey, the skills outlined here are the first you must acquire to…

Bite-sized knowledge
to upgrade
your career

#statistics

IDEAS ABOUT #statistics

OLS (Ordinary Least Square) Assumptions

How to Lie with Statistics

It's time to

READ

LIKE

A PRO!

A Guide On How To Become A Data Scientist (Step By Step Approach) - KDnuggets

Bite-sized knowledgeto upgradeyour career

#statistics

IDEAS ABOUT #statistics

OLS (Ordinary Least Square) Assumptions

How to Lie with Statistics

It's time to

READ

LIKE

A PRO!

A Guide On How To Become A Data Scientist (Step By Step Approach) - KDnuggets

Bite-sized knowledge
to upgrade
your career