The 5 Basic Statistics Concepts Data Scientists Need to Know

In data science, probability is the percent chance that something will happen. A zero(0) in this case means the event will not occur, while the digit 1 denotes that we are certain it will happen.

The common probability distributions are:

1. Uniform Distribution: It is a simple off or on distribution, where anything outside the given range is 0.
2. Normal (or Gaussian) Distribution: This distribution has the same standard deviation in all directions. We get to know the average dataset value along with the spread of the data.
3. Poisson Distribution: This is similar to Poisson Distribution but also has skewness, in which the variation tells about the spread of the data in different directions.

The process of reduction in the number of dimensions (or feature variables) in datasets is known as Dimensionality Reduction.

If a cube has 1000 points, we can reduce its dimensionality by simply taking the 3D data and viewing it as a 2D model. We can also remove feature variables to reduce the data volume. This is generally done with features that have a low correlation with the dataset and is called feature pruning.

Sometimes if we want to compare two datasets, or classify datasets that have an uneven number of samples for different sides or types. Just by taking fewer samples (undersampling), one can even out a dataset.

Oversampling is a way to copy datasets to have the same number of examples as the other class. The copies are produced maintaining the distribution ratio.

Based on the concept of probability, Bayesian Statistics computes and analyzes prior data to forecast the future trend. If there is a specific change in the present, the prior data will not reflect that.

Frequency analysis, therefore, is computing the likelihood of a specific occurrence, where new information isn’t computed.