Hello and welcome in this course, we are going to learn about the techniques and the algorithms which
will help us perform predictive analytics on imbalanced datasets before starting.
You must have had a question in your mind that why do imbalanced datasets need extra attention in this
lesson?
We are going to understand the reason behind imbalanced datasets requiring an extra attention.
So without wasting any time.
Let's get started.
First of all, let us know a little about the imbalance datasets.
So an imbalance data is a data set where the number of observations for class is not distributed equally.
The distribution can vary from a slight bias to a severe imbalance.
And in some of the cases, the minority class can have huge gaps of distribution, such as a minority
class having only one percent of all the data or the majority class having 99 percent of all the data
imbalance.
Datasets pose a serious challenge for predictive modeling, as most of the machine learning algorithms
used for classification are designed around the assumption of an equal distribution for each of the
classes.
Hence, if you use the same machine learning models for imbalanced datasets also, then it might result
in very poor predictive performance, especially for the minority class.
Now let's suppose we are solving a problem where we have to detect the fraud in an organization.
And we all know that the portion of fraud in any organization is very less in general.
The frauds are very few in number.
So fraud detection is an imbalanced data problem here.
The major problem that we need to solve is to identify the frauds, whereas if you use the general and
traditional machine learning models, then these models will ignore the frauds, hence destroying the
whole problem to escape and solve these kinds of scenarios.
We have some special techniques, such as statistical sampling of data using special machine learning
models designed for imbalanced datasets.
Don't worry if you're not familiar with these concepts because we are going to learn these techniques
in the upcoming lessons.
Until then, stay motivated and happy machine learning.