Hi everyone. Welcome to the last unit of the SAP Data Intelligence for Enterprise AI
openSAP course. SAP Data Intelligence is the platform
to scale machine learning for the enterprise. With SAP AutoML,
we want to progressively automate as much of the ML workflow as we can,
to enable ML for everyone. What's the process behind that?
Basically, our AutoML solution consists of three main ingredients.
First, we spin up a portfolio of many different canned methods,
for each step in a machine learning pipeline, like data preparation, feature engineering,
and, naturally, the selection of an ML algorithm and its hyperparameters.
This is our configurable toolbox, which we are constantly extending.
We then use an algorithm to decide which parts of this toolbox should be used
to generate a pipeline that achieves the best performance on the task you defined.
Last but not least, we leverage the compute available in the cloud,
to try many different combinations of pipeline elements, to find the best ML pipeline for the respective dataset.
In each iteration, the machine learning algorithm controlling this process
learns something new, until the budget we defined is depleted.
We actually use machine learning to train machine learning models.
Let's see a demo of AutoML. Here you can see this scenario, Payment Risk Prediction,
that we have developed, and we want to define a dataset for it.
So the dataset that I want to choose is payment_risk_training,
and I want to use the payment_risk_test for my testing. Then I can define the experiment target,
which is provided to me as a dropdown menu, and I can choose the column,
which serves as a target label for me. So then, the task that I want to define is a classification.
And basically, the strategy that I want to use to split my data is a holdout set,
and the metric that I want to use to measure the performance is the log loss that I define here.
There are also some other, for instance, metrics, or strategies for you,
maybe cross validation, or other metrics that are available to you,
and you can choose them from the dropdown menu. After that, in the budget step,
you need to define what budget you have, for instance, how much time you want to assign
to that process, let's say 30 minutes, or you're okay if your, for instance,
training takes about a day, or a couple of days, or the number of workers that you want to work
on this single process. So, for instance, I want to use five workers in this case,
but I can also define the number of them, increase, or decrease them.
Then, I can see an overview of what I have. So this is the dataset that I'm going to use,
the objective that I have defined, for instance, this is a classification task
that I want to run, I want to use a holdout set for the strategy
of splitting my data, and the metric I want to use is log loss.
And then this is the budget that I want to assign to running this data,
and running my models on top of this data. Then, the execution has been scheduled,
and after that is completed, it will show me the result of the execution.
So as you can see here, there are different models that ran in parallel
in order to find the best performance for me, XGBoost, TensorFlow,
for instance, LightGBM, and so on and so forth. And you can see what has actually had
the best result for me, for instance, Random Forest Classifier
had the best classification result for me, had the least amount of log loss,
and the training duration for it was around five minutes. So if I want to learn more about that, I can click on it,
and there I have my pipeline layout, with more information around that process,
for instance, I read that data, then I looked at the column that I have,
I removed zero variance columns, and then I, for instance, did some pre- processing,
I imputed the null values with the most frequent value in that column.
I did some data encoding, and then I also did some categorical encoding.
So, as you saw, I didn't really do this, but the machine learning,
the AutoML, actually took care of all that for me. Then, I didn't do much of a normalization,
because I didn't need to bring all my data on a single scale,
but after I did all the pre-processing, and the feature engineering was completed,
then my prediction ran, and the random forest was where I got the best result.
So as you can see, I just defined a dataset, I just defined a label,
and the strategies that I wanted to use, and then it actually defined the best algorithm for me,
and also I can click on it, and look at the hyperparameters that return the best result,
for instance, the number of trees that are used in the random forest,
or the maximum depth that it actually processes, or the feature fraction, minimum samples for split,
and so on and so forth. And, after I am happy with this result,
I can just deploy that, and apply it on, basically, the unseen data,
and you can consume that as a REST API in your business application.
So, let's learn more about how to get started, and where to find more information on SAP Data Intelligence.
We have a product Web site that has a lot of information
about SAP Data Intelligence. So, when you come here, you get information about the product,
about the support, about the training, and so on and so forth,
for instance, you can go to the features, and there are some features available to you,
for instance, you can look at, explore some of the use cases,
you'll want to find a similar use case to your own industry, or the problems that you want to solve,
and let's say that, for instance, I want to explore some of the use cases,
and there are some use cases available to me, for instance, I want to look at the predictive quality,
and there I can see some information about the story that we developed with one of our customers,
and so how we developed that, the challenges that we faced,
and then how we can dig deeper, there is a demo, a webinar, for it,
where you can go and learn more about that. Also, you can get more information
about some technical information, for instance, the documentation of SAP Data Intelligence,
how to get started with it, the modeling, how you can install it,
how you can provision it on the SAP Cloud Platform, and basically everything that you need to know
in order to get started with it. You can also go directly to the SAP Help Portal,
or the solution brief, or SAP PartnerEdge, if you are a partner,
and also go to SAP Cloud Platform cockpit, to learn how to provision an SAP Data Intelligence instance.
As we mentioned, SAP Data Intelligence is developed by two groups at SAP,
which have standalone products, as SAP Data Hub,
and SAP Leonardo Machine Learning Foundation. To learn more about these two products
and their capabilities, I invite you to watch their openSAP courses.
The SAP Data Hub openSAP course is offered by my colleague, Tobias Koebler.
By watching this course, you will learn more about the pipeline modeler, the Kubernetes technology
behind the metadata management, and so on. Also, to learn more about the
SAP Leonardo Machine Learning Foundation and capabilities, you can check our Web site or online courses,
which we have provided links to here. Congratulations on completing
the SAP Data Intelligence openSAP course. Hope you enjoyed this course,
and good luck with your course assignment.