Hi and welcome to unit five of the SAP Data Intelligence openSAP course.
In this unit, we will focus more on the SAP Data Intelligence's launchpad
and components, and we'll see a demo of the system and the way we can extend its applications.
SAP Data Intelligence launchpad follows the SAP Fiori design patterns.
It is the single point of entry for all of its applications and components.
It is extensible, and is modifiable for user context and authorizations.
Here you can see a view of all of the SAP Data Intelligence's launchpad components.
We also mentioned some of these components in the previous unit.
Components such as Connection Management, Metadata Manager, Vora Tools, Pipeline Modeler,
System Management, and last but not least, the ML Scenario Manager.
Let's see a demo on these components. As you can see here,
this is the SAP Data Intelligence launchpad, where you have access to all the components
that are available to you through SAP Data Intelligence. You can also add a quick link to an application
that you have installed on top of Data Intelligence, or that you have added to it,
through that add button. You also have a question mark on top of the page,
where it allows you to, for instance, get some more information
about your SAP Data Intelligence, for instance, what version it is, a Kubernetes version.
And also, if you click on it, you can, basically, go to the help documentation
where it has more information about all the different components
that run on SAP Data Intelligence. So for instance, it takes you
to the SAP Data Intelligence product page on the SAP Help Portal, help.sap.com.
Now, let's take a look at the Connection Management. As you can see, for instance,
in my landscape, I have already defined some of the connections.
Also, for instance, you can see a BW connection, you can see a connection to the semantic data lake,
which comes by default with SAP Data Intelligence. I have a Kafka defined here.
And also, I can click on the Create button, and there are some connections
that are available to me natively. So for instance, GCS,
or let's say that I have, for instance, my S3 connection, or OData connection,
and by clicking on one of those, I get access to all the configurations,
for instance, ADL here, where I can go and enter the account name,
the tenant ID, and all the information that is required, in order to establish that connection.
After I'm happy with all that information, I can test my connection,
or I can build that connection, if everything goes well
and the test connection is okay, and then I have access to all the sources
through that connection. Then I can come to my Metadata Explorer,
another component, and look at that connection that I have, and also browse a catalog,
or, for instance, look at the datasets that I have, and profile those datasets,
and get some statistical information around them. So for instance, here if I click on Browse Connections,
as you can see, these are all the connections that are available to me through my landscape.
So I have the ABAP connection, I have the S3, I have my semantic data lake,
and if I drill down to that connection, for instance, I have a worm folder here,
and I have a dataset there which is called bank_cleansed. And I can click on that,
and then decide what I want to do with it. For instance, I can choose to view a fact sheet,
which has some statistical information about the dataset. For instance,
it shows me the column types that I have. I can click, for instance, on the column type here,
which is, let's say, for instance, the character, and that will show me...
And then particularly choose a column, let's say job, and then see the top 10 distinct values for that column.
Or, I can tag that dataset, for instance, say okay,
so I want to say that this dataset is related to bank, has some information about banks,
and then also needs to be cleansed, and this is something that I want a scenario
for my bank scenario, so I want to add that tag. And that way when I look for
any dataset with that particular tag, this dataset will easily come up, no matter
where that dataset actually resides. I can also see a preview of that dataset.
So I can choose, for instance, how many rows I want to see,
I can choose between 100, 500, 1,000, and also, see what my dataset looks like.
And then after I see, for instance, some null values of some data that needs some cleansing,
I can use my data preparation, the self-service data preparation feature,
which comes with SAP Data Intelligence, and allows you to interactively play with your columns
and your dataset, and depending on the type,
there are different functionalities that are available to you.
For instance, if I'm talking about the numerical values, maybe I want to convert them to text,
maybe I want to remove some duplicates. If I'm talking about, for instance, some categorical value,
let's say here, the education, I can decide how I want to replace the null values,
or how I'm going to replace particular values with some x values,
with y values, and also split the column,
or, for instance, convert that to numerical values. Some of your machine learning algorithms
may only accept some numerical values, so that makes it really easy for you.
So after you're done with that, you can basically,
anything you do on your dataset, it creates a recipe for you,
and then you can apply this recipe on the dataset that you have,
or you can take that recipe and apply it to a new dataset.
So when I say run this recipe, then I can say where I want to save the result
of this data preparation. And then, for instance, this is a dataset name,
I want to save it as a CSV file in my data lake, and I want this to overwrite
whatever data is already available there. So the other capabilities
that are available to you through Metadata Explorer, one of them is Rules and Rulebooks,
which basically allows you to define some rules on top of your dataset.
So we already have some categories for you, that are important
for looking at the data quality. For instance, accuracy, completeness,
conformity, consistency, integrity, and so on and so forth.
You can use any of these categories, or you can go and define a new category.
For instance, you define a rule that says, for instance,
I want to check the data that is a valid US country. And so the parameter that is passed through this is,
basically the country is the parameter, and then the two conditions that I define for it
are that the country is not null, or, for instance, the country
is in the set of US, USA, United States, different variations of US.
And I can also define other rules, for instance, you have all the available conditions,
is equal to, is bigger, and so on and so forth, which makes sense in some conditions.
And after you have defined those conditions, you can come and create a rulebook out of the rules
that you have, and these rules that you have created
could be assigned to any rulebook that you have. So for instance, here I assign the accuracy
and conformity rules that I have created to the rulebook that I want to use.
And also you can define the thresholds here, and say, for instance, any quality
over 80% is acceptable for me, is a pass, and anything less than that, for instance, is a fail.
And also, you can create a scorecard, where you can see the information about the data
that is coming through, and how many times it came through, look at the trends, and how many times
it failed that quality test, or how many times it passed it,
and then drill down to your dataset, and reconsider the dataset that you have.
You also have a Discovery Dashboard, where you can, for instance, get a view
of the dataset distributions, based on the type of connections that you have.
For instance, here you see a pie chart of S3, HANA database, data lake that I have.
I can also look at how many fact sheets are available for the datasets
that I have through different connections. I can look at, for instance, the labels that I have
and I can also, for instance, see how many times the data has been profiled, or indexed,
or published, or the rulebooks, or the rules that are assigned to particular datasets,
and basically get an overview of all the data that I have. So the idea of Metadata Explorer
is that you look at the data about your data, because when you're dealing with Big Data,
it is really hard to go through all these different data sources
that you have, and look at all the data that you have. So the idea is that the metadata
should be identifying enough, and descriptive enough,
that it gives you all the information you need about the data that you have.
So let's go back to the launchpad, and there you can see other components.
For instance, let's say that we want to look at the ML Scenario Manager.
So after you are happy with the dataset, then you want to develop your data scenario.
So I say that I want to create a new ML scenario, and for instance, this is the name of my ML scenario,
and I want to, for instance, get some information about account balance.
I want to develop a prediction scenario, where I want to look at some features,
and based on that, see if I can predict the account balance, based on, for instance, the demographic information
about the people. So, my scenario is called Account Balance Prediction.
And when I create it, every time I add an artifact to ML Scenario Manager,
it creates a new version for me. And the good thing about this
is that you have all the artifacts related to one single scenario, data science scenario,
in one single place. So you can register your dataset,
but also you get access to your Jupyter Notebook
where you ran your experiments. So for instance, here we clicked on the Jupyter Notebook
we have developed as part of that Scenario Manager. And then I go to my Jupyter Lab,
and I can choose, for instance, Python kernel two or three,
and all the libraries through Python will be available there for visualization,
or running some machine learning algorithms like scikit-learn and so on.
And after I'm happy with developing that experiment, I can use something which is called my modeler,
in order to operationalize what I have done. We talked a little bit about the pipeline modeler
and, basically, the features that are available to you through the pipeline modeler.
So we have about 250 graphs, that are predefined
and come with SAP Data Intelligence. And, for instance, you can use those prebuilt
or predefined graphs, in order to learn how to use different operators
as part of Data Intelligence. And also, you can see the categories here.
So if you see it's very crowded, and too many items, you can basically check
or uncheck some categories, and focus on one singular category.
But also you can look at the operators, we have about 300 operators
that come with SAP Data Intelligence. For instance, here I want to look at some categories,
for instance, on Vora, what operators are available for me, in order to work with my Vora,
which is a scalable relational database for SAP Data Intelligence.
Or I want to look at the operators that are available through data quality,
which is, for instance, validation rule, anonymization, and things like that.
Or for instance, I want to use MLF Functional Services, and we will talk about all that.
This is just an overview, for you to see what is available to you through SAP Data Intelligence.
Other connectivities, for instance, you can connect to Oracle Database,
or SQL Server, or for instance, you can connect to DB2, or Google BigQuery,
and the good thing about this is that you build that connection, and you bring this data,
and then you go through them, you can use these connections multiple times.
When you drag and drop an operator into your pipeline, you can access the documentation on the right- hand side.
For instance, Validation Rule here. Just wanted to show you can access the documentation there,
and of course, you can click on that question mark on the top right of the page,
and then access that documentation that way as well. So it has information about how to use that operator,
what are the parameters, and even examples of how to use them.
So you can use these pipelines in order to train your system,
and also, after you're happy with the training, the result will be saved in your ML Scenario Manager.
Let's say as the metrics that you got out of your training, the KPIs that you got,
but also the models that you have created, will be saved as part of the ML Scenario Manager.
And then you can deploy that, and then it provides you with a REST API,
which you can consume as part of your application. Another component is the monitoring,
which allows you to go and look at how the processes ran on SAP Data Intelligence.
For instance, I clicked on the things that were red, where there was a problem,
so the pipeline died. And then I can click on it,
and I can drill down to the problem, and see what actually went wrong.
The same thing with the pipelines where the execution was completed
and there was no problem. And you can also schedule this execution,
and you can monitor this execution as well. Also, if you want to run a pipeline,
you can run it as is, or you can schedule the execution of it, so that way you can take care of
the operationalization of the pipeline that you are creating as well.
Another component is basically the System Management. System Management allows you
to look at all the applications, users, files, and tenants, so for instance,
if you go under your Workspace, you can decide if you want to import some files,
or you want to create a folder. And yeah, so you can export the files
that you have, and then it exports the result in your computer.
You can also export a solution that you have created in SAP Data Intelligence,
and then import it into another system. You can also get an overview of the dataset
that you have created. And of course, you can make that available
as part of the Tenant Workspace as well. Vora Tools is the last component
we're going to talk about. Like I mentioned, Vora is a scalable, relational
database for SAP Data Intelligence, which has the capability
to run some queries in the memory or on the disk, and you can use, basically, SQL
in order to communicate with your Vora tables, and components we use,
and so on and so forth. Okay.
So let's also look at the extensibility capability which comes with SAP Data Intelligence.
We also mentioned that you can go and you can add new applications
to your SAP Data Intelligence launchpad. For instance, if I want to run a lot
of TensorFlow applications, then I want to create a TensorBoard,
then I come to my System Management, and under the apps, I say that I want to,
for instance, create a file. There I can copy and paste my JSON file,
that it have developed, for instance, for my TensorBoard,
and once I paste that, then I can run this.
And after I run this, then that TensorBoard will be available
as an application, as part of the SAP Data Intelligence. Then I can click on my TensorBoard,
and then it will take me to the application that I have added to my launchpad.
In this unit, we showed you a demo of the SAP Data Intelligence launchpad,
and introduced you to its different components. In the next unit, we will focus on one21
of the main components, ML Scenario Manager. All the best for your self-assessment quiz.