Welcome, everyone to this course
in Python programming.
My name is Saurabh and in this tutorial we are going
to focus on each and every important concept of Python.
So without wasting any time,
let us move forward and have a look at the agenda for today.
So this is what we'll be discussing.
Today, we'll begin by understand what is Python,
how it works, how we can install Python.
Then we'll understand what are the sequences
and file operations in Python.
Next up, we are going to focus on functions
and Oops concepts.
After that, we'll be working with modules
and how we can handle exceptions in Python.
Then you will be introduced to a couple of very important
libraries called NumPy and Pandas.
After that, you'll understand how you can perform
data visualization using Matlab.
Then you will understand how you can perform
data manipulation using all the three libraries
that I've just mentioned.
After that, you are going to develop
really cool applications,
one is web map using Web Maps using Folium,
another is motion detector using OpenCV.
I hope you guys found the agenda very interesting.
And Mr. Varun will be the instructor
for this particular session.
He has rich experience in working with Python.
Over to you, Varun.
Hi, everyone.
I will be your instructor for this course.
My name Varun.
Introduction to Python.
So the question is
which programming language should you start with?
I mean, of course, you have already started with one
as part of this course, but in Dave's opinion
and my opinion, it has to be Python.
Simply because it is very, very simple and easy to learn.
In a few moments when we start looking at Python code,
you will notice how it looks like English.
It doesn't even look like code.
Many people hear the word code,
they start thinking of ones and zeros
or they start thinking about oh it's going to be like
a puzzle that you have to solve.
Nothing of that sort, really.
It's just a set of detailed instructions.
Like if you can write instructions about how to send a email
also about how to send a post mail, if you can write very,
very detailed instructions about how to send a post mail,
you can probably code.
It's like that.
Next is that it is free and open source.
So in the sense that you don't have to pay anybody money
for using Python.
Now, one question might be that, hey, what's new about that?
I don't need to pay anybody to speak English,
and English is a language.
But the thing is that English, Spanish, French,
these languages, there is no creator of these languages.
They're commonly developed.
So nobody really owns English as a language.
Nobody has the right to English.
Whereas, if you look at programming languages,
they're created by a small set of people.
10, 15, 50, 100 people.
100 people is not a big group of people,
it's relatively small as compared to people
who developed English.
People who developed English words will be the authors,
the writers, and the regular folks as well.
So that number would be millions of people
who developed the language.
So open source is a concept in software programming.
I strongly suggest that you go on YouTube
and search for open source.
We'll come across some very neat documentaries,
I suggest you have a look at them.
But comparing this with sort of what is paid for,
Matlab has a version which is paid for,
which has some extra functionality
which you need to pay for.
But Python is free, completely.
A lot of Python libraries are available for free.
There is a language called Wolfram, W-O-L-F,
like a wolf, R-A-M, which is very expensive.
You need to buy a license, you can't just use it,
it's not allowed.
It's like Windows, you need to pay a license fee to use it.
Now it's a high-level language, so believe it or not,
but the first three lines over here are actually
valid Python code.
Forget the fourth one.
Some of you might be able to understand that,
but the first three lines are valid, valid Python code.
That's why Python.
You don't have to deal with the right side.
Don't think about this, this is you'll never,
almost never going to look at this and be supposed
to figure out, oh what does this mean.
You're only supposed to look at stuff like this
and say okay, what does this mean.
Then it's portable, portable being that you can use it
on multiple devices whether that's a Window's device,
or a Linux device.
Python is very compatible that way.
It will learn different devices.
It's supports different programming paradigms.
So paradigm is just a fancy word for styles.
If you are a beginning with programming,
don't bother with it unless and until you have,
you know 600 to 1,000 hours of programming under you.
Don't bother with it at all.
This is not relevant for you.
However, if you have some experience with programming
and you know what a programming paradigm is,
just know that Python supports multiple and for those of you
don't know about it, don't worry about it.
You will get there eventually.
I'm not saying it's not important, it is important,
it's just that right now it's like running.
Like right now, if writing a line of code is
the starting point, it's like stumbling or crawling,
thinking about paradigms and styles of programming
is kind of like running.
So I would not recommend thinking about it too much.
Next is Python is extensible in the sense that you can
invoke C and C++ libraries from Python.
So you can sort of call code of other languages
using Python.
That way it is very extensible,
increases the usability of it.
Makes it work with other systems.
So Python's also used widely by a lot of companies.
Let us look at a few of the giants.
So this would include YouTube, Google, Dropbox,
RaspberryPi, BitTorrent,
Nasa, Netflix, NSA,
and I mean, it's like amazing, right.
I mean, you look at how varied and wide different
these companies are.
They are internet companies to you know,
national space agencies, to national security agencies.
Really though breadth and the depth of it.
It's being used by all of them in some way or the other.
So it's not that you shouldn't say that hey Google
is built on Python.
No, Google uses 10 languages.
It's a huge company, right.
Same for Dropbox.
Oh by the way, the creator of Python works for Dropbox.
He actually does work for Dropbox and he has it in
his contract that he will devote a certain percentage
of his time, like it's about 30 or 40% of his time,
for developing Python.
So that is something that he got
as a part of his job contract.
Of course he's a very celebrated person that way.
Not regular people or I guess you as well,
can really negotiate something like that.
But yeah, he did because he's the creator of Python.
But yeah, the point is that best of the places
in Silicon Valley and government agencies,
they are using Python.
Now the best part, yes, this is my favorite part,
because Python really works for most of the thing out there
like you want to develop a website, you can use Python.
You want to do data analysis, you want to do data science,
you can use Python.
You want to do automation testing, you can use Python.
You want to do collect data from the internet,
which is web scraping, that is extract data
from the websites, you can use Python.
Now one cautious thing, you don't need to do all
of the things to say that you are a Python,
a starter Python developer.
Like for example, I've never done this one, testing.
I've not never really done scraping.
I know how it is done, but the point to note is that
there are very slight differences between these.
It is then that you have to learn an entirely new thing
then you are doing this versus this.
You have to learn the concepts related to it,
but that's more like business logic
rather than programming logic is completely changed.
Right, the golden question on a lot of people's mind,
are there Python jobs in the industry?
So this is a job trend from Indeed.com.
Don't focus on the small percentages, I know they're small,
but they are of the total number of jobs, okay.
They're not just, it says .85%, but it's of probably
10 million or 15 million job postings.
So you can notice how the trend is good for Python.
It's going a little sideways, but then it keeps doing that,
but if you compare it to PHP or C++,
definitely a huge difference.
Then of course, the popularity.
This is even more relevant,
how much is the community developing?
This indicates Google Trends would indicate both searches
using the word Python either by experience or by new people,
all by industry.
So even if I have to hire Python developers
I might go to Google and say hire Python developers
and then it will show up here.
So this is an overall accumulation.
This is a summation of how popular Python is compared
to other languages.
Like these indexes, 99, 66, so on and so forth,
these are indicative about that.
So let's look at how to install Python.
We have sort of understood till this point what is Python,
why it is used, what is programming, why it is used.
Let's look at how to get started.
Now, get started, the first thing is to of course,
buy the card, or get the Python.
You go to Python.org, you go to the downloads tab
and you download the latest version of Python.
3.6, 3.x is the latest version.
2.X is also out there.
It's being used by a lot of projects, even my projects
these days use 2.X, because they have like one,
12 to 15 months old now, the kind of projects
that I'm working on, so I mean, it isn't that you do 2.X.
It isn't that for a big (speech unclear) in Python
specifically, you would find a massive amount of difference
between a 2.X and 3.X.
I mean, it's like you know, having a car and then it's
sports version of that car or the plus version
or the luxury version of a car.
The differences are minor for the regular user,
but if you are a Python user, then of course you might find
significant differences.
But don't be confused by it.
It's not that companies tell developers who you know,
know Python 2.7 and don't know the latest version.
The job market doesn't work like that.
They will go for Python.
They will ask you if you know Python and then the version
difference doesn't really make too much of a difference.
You open the link and click on run.
Click on install now.
This is very important.
In case you have already installed Python and you forgot
or you unchecked this for some reason,
please go back, uninstall Python, and reinstall it,
and make sure that you selected this box.
Nothing will work if you don't do this.
So please, please, please, follow the step.
And that's it, you're done.
As easy as that you have Python on your system now.
There's a Python GUI developer for you.
Now next is Python IDE.
The one question is what is an IDE?
IDE is an integrated development environment.
It is used as a code editor, including a series of parallel
components in it and attachments.
So fancy words aside, if you can ask me,
hey, can I write code or can I write Python in a notepad
or a Word doc?
My answer to that question would be yes you can.
Now if you ask me should I write code in Notepad?
My answer to that would be no, please don't, never.
So you get the difference, you can do it but you should
definitely not do it.
So it's not that your code wouldn't run or something
of that sort, it's just that IDEs are tools which are meant
for doing this.
They are specialized development environments
for you to code in.
It helps you in certain ways.
That makes sense, right, because programming can be very
difficult without them.
So definitely please choose an IDE and stick to it.
One thing that you noticed, there are line numbers.
These are very, very useful by the way.
As you will experience, if you're talking to another
developer, the best way to talk to them is line number three
then edit on this particular line.
Can you start reading from this line to this line,
that's how I deal with a number of situations.
I will talk about a certain line number.
Okay, let's open something called the terminal.
So on Mac and Linux, this will be called the terminal.
On Windows this is called the CMD.
See unit two, search for CMD.
Once you install Python, what you can simply do is type
Python on the command line and press enter.
It opens the Python interpreter.
I can write Python code right here and it will work.
This is valid Python code.
It comes with a help function.
If I type help followed by braces,
it will launch help for me.
If I type in keywords, right, so there are certain things
called keywords in programming.
Let me just zoom into it, yeah.
So I can exit from Python.
Look at the definition.
Let's get started.
I will show to you this is how you can open the Python
interpreter, you just need to go on the particular page
and type Python and it will open it up.
Let's come to Pycharm for a minute.
In Pycharm, the way it works is that of course,
I already have this, but in case you would not have this.
So let's just remove it all together.
What you need to do is you need to create a new project
and then you need to give it a name.
The name could be test
and you do a create.
We do open it in the current window.
Once it's in the current window, what I can do is that
I can go to new and then I can select Python file.
I can give it a name.
Automatically creates a test as pv extension by itself
and it stores it.
Even if you go into a system and you'll see that it has
been created and you can start writing your Python code
over here.
Now programming has something called comments.
A lot of time what happens is that you will write a piece
of code but you will forget what it does.
It's not something that only you go through.
It's a standard thing.
Even if I write code I can bet my life on it
that three days later I will forgotten about 80% of it.
I would know what it does, but if it's more than
100 lines I would have to read through it.
Now what developers do for this situation is that
they add comments.
Comments are nothing but simple English explanations
for what you have done so that you don't need to remind
yourself again and again of what this particular thing does.
It just helps to keep a track of, it's like notes.
It's like small notes and the programming languages
basically ignore them.
Now there are two ways to write comments.
This is a new Python file for the class.
Now that's it.
This is not valid Python code.
This is not code, this is English.
This is one way to write it.
Other way to write it is using three single quotes
and you write it like this.
This is a new Python file for the class.
These are the two ways you can do it.
The purpose is basically to just remind yourselves
or to other developers.
So this is one thing that a lot of people when they're
starting off, your code will be viewed by other developers
at some point.
You will start sharing your code, either within a company
or otherwise.
So when they look at your code they should be able
to understand from your comments what it does.
Not just the whole file, not like a description that this
file says something to the database, but like what each
and every step is along the way.
So like, for every 10, 20 lines of code,
probably one comment should be there.
That's like a best practice.
So next is indentation.
You need to think about this in the way that Python
follows indentation like comment system
on social network websites.
So if you've ever seen Reddit,
let me just open it up actually, okay.
So if you look at Reddit, if you go to the comment section,
you will see that there's a nesting.
You can know that okay, Ahah is being responded to
by RetardedClownFace or this guy's being replied to
by this but this guy's being replied to by this.
This guy's being replied to by this guy, so on and so forth.
It's a nesting.
You can see that there is a division.
You can see that this comment is a reply to this comment.
This is something that we see everywhere.
We see this on Facebook,
we see this on Instagram, everywhere.
Now, this is how Python sort of declares blocks of code.
So in case you're not familiar with what blocks
or closures are, it's okay.
You're going to come across it when we go further in Python
but just remember that indentation is a very important
concept in Python and this is how closures
or blocks of code are defined.
So how does Python code execution happen?
You write code and it is saved with a .py extension,
like I just showed it to you.
The code is converted into Byte code for machine
to understand.
That's done by the Python compiler or the interpreter.
You don't need to worry about this step.
It happens automatically as you run the file.
It gets executed.
Just need to care about step one, two and three are taken
care of by automatically.
It's not something that you have to do much about.
Let's created a first "Hello World" program in Python.
So how do you write hello world, how do we greet the world?
Well, this is how we do it, as simple as this.
Print, open braces, then a string.
This is a string.
You can't write this, this will not work, okay.
If you see, this is already showing an error
and of course this is where Python is smart.
Python is smart.
If you notice, it is underlined this file.
It has underlined the folder containing it.
This is telling you, before you even run it,
that this is not going to work.
So anytime you're printing string, a string being anything
which is like a message, right, it needs be in strings.
Use the print statement, use the brace opening
and the brace closing like this.
If you miss the brace it won't work, you have to include it.
Then let's run it.
So the way to run it is you right click
and then you click on run hello,
where is it referenced to the name of the file.
This is where it will run.
This terminal is already present.
Hello world is printed two times.
Hello welcome to Edureka gets printed
and then something odd happens.
Happy learning welcome to Edureka.
This gets printed on two different lines.
Why is that?
We just wrote a single line.
So this is where this comes in, \n.
This is what it's called is break line character.
It will break the line.
If you're typing in a Word doc and you press enter
it goes to the next line.
It's a new line character.
If I type it two times, we should expect two spaces.
If you notice it is gone two spaces deep.
If you do it another time,
let's suppose you do it four times and you save and we run.
It goes even further.
The reason why it is not dividing in the same end
is because there's a space character here.
If I do it like then, then it'll start on the same line.
Now see, it adjusted.
So this is how you do it.
So it could be single code to double codes, doesn't matter,
but I would suggest following one of them.
This is the output you will get after running the file.
Next is variables, variables are pretty much like
mathematics variables over here.
You take a particular name that you want to give it.
So you want to set A as 10, you want to set B as edureka.
This is an integer in Python.
This is a string.
Notice that now we're using single string as compared
to double quote.
You can do print AB, A, B and notice the comma.
So ever here for example, I could have done this
and this would have worked.
So print can separate multiple strings or things
that you want to print via comma.
So you see both of them get printed.
Now let's come here and let's print this.
The 10 and edureka both get printed.
Of course they're coming with these braces.
So in case you want to just print it,
print them individually, that's a separate thing.
I mean, you will have to separate it.
But in case you just want to see the values on your terminal
this is the way to do it.
Now, another example over here is this one.
So what this will do is that it will move in order.
It will assign X to 10, y to 20, and z to 30.
Let's try it out.
Okay, now of course this is very convenient way of doing it.
Now the alternate way of doing it,
of course you can always do it like this as well.
That it increases the vertical length of your file.
I don't find it to be neat and you will see expedience
of level of doing this way.
It's just more convenient.
It just comes in a single line, that's it.
It's just a matter of convenience.
Otherwise, this is also correct and this is also correct.
Correctness is there, it's just that line number nine
is better than doing it in three lines.
This is more concise.
Now, a word about identifiers.
This is a line identifiers.
When you hear the word identifier, A and B are identifiers.
They're identifying a particular variable name for you.
Now there are certain rules around it.
It is used to identify to variable function, class,
or any other object right.
It starts with a letter A to Z or A to Z capital or small
or an underscore and followed by digits, zero to nine.
So you can't start a variable with a digit like this.
It shows an error.
What you can do it you can do this.
Or you can do this.
This works, but doing this won't work.
So this, this, _B, __b,
and all of this, they are valid identifiers.
But you can't start it with anything else apart from
letter A to Z, capital or small, underscore.
You can't start with the digits.
Python doesn't allow special symbols, like add,
dollar, percentage, star, braces,
exclamation mark with an identifier.
Now there are certain naming conventions in Python.
There is something called class in Python, it's a concept.
Maybe some of you are familiar from other program languages.
It starts with an upper case letter
and all other identifiers start with a lowercase.
So only the class name starts with an uppercase.
Now there are certain other rules as well.
They are related to classes.
I think it would be better if we covered them then.
All of these things like to identify this as private,
strongly private, two trailing underscores.
Now all of these are some concepts that we'll learn later.
Let's leave them for that.
But for now you just need to know that okay,
this is how you name a variable in Python
or an identifier.
Now how are variables stored?
There are reserved memory locations.
What happens is that in the computer's memory,
which is the RAM right, it creates a space and stores
that variable in the memory
and it reserves some bytes for it, that's it.
That's all that's happening.
It's just remembering them, storing them.
So it's not that while I am writing it is doing that,
it is when I'm running the file.
So it is not going to do that right now
as soon as I create it.
So don't think that if you create too many variables
your typing speed will slow down or your laptop
will start hanging up.
No, I mean it might do that,
but when you're running the file and it might be for
different reasons that just you know,
simply having too many variables.
So but just notice that this is just an instruction.
This is just nothing but instructions.
When you run it that's when it really does something
on the computer.
Right now you're just typing it like a Notepad document
and you're pressing control S.
But when I run it and by run I mean I go ahead and do this,
that's when it goes to memory
and it does all of these things.
Yeah, so we have already covered this right,
where we run it and the values get printed.
Now let's look at some standard datatypes
available in Python.
Even before that, let's try to understand what is meant
by datatype.
Now, a datatype is basically a way to say that something
is an integer, something is a string,
something like what kind of data it is,
whether it is a numeral or it is a string
or it is a Boolean or something else or something else.
So Python has two kinds of different datatypes,
or any program language for that.
One is immutable and the other one is mutable.
So the word mutable and immutable comes from,
you need to think of mutation.
The one that you might have studied in biology.
Mutation fundamentally means change.
So mutable datatype would say that it can be changed,
that the data can be changed.
Immutable is that it cannot be changed.
So lists, dictionaries, or sets,
these are something that you're going to study.
They can be changed, but immutable datatypes,
numbers, strings, or tuples, they cannot be changed.
Now one source of confusion that you might have
is you might be thinking so, if I do a=121 like this
and then I do a = 121, or let's suppose right,
I do this and then I do a = 3,
then this should not happen.
Because a is an integer and you're changing it.
I'm first setting it to a random value 121,
and then I'm setting it to three.
But that is not what it means.
What it means is that, see this is just an identifier.
This is not the data.
The data is on the right side of this.
So when it says that it is immutable, what it means to say
is that whatever you do, you cannot turn 121 into three.
Which if you think about it makes sense, right?
So this is the level of depth by the way,
that happened in programming.
Something as obvious that one is not equal to two
or 121 is not equal to three, cannot be three.
It is just that.
But this is a different thing in the world
and this is a different thing in the world.
There is no way that it can convert this into this.
Like you cannot convert gold into silver.
Gold is immutable.
Gold is immutable, where you cannot turn it into silver.
You cannot turn it into any other element.
But then there are certain things in the world
which are mutable, that you can take them and turn them
from one thing to another thing.
So for example, you can take a seed and turn it into a tree.
That's mutation.
But if you're trying to mutate gold into a tree
or gold into silver, it will not happen.
It will remain gold, even if you mix impurities in it.
It will be impure gold, but it will still be gold.
So you need to think of it like that.
It's a very basic concept but it's something which is there
across programming, that the entity cannot change.
Now let's talk about the immutable datatypes,
numbers, string and tuples.
No Python supports three different kinds of numeric values,
integers, which are signs.
So you can add a plus or a minus in front of that.
There are floats, which are real numbers, decimals.
Then there are complex numbers.
When it comes to representing numbers, you can represent
them in three different ways, binary, octal or hexadecimal.
However, please note that you typically will not be using
these representations.
Typically, you will be using numbers
like you use them normally.
So to define an integer you set base of 10,
to define a float base of 10.65 and for complex numbers.
In case you remember this or you're familiar with this,
this is 10+6j.
This is the imaginary value and will operate like this.
Let's quickly have a look.
We have the same file here, let's run it,
and it outputs like 10+6j.
If I do 10=5+4j
and I print D-C,
let's see what do I get.
So if you see, it does done the subtraction,
5-10 is minus five, 4-X is minus two and a get a result.
Next is strings, a continuous set of characters
represented within any quotation is called a string.
So whether it's singular or double quotes.
Now, Python does not support a character type.
This is basically coming from other languages which have
something called its char, as a character.
Python doesn't have that.
Python just has continuous strings.
In case you're not familiar with character it is okay.
But character datatype is no different to character
in the generic way of alphabets.
It is referring to it in the programming context,
where there is something called a character,
it's char, datatype, in Java, C++, so on and so forth.
Python doesn't have it.
Python just has strings.
It cannot differentiate between single and double quotes
or does not differentiate between single and double quotes.
As you've already tried it, this works for us right.
We have already done the print statements using strings.
Next is tuple, so tuple is a fixed list.
It comes within parentheses.
For example, even this right now, the way it is printing,
it's a tuple.
If you look at this, this is a tuple.
I can do this, now notice this.
I can print this and this will run.
Now I can also do the following with a tuple,
I can print and individual element.
I can print the zero index.
These are zero index, meaning or let me change it,
so let's suppose these are fours, right.
Start off saying that this is the first and the second
and the third.
It works with a zero index.
It says this is a zero,
this is the first and this is the second.
Now I can do this as well.
So this is me reading an individual element of the tuple,
four, five, six.
However, notice if I try to do this.
Python has already highlighted this.
It has a problem with it.
It you hover over it, it's just saying that tuples
don't support item assignment, which is basically meaning
that hey, you can't do this, you cannot change it.
I cannot change this, let me run it.
It was throwing an error.
It's throwing an error tuple object does not support
item assignment.
So I'm trying to change this datatype.
It's like I'm trying to change gold into silver
and saying sorry, can't do that, that's not possible.
You cannot modify this.
Now let's look at the mutable datatypes.
The first one is a list.
So we just saw a tuple, it seemed like a list of items right
but it was called a tuple.
But Python also has something called a list.
Now, lists can contain various elements
and number of elements if you want to create an empty list,
this is how you create one.
Square bracket, open and close, this is an empty list
with zero things in it.
This is a list with one, two, three,
and four things in it.
Now if you notice, all of these are different types
of data.
This is a decimal, this is an integer
and this is a string.
You can print the list and pretty much make a tuple
you can assess any element within it.
Let's run this.
What you can also do is that you can update it.
So we can say A is equal to changed first, second element.
So because this is, again, this is zero indexed again,
by the way.
Like tuple was zero indexed, this is also zero indexed.
Zero index being that this is referred to as a zero.
It doesn't count it as one.
So that is generally how things work in programming.
Almost all programming languages have it this way.
So if you see, I change this second element.
So the same thing was throwing an error in the tuple,
but in a list it is doing in this certain way.
It is allowing me to modify it.
Now a list can contain other lists as well.
So you know, you can also do this.
And you can also do, you know, four and then third element
and it can also contain tuples.
It can also contain a tuple inside it.
This is a particularly complicated example of,
not complicated, but something that you would normally
encounter, but I'm just trying to showcase it
to make a point.
That a list can be of anything.
So think about the English word list, it can be of anything.
What I've done here is that I've said that zero element,
which is this, this is a list, give me the second one.
So it gives me four.
If I say one and then I run it, it'll give me two.
I can run it on the first one as well.
Yeah, it will give me four.
If I need to access the tuple, think about which element
it is, zero, one, two, three, right.
I want to print five, then I'll do this.
B3 to, I'm just counting.
I'm counting zero, one, two, three.
Okay, I got the tuple till this point.
After that, I want to access five.
It is zero, one, two, I go to two.
Now slight differences, lists are putting square brackets,
tuples are within parentheses.
Lists are mutable, tuples are immutable.
Tuples are faster than lists, letting that owing to
the immutable nature of it, but that's not the real reason
why you will use a tuple.
You will use a tuple if you want to have a fixed value.
So if let's suppose, you are making a simple calculator,
you know which can do scientific calculations.
Let's suppose by using the value of pi
and scientific constants that can calculate the area
of a circle or a sphere or a cone or whatever.
It needs to use 3.14, the value of pi,
and other constants and you need to maintain that
and need to ensure that it is not modified accidentally
by you or your programming.
That is when you will use a tuple.
Or you would use a tuple if you had to maintain a list
of countries because you don't want the list of countries
to get changed.
You don't want, accidentally, when you're coding,
you end up changing an element within that list
because you know, it will cause a lot of problems.
So basically, to understand why you need to also
think about and understand why immutable datatypes
are needed because if you look at lists and tuples,
the only difference really here is--
So by the way, I can do this with a tuple as well.
I can do the very same thing.
I can simply do this, see, and I can run it.
It will work as it is.
It just did, I just printed the element, right?
Now, the only difference is that lists are slower,
but more importantly, lists are mutable.
So mostly you will find yourself believing that
(speech unclear) dealing with lists rather than tuples.
Tuples are needed for very specific scenarios.
So don't get confused by why do I have two,
can I use either one of them.
Focus on lists for now.
Tuples will naturally come to you when you are presented
with a scenario.
The tuples are more in the niche, where they are part
of a particular case.
Next are dictionaries.
So dictionaries, as the name suggests, it is like you would
go through a dictionary right.
What is the meaning of the word naive,
what is the meaning of the word precarious,
what is the meaning of the word apple, so on and so forth.
Against every word you will have a meaning.
So the word is like a key
and the explanation is like a value.
This is a key, age is a key, this is the value.
This is the key, this is a value.
If you look at is, this is very readable.
They're separated by colons, okay,
and the need to use this is that it makes your data
very readable.
Key value pairs is something that comes naturally.
So if you look at a fill form, any sign up form
that you have every filled,
it asks for an email and a password.
Imagine that being stored here.
So in the sense, let's open this up,
so this is being handled like this.
Now, suppose you can also argue that hey,
you know I can also store it like this,
why shouldn't I store it like this?
I'm gonna store it like this and instead of, you know,
doing, oh by the way, so what you would have done over here
is you would have used square brackets and then you would
have typed the key.
Let's run this once.
Right, same output.
So you can argue, hey, why wouldn't I use a list
or a tuple, why would I use this?
So one primary reason why you would not do this
is because this is way more readable.
Imagine how other lines of code, imagine this was declared
on line one and you're using this on line 100,
you don't know what is zero.
You have to constantly refer to what is being stored
at zero.
But if you look at it, immediately oh, okay,
you're trying to print the age.
So for developer readability, dictionaries are way better.
You will find that you will use these a lot as well.
Once again, so the keys have to be strings.
The keys have to be strings, but the values can be anything.
The values can be anything in the sense that
it could be an add as well, or it could be you know,
a dictionary as well.
A dictionary within a dictionary and add A within
a dictionary, all of them can be mixed and matched.
Last one is sets, it's an unordered collection of items.
Every element is unique and you use curly braces
separated by a comma.
If you notice, if you try it out,
once you've tried finally, so you might assign it
but it will automatically just print or consider
the unique ones.
That's it, it wouldn't bother with the non-unique ones.
Next is Python operators, operators being any sort
of operation on your datatypes.
So two plus three is equal to five,
two and three are operant and plus is called
as an operator.
Arithmetic, assignment, comparison,
logical, bitwise, identity and membership.
These are the different kinds of operators.
Let's go through them.
First one is arithmetic, quite simple that we have done
in maths, a+b, a-b,
a*b, a/b, this gives you the remainder.
So this is called a modulus operator,
let me quickly show it to you.
Let's give zero four modulus five gives us four.
Then of course there's 4+5 and then 4-5, right.
Exponent is 2**3
or 2**2, sorry, I misprinted that.
Yeah, two, four, and then there is float division,
which is so this is float division.
This will return the float.
Now assigns value from right to left.
So first this will be calculated
and then this will be assigned.
So if this seems confusing to you, it starts from the right.
It doesn't start on the left.
So it isn't that it will be confused with more to do
it will do a+b and then reassign it to a.
A-b and then reassign it to a.
A*b and a/b.
What is written over here is a shorthand notation.
So in Python you could, let's supposed I had a=2,
b=3, now I could do a+b, then a is five.
Or I can also do a+=b
and then a is eight.
So this is because I added again.
I can do it like this again, sorry,
a, b=2, 3 and I can do a+=b.
These are the shorthand notations which give the same
effect, which says that a, take the left one,
and apply this operation and using the second operant
and just assign it back the value.
Next are comparison operators, so these give you a result
that's true or false.
These give you Boolean values.
So if you do a=b it says false.
You say 1=1 it says true.
If I do it a is not equal to be it'll give me a true value.
I do a is greater than b, give me a true value.
If I do a less than b, it'll give me a false value.
'Cause a is two and b is three.
The a is five and b is yeah, in case you're getting that
confused, so less than b, five is less than three false.
Five is greater than three is true.
This is naturally true, and I can also do a greater than
equal to b and a less than equal to b.
These are keywords, these are not strings, by the way.
So these are not strings, these are Python keywords.
Show it to you here, I can also assign them like this.
If you notice the color, the color is different.
It's not like just another value.
It's been treated as a special value in Python.
Yeah, so then there are logical operators as well,
a and b, a or b, and not a.
They are pretty much so a and b will give me three.
It'll give me the lower value.
A or b will give me the greater value
and not a will give me false.
So because a is a truth, not one will give me false.
Not zero will give me true.
So anything greater than zero and not false will give me
true and not true will give me false.
Anything which is not zero and not false
is going to give me a false and apply the not operator.
Going back to what was this doing, why it was giving
the lower value and why it was giving the high value.
It is the way Python to use with this particular thing
where or on two particular things, if we compare them,
and it will convert them into binary
and then it will automatically perform the operation
that is (speaking unclearly).
This is more related to binary logic
rather than Python logic.
So this is the result that you will get even in mathematics.
You had a math question like three and five, right
and three or five.
This is not about Python really,
this is more about mathematics, about why operators
three and five should be five and while the operation
of two of five should be three.
Work logic.
So I'll just sort of leave that for you to kind of
rediscover, but that's really the school mathematics.
It has got nothing to do particularly with Python.
This is just the way you do it in Python.
You simply use it like English.
So if you notice this, this might not seem like English
to you right now, this might not seem like code to you
right now, but all of this is valid code.
I can do this as well, right.
And I can run this.
See?
I'm getting a result, false.
This is all valid Python code and A or false returns
this value.
So or basically always this returns a high value
and always returns a low value.
If I have one and one, but if I have five and five,
and will return the lowest value,
which is five in this case.
But if I have five and four,
it will return the lowest value.
Or always returns the highest value.
Next are bitwise operators, these are done on bits,
essentially a binary AND, binary OR,
binary XOR, binary NOT,
binary left shift, binary right shift.
Good to know about them, but as I said earlier,
you will rarely be dealing with binary operators in Python.
Binary number, very, veryrare case.
Otherwise, when you're dealing with that, it's going to
pretty much like mathematics.
This has got very little to do with Python,
it's got more to do with maths.
Then Python also has identity.
Again, very, very readable.
So five is four,
of course you'll get a false.
Again, five is not four.
I can do it using variables as well.
I can do the same thing saying that...
Of course we're going to be, so you might be asking,
hey what is the use of this, why is it printing true
false true false?
Of course these are useful comparisons.
You can later set up logic, something like if C is D
then do this, otherwise do something else.
So if C was you know, suppose today is equal to Wednesday.
This is yoga day, it's Thursday.
Of course, now you can build a logic, if today is yoga day
and I can run it.
It's false.
So now notice how readable this is.
It's reading like English, right.
That's why I love Python, that's why a lot of people
love Python because it's very, very verbose.
The word is verbose is very easier to just read through
the code and understand okay, what does this mean.
It reads like English.
Next is in, so membership operators what they do is that
they check if a certain thing exists within a certain thing.
So this applies for dictionaries, this applies for lists,
it applies for tuple.
So I can do something like this, let me run this.
Four not in list, false, right.
Over here what I can do is, comment all of this out,
right, so it's membership checking.
It's checking if a identifies a member of this or not,
of this list or not.
This works on lists, it works on dictionaries as well.
In dictionary it will work like this.
Print age in A.
We're just trying to check if a key is present or not.
It says false.
So wait it says true here.
Let me comment this out.
Let me comment all of this out.
Now if I were to by mistake do this, it was in this case
it will say false.
Again, for comparison's sake,
if I run this right, and the point of this of course,
not to in the case of a dictionary if I do this,
where I use a key which is not a part of it
and I run it I'll get an error, key error, age.
It doesn't recognize it.
So in case I need to check for a key before I use it,
I can use something of this sort.
Now let's look at conditional statements
and conditional statements are used to,
so this is what we are sort of building up to this point,
they are if and else if.
If a condition is met as true do something.
If it is not do something else.
We do it every day.
If today's your Wednesday, go to yoga class,
otherwise, no yoga class otherwise chill at home.
That's just something that comes very naturally to us
as human beings now and naturally we need it
in programming languages as well.
Of course, the way it works is that you start a program
and this is sort of called a control flow.
If X is less than Y, do a bunch of stuff.
If it is not, then check again for a certain condition.
If that is true then do a bunch of other stuff,
otherwise do this.
That just is how programming works a lot.
So a lot of business logic or logic of how you want
to your application to work gets built in over here.
Okay, now let me talk about indentation once more, okay.
Let me first comment all file operation this.
So the thing is is that when you type instructions
in Python, how do you tell Python that hey,
these bunch of instructions need to be followed
and that if and these bunch of instructions under this
else if, right?
So essentially this is a block of code.
This is definitely a block of code where the instructions
won't just come on a single line, right.
Naturally, you write more complex things over here.
You might need two, you might need to write a hundred lines
if this condition is met.
You might need to do a hundred things.
Now how does Python know that this is within this?
Like, one way to do it is of course, I mean why can't I
write it like this?
I can, why not?
But then, does elif come under tired instruction set
that I've been writing?
Or does it map to this?
Because not all of them are on the same line.
So should my statement go here or here?
There is no enclosure.
There is no clear path defined.
The way Python does it through indentation
that we talked about earlier,
where this needs to be at a distance of four spaces.
So the IDE does it by itself as soon as
you're sort of typing.
If I'm typing, let's suppose if X>Y
and I press colon, it's always followed by a colon,
by the way, so notice the colon and I press enter.
It will automatically go to four spaces.
It doesn't even ask me.
Tab in the Pycharm works with four spaces always.
Otherwise you can give four spaces by yourself.
So there's one, two, three, four.
But your code will not work if you don't do this.
Your code will not work.
It's not just, there is a very strong reason
why this is like this.
The reason why it is like this in Python is because
the developer of Python wanted to make
Python read beautifully.
While it may seem like an odd reason,
but if you look at code production level code,
so which I mean if you look at code written by great
software developers, good developers,
not even great, just even to be good enough,
considered to be good, they will write code which is
very readable, very clean.
Guido van Rossum, the creator of Python,
wanted to enforce it.
He said it was not a choice.
I tell you how sort of otherwise it typically gets
done in other languages.
So in other languages, you would have something like this.
Okay, and then you would have something like this
and then you would have curly braces, right.
Then you would have, let's suppose, a bunch of lines
let's suppose print X, right.
Then this would have print Y a bunch of times.
Now this looks dirty because I mean,
I could have indented it and it would become a little more
readable, like this.
Ignore the underlinings because this is just not
valid Python code, I'm just trying to show an example here
of how other program languages do it.
So they do it by curly braces.
They indicate what is part of the closure
or a block of code using curly braces.
But Python doesn't have that.
Python doesn't have curly braces enclosing
a bunch of instructions.
What Python has an indentation, this.
Bases that, it takes a decision.
So if it is true then it goes inside.
If this is true it goes inside it.
Otherwise, finally doesn't check anything and just executes.
So let's run this.
X<Y and on line six.
If you just wanted to do an if else statement,
you can do that as well.
You can add conditions over here and simply run it.
Greater than 25, a is less than 10,
a between 10 and 25,
greater than equal to 10.
Pretty much like how you would write it in mathematics.
If this is true then do this, otherwise do this.
Please note that these and these are not related.
Both of them will always be executed.
So if I do this and I make a as 10, right,
we will see both the statements.
So less than 10, in between 10 and 25.
But if this was under this, now this is an if else block,
then and if block.
Because the tab.
So to determine what is inside what, you need to really look
at the indentation level from the parent statements.
So for this line to be printed this and this
needs to be true.
For this line to be executed only this needs to be true.
For this to be executed, this line needs to be false,
but this line needs to be true.
So the control wouldn't go inside this, but this if else
is for then this outer block.
So in the terms of comments, this is a parent comment
and this person is replying to that parent comment.
It is not an individual comment by itself,
it's a nested comment.
If this is like this,
then both of these are parent comments.
Next is loops, there are three kinds of loops in Python.
The purpose of loop is to do the same thing again and again
and again, like on repeat.
If a condition is met, so it would check for a condition
and it would continue repeating according to that condition
a number of times.
So a value performs until an expiration is true.
It checks, goes in the body of the loop,
otherwise it goes out.
Please note that the while loop, again, indentation block
followed by a colon.
Now in this case what you are doing is that
we are printing tell fold.
We are going to increment the value one by one one by one.
We are going to count till four.
Let's run this.
One, two, three, four and then good bye.
As soon as count becomes five, it will not be true anymore.
It'll come out of the loop.
Let's mute this and instead run this.
So if rank is not equal to 12 it will print from
five till 11.
Very simple.
So this is a little guessing game that you can try out.
Input number 10, the number is too small.
Input number 15, the number is too large.
If input number is 12, exit, I mean you will end the loop.
Congratulations you made it.
Now for the for loop, this is by far the most important
one that you will be using.
It iterates a word arrange.
Now arrange could be a list.
So if you wanted to print a list or a tuple
or even a dictionary items, anything which is a collection,
right, that it what it takes over and will increment,
it will keep going iterating over or stepping over the
each individual item.
So let's look at the statement here.
So we have a list of items, fruits, and what we can do is
ignore this.
This is not relevant.
Fruit and fruits, print fruit.
Again, simply run this.
So it runs bananas, apples, oranges, in that order.
This is the list, then again it will do the same thing.
The entire list, one, two, three, Python.
Right, so you can use for a list to click the factorial
of a number.
All you need to do is reproducing the number by a one
and keep multiplying it and keep using it
again and again and again.
So you just subtract it, multiply it,
multiply it, multiply it and then you will get an outcome.
Of course you can have nested loops.
So you can have a loop inside a loop.
Again, same principle of indentation applies.
You can have a by while loop instead of for loop,
or a for loop instead of a loop.
Purely depends on what you're building,
on what you are trying to do.
So it depends on the logic, right.
Let's suppose the logic is as simple as you are writing
a robot and the saying while Thursday for zero
to one million keep taking the left foot
and the right foot forward.
Left foot forward, right foot forward.
It will keep doing that until it is Thursday.
So it will take as many million steps, or let's suppose
it's about the instruction that's something like this.
While day equals Thursday
for three, four right.
So you are basically telling the world that hey while
it is Thursday, just keep doing this all the time.
That really depends on what you are trying to do
with what your application requires.
So again, this is something that you could try out
where you could write a code in Python,
just simulating a ATM, where you ask for the four digit PIN.
Then you show a menu saying that while balance
while overdrawal, pay and return card.
Then the user selects a value or you know,
it keeps repeating it a number of times
or you could set a limit on it.
While the number of tries is not five, keep asking the user
to make a selection.
Now, there are certain control statements in a loop.
If you want to sort of break out of a loop,
let's suppose you're trying to looping over a
(garbled speech) or something,
or you are trying to loop over something and you,
a certain condition is make you want to break out of it.
You want to finish the loop right then and there.
Then you would use a break statement.
If a certain condition is met I really do not want to do it
then it would use a continue statement,
where it would skip the loop.
It would skip where it is in the loop and go to the next
and then of course there's a pass statement.
This is used not just in loops, this is used in classes,
functions, multiple other places as well.
This is used to define an empty body.
So let's suppose for I in one, two, three,
I will just say pass.
Just because I have not returned any code yet,
I'll worry about it later, but it's just that
I want to, because if I don't do this then this is
not entirely correct.
If you notice, I cannot write any code after this,
even if it is, you know legitimately true,
because the indentation,
Python doesn't know that I've closed this.
I will use a pass.
Pass, again, is at an indentation.
It is a default sort of way to tell Python that hey,
this is over and just go to the next line, which is 24,
and don't consider trying to inside the for loop.
So as I was saying right, if you're printing numbers
between 10 and 50, but as soon as you encounter 30
you want to break out of the loop.
It will stop the loop.
If you really want 11 and you say that j = 5
and after that you want to continue to the next one,
it will skip the five.
It will skip over it and straight away go to seven.
Let's try out this one.
Let's suppose range is one to 11.
So let's run this.
Wait, I need to mute all of this and this as well.
Yeah, so if you notice that this time five is not being
printed because on five we just skipped.
I place this two places above over here and run it.
Then it will print everything because you're printing
anyways, so I mean, essentially the use case of continue
is that if you do not, for some reason, want to press
a certain value that you're going to encounter.
If a certain condition is met you don't want the loop
to continue after that.
You want it to go to the next one.
But if you want to break all together right,
and you were not expecting this to happen
and it will just stop.
Continue will go to the next item.
The next thing is command line parameters.
When you're executing item file, right,
you might want to pass certain arguments to it.
Arguments being certain values to it.
That happens to the Python command line argument.
So Python is a module and built being that is straight
out of the box, you don't have to install anything.
It's like a library, that sort of module is, essentially.
You have a Python sys module and what the sys module does
is that it stands for system.
It is first your operating system and it allows you to
access the arguments that you are passing.
It's a list of command line arguments and len(sys.argv)
is the length of command line arguments.
So let's look at one of the files for this.
Let's actually create a file.
First thing that you need to do in the file is import sys
and then you want to say len(system.argv)
and then you want to say sys.argv.
Now, one thing that we haven't covered until now is how
the Python file will be run.
What I'm doing is right now I'm going to the folder
which contains this folder DS mod one, okay.
It contains something with the letters there.
Okay, then all the files including test.py.
To execute the Python file,
all I need to do is run Python test.py.
Then I need to provide the arguments.
Now notice that the length of arguments is six.
It's not just one, two, three, four, five.
The name of the file is an argument in itself.
So this can be confusing at first.
The first thing that it takes as an argument
is the file itself but then next is followed by whatever
you give it.
This is one, two, three, four, five.
Okay, there's .ABD, BCD.
This is what (voice cuts out).
(light instrumental music)
Before we do any of that, let's suppose one problem
or one thing that you want to do with a Python file
is you want to ask the user for an input.
Let's suppose we want to write a simple script which will
ask the user for two numbers and add them or multiply them
or do some sort of an operation with them.
So how can we do that?
The answer is that Python provides you with an input
function and the way the input function works is
that it's an in-built function in Python.
You call the input function, pass a string.
The string is the message or the question that you want
to ask the user.
So let's suppose you want to ask the user,
give one number to multiply with or give one number to add.
That will come between the strings.
Now whatever the use types in as the input,
that gets stored into the variable on the left side.
The function basically return of what is does for you
is that it prompts the user with a line,
which is understood in a language that the user
will be able to understand.
The user types in something and then it gets that value
and gives it to you in a variable.
Let's look at a quick example.
Let me comment this and here on this file.
So it's asking for my input.
Let me enter three and it says the received input is three
which means that the input was stored in str.
Let's look at another example.
Suppose you want to ask the user for their name, their age.
Same thing.
You provide the string that you want to ask using.
It gets stored in this variable here
and then you can just print it.
Then of course if you want to perform this operations
you can go ahead and do that.
Let's quickly run this as well.
So I just faced an error.
Now what happens is that input expects, so if I'm not
giving it a number, if I'm giving it a string like my name,
I have to provide it in quotations.
So it will work if I do it like this.
For numbers I don't need to provide any strings.
Let's run it again.
Suppose I give my name as Varun, that won't work.
If you're inputting a string then it has to be
within single quotes.
It can't be without single quotes.
If it's an integer then it doesn't need any quotation.
Next, let's look at files in Python.
So if you remember we discussed that let's suppose you have
to build a software which would manage files for you,
put them in folders, sort them, so on and so forth.
Python plays very neat and nifty materials to be good files.
So let's look at that.
When it comes to files, there are a couple of typical things
that you will need to do.
You need to open and close a file, even if it's a Notepad
file, or a Python file that you must just created
after the first one.
The thing that you do with any file is that you open it
or you close it.
The next things are you need to write a file
or to read from a file.
Then sort of maintenance operation such as renaming
the files and moving the files or deleting the files,
stuff like that.
So before we do anything, you need to understand that
opening and closing a file how does that sort of
work in Python.
The thing is that opening and closing a file works
with modes.
It's not as simple as opening a physical file
in the sense that there are different modes in which
you can open up Python file.
There's a read only mode, there is a write mode,
there is a write plus mode which creates the file
if it doesn't exist, and all these are for the sake
of protection in the sense that if a file was always open
in write only mode right, if it was that you can write
and read to it all the time.
You might end up doing certain things to a file,
certain operations, you might end up messing up the data
inside the file.
It is basically a best practice that you open the file
with the correct kind of mode based on what you're going
to use it for.
If you're not going to write to is,
don't open it in a write mode, open it in a read mode.
Then of course, there is closing a file.
So suppose you are opening a huge file.
It's a file which is 500 mb in size.
It contains 10 million addresses or names.
If you do not close the file, what happens is that
when you open a file it gets loaded into the RAM.
If you do not close the file properly, you do not do that
management bit of the file properly,
then it will remain in the memory and that can potentially
cause a memory leak.
So just like one of the precautions to keep in mind
that no matter how you're doing it, opening closing
of the file needs to be kept in mind.
So open the file, you will be using the open function
of Python.
It take the file name as a first parameter
and then the access mode is the second parameter.
The file name actually is the name of the file
and access mode is what I just was talking about.
There's read, write, or append.
So here are the different access modes.
R, this is the default mode,
opens the file for reading only.
There is a mode called rb as well.
By the way, you pass them as strings over here.
This is again, this gonna be in quotes,
string r and string rb.
It opens a file for reading only in binary format.
It is rare that you will use this, but it's just something
which is good to know about.
For typical files which are sort of decently maintained,
not even well maintained, you would not really need to go
for rb and the difference, unless you're dealing with
some sort of malformed file, is not that much.
Or something specific that you're doing with it
where you need it in the binary format.
Then there's the r+ mode.
It opens it for both reading and writing.
Then there's the rb+ mode, it opens the file
for reading and writing and in the binary format.
There's the w mode, it opens the file for writing only.
It overwrites the file if the file exists.
So this is the significant difference between w and r+.
Where w will basically overwrite the file,
just like creating a new file, irrespective of the fact
that it exists already.
Whereas r+ will not overwrite the file.
Wb, it open the file for writing only in binary format
and like w, it overwrites the file as well.
Another mode is a, it opens the files for appending,
which means that it only allows you to add to the bottom
of the file.
It doesn't allow you to part of it is already there.
So this is sort of makes sense if you're doing
update or append on the system where you do not want
the prevents the codes to be manipulated in any way.
Ab it does the appending in binary format.
A+ opens the file for both appending and reading.
Ab+ opens the file for both appending and reading
in binary format.
W+ and wb+ is something that we have already covered.
Now when it comes to writing files,
you take the file object, which is obtained over here.
You open the file, you get a file object.
This is basically presenting your file in the code
and then you say .write and you pass the string
that you want to write.
The write method has, of course, I mean because you were
talking with the binary bit as well earlier,
that you can open a file in a binary format.
So write just doesn't take strings,
it can take the binary data as well and not just text.
The read method reads a file string from an open file.
And again, it can read binary data as well.
Notice the counts.
So if you give it a count, that is the number of characters
it will read at a time.
It doesn't offer parameters, we do not give it a count
it will ready everything.
Let's look at renaming files.
So for renaming a file, you need to give the current file
name and then the new file name.
It takes two arguments and rename, if you're wondering
what is OS, it's a module available in Python
which will deal with the operating systems.
We are going to cover it later in depth,
but just know that OS represents your operating systems.
Because when it comes to renaming, it is really usually
done with the help of the operating system.
Not that you need to know about operating systems
in terms of how they're built or written,
but this is just is a nifty utility available in Python.
For deleting files, we will again use os.remove
and the file name as the argument for it.
So let's quickly look at a couple of examples.
So let's start with this.
We open the file, notice the mode is w+.
So the file doesn't exist write now
but we'll create the file.
Now I'm going to loop over numbers one to 10
and write these 10 lines.
Let's see if I'm successful or not.
Finished, I have a file that says hello, welcome to Python.
Maintains because this starts on one.
If I do just zero, we do that.
So let me run it again.
Now it has done it 11 times.
It will simply override the file each time
because it's in w+ mode.
Now, let's suppose we had to read the file
and we will run it.
What did we do wrong?
What we did wrong here was that it's in w+ mode
so it is already emptied the file.
But let's run this one again and rerun this.
Now we have a fill file, let's mute this out
and now open the file in r mode and now run this.
Now we have this appearing 10 times.
We have a newfile.read.
As I said, you could pass it a counter as well,
so I could pass it five and it will then only read
five characters at a time.
You get a very weird output.
Don't be confused by it but it's just that it's reading
only five characters at a time and that's why
it's coming like this.
If it was reading 500 characters at a time,
then the output would have been different.
500 works similarly for us just reading 50 characters
at a time.
So I think the first time it would have read all of this
and then it will have read this.
Now let's try the OS function as well,
in case you want to remove the file.
Let's first rename the file
to new.txt.
We don't really need to open the file so we can just
remove it and we can run this.
If you notice, this has been renamed.
That file disappeared and we now have new.txt.
Now we can simply go ahead and remove.
We run it and that gets removed.
Did you notice it got removed from here?
So the file.close method closes the open file.
A close file, of course, cannot be read and in case I was
to reassign new file.
Let's suppose I was doing this and then I take this variable
and I assign it some other value,
what Python will do is tell it to automatically close
the file whenever I do a reassignment.
So it will automatically close the file if I reassign
this variable.
That is where Python is of course, really helpful as well,
because that's one of the things that if you remember,
that Python is supposed to be a very friendly language
to learn.
Now let's try another one where we close the file
and then we try to write to the file.
Let's see what happens.
It gives us an error.
Yeah, I'm trying to open it from the read mode
and the file does not exist.
Let me run it again.
Now there it is, I/O operating on closed file.
It says the file is closed.
Now there are various attributes of methods available
on a file.
One is file.close, we just saw that.
There is file.mode as well.
Let's see what we get with.
So we get w+.
In case you need to, this is used for checking.
You can use it for any sort of checks.
In case you're confused about it, like if writing a large
piece of program, at some point there are multiple files
and you don't remember or you want to have a strict checking
before you do something on a file, maybe you do not want
to write to a file or you want to add a constraint,
that is when this will help you out.
Now the file.name will of course,
return the name of the file.
Let's do that as well.
Let's do another one as well, we have done the mode
and then there's file.softspace as well.
Let's run all of these three.
So we got the name of the file as well.
Softspace is nothing but it returns the Boolean,
which is zero or one, whether a space character needs
to be printed before another value
when using print statements.
Then there's a mode, there's name
and then there's newfile.softspace.
A few other methods available are seek.
It lets you seeking through a file.
It's searching for a particular thing.
It takes in the number of characters for line read
creates a number of characters.
Similarly, seek also takes the number of characters
where you can tell it hey, start reading after 40 characters
have passed.
Tell gives you the current position of the file.
Each of these things over here in the file,
this is a position.
This is a certain line in the file.
So let's try this out and see what we get.
Let's open the file in read mode, run this now.
Right, so it's seek two zero.
Now let's make it 100 and see if that changes anything
in the output.
We're seeing that the second output is changing
based on how much we are seeking.
So if I do it make 1,000, if you go to the 1,000 line
and we can also do a third read at this point.
This is saying that I'm gonna find the first five characters
and I'm gonna do a read and then I'm going to do a tell.
This is basically this of course,
newfile.read will read the entire file, okay.
This is not just going to read one line,
but it starts reading after the five characters.
So the first five characters would be H-E-L, right,
and then two characters for the empty space.
It also counts as a character.
Hello first, the space, one, two, three, four
and then there is new line character.
There is an enter character,
which makes it come to the next line.
So, the five characters.
We have already gone through this then,
the different operations performed on a file.
Now, most of the time, a lot of time in programming
you will hear the word data.
Essentially, that's a lot what programming is about,
about logic, about data and dealing with data.
If your different types of data, how do you arrange them,
how to you organize them?
It's very, very, very important in programming
and in Python and Python provides some excellent ways
to do this.
So Python has something called sequences.
Sequences are nothing but continuous collections
of datatype.
Data which allow the different operations on them.
Whether that's indexing or slicing, or something else,
as you will look.
Think of sequence like how you would think of a sequence
in a real life, like a sequence of bikes,
like a row of bikes standing out side an office
or a row of bikes standing at a traffic light.
You need, in a show room, you need to think of that
as a sequence.
So let's look at sequence operations.
Most common sequence operations are concatenation,
repetition, membership testing,
slicing and indexing.
Let's look at concatenation.
Concatenation is nothing but gluing two thing together.
So you have four bikes over here and then you have four
other bicycles over here.
Concatenation is joining the sequences together,
create a new sequence.
Sequence repetition is basically taking a sequence
and sort of multiplying it by a number.
So which is create, duplicate, sequence it twice the number
of items or it could be three or four and it could just
create a number of items like that.
Next is membership testing.
What this means is that you want to check if a certain
type of bike is a member of this collection of bikes or not,
in the sense that this is not a member of this right.
You cannot locate this particular image or bike style
over here but this one is a member,
like over here and over here.
So there are two occurrences of this member
of this sequence of bikes.
Then there's a concept of sequence indexing,
where items are zero indexed, where this is referred to
as the index zero to zero items.
This is the first, second, third,
fourth, fifth, and so on and so forth.
It is not indexed like one, two, three, four.
Rarely, I don't think, I know of a programming language
which is one index.
Most programming languages work on a zero index basis.
Slicing is basically taking a portion,
which is saying that hey, I want index one to four,
which is one, two, three, and four.
So I want to slice starting from this one
and going up to this one, but not including the last one.
So this is not including four, this is one, two, three.
So four minus one, three, starting with one though.
One is included, four is not.
The last one is not included, before the last one, this one,
is included when you're slicing.
Slicing is basically taking a sub portion of any less.
Think of it like cutting a slice out of a fruit
or of an apple.
You're taking a slice out of this thing, which is like this.
Let's look at a couple of code examples.
First one is really simple.
You have a list which has a list of courses, let's suppose,
are programming languages,
Hadoop, Python, Android.
You can print the first item on the list.
You can slice from zero to two,
just include Hadoop and Python not Android.
Or you could do list of minus one.
So let's just try this out.
So it took Hadoop and Python.
One prints Python not Hadoop and then negative one
basically takes the last element in the list.
Negative one is reverse, I could also do negative two
and I can also do negative three.
Let's look at that.
We can always do this, plus react angular data science
and we can look at the output over here.
See, so we got a concatenated list,
listing react, angular, data science, right.
This got added to it.
We can do a list or partition as well.
Let me comment this, instead have it like print (list).
So I'll just use a different example where I want to
multiply by three instead of two.
If you see, now the same this is appearing three times.
This was appearing once, Hadoop, Python and Android,
and now it's appearing Hadoop, Python, Android,
and again Hadoop, Python, Android.
Membership testing, this gives a true or false answer.
So I can do Hadoop list and do angular in list.
Some of you might remember that we have done it
the last class.
The answer is true or false.
You've already done indexing and slicing.
So let's look at the type of sequences in Python.
There are lists, tuples, strings, sets
and dictionaries available in Python.
Let's go through these one by one.
Lists, it's the most versatile datatype available in Python.
It's basically a comma-separated values
within square brackets.
Some of the operations that you can do in a list is
you can update a list element, you can get the length
of the list, you can concatenate it, as I just showed,
you can slice it, you can delete the limits in a list,
and you can repeat limits in a list.
That is by multiplying it.
So when do you want to use lists?
If you have a collection of data that does not need
random access.
So what is meant by random access is that you want to access
it sequentially.
In the sense that if you talk about random access.
If you talk about list four, like this, or list of three,
you really don't know what this means until and unless
you know the list, right.
You don't know what this is going to be for this represents.
When you're writing code and unless and until you know
what the second element in the list is,
you don't really really know what it is exactly going to be.
You might know it's going to be one of the courses,
but which course, or whether it's a course name
or the course price, you don't know that.
List is useful if you want to just sequentially access it
and process it.
Of course when you have to deal with values that can be
changed, so as you might remember, tuples are immutable
while lists are mutable.
So when you want to change items, that's when lists
will also be helpful.
This is something that I just showed to you.
You can index the list, you can slice it
and then you can access it and reverse as well.
Let's talk about updating the list.
Let's suppose I want to install Java instead of Python
as the first element.
This is how I will do it.
As simple as just taking it out
and assigning it a new value.
Let's run this.
Now we have an updated list.
It says Hadoop, Java and Android as compared to
Hadoop, Python, Android.
To delete the element, what you do is you simply say that
the keyword in Python is del, it's an invert function.
You call del and you pass it the, the item in the list
that you want to read.
But note that we are not passing in the index,
we are passing in the entire item in itself.
The way delete might work internally, that it will look
for that item and then it will go in and delete it.
See, so we removed this one.
Let's look at pop, what does pop do?
If you have a list of this sort, it gives us four.
What pop does is that it just accesses that particular
element and returns it to you.
If you notice the list, it basically removes the element.
Do not think that pop is the same as doing this.
It might look like that, because you could say hey,
it's a spending four right,
so why can't I do it like this?
But eventually pop is removing the element
from the list all together.
Popping it out of the sequence.
Remove just totally removes the element, doesn't return it.
This is also something called list.remove
and if I do list.remove three instead and let's suppose
I do this and then let's see the output for this one.
This doesn't return anything, it just removes it.
There's a difference between pop and remove.
Please don't get confused about it.
Pop will actually give you the item that it removed
so that you can use it.
Remove is more or less used when you are done with that item
and when you don't want it in the list.
Pop is more like going to the showroom and getting a bike
out of the collection of bikes, whereas remove is basically
taking the box in a collection of bikes
and just destroying it.
There's a difference.
You can also use the type function on list.
It'll just return that it's a list type.
Then what we have here is something very interesting,
it's called list comprehension.
Let's look at the example given here.
This is one of Python's very talked about or well-know
features among Python users.
You can basically create a list by doing this.
Forget the print statement for a second, I can do this.
What this does is you need to start reading it from here.
For x in[1,2,3,4,5]].
I know the syntax looks a bit weird.
First and foremost, it needs to be within square bracket,
so let's remove this for a second.
Then square brackets.
It's already a list, but it doesn't have anything.
What we see, how I would have started writing this
expression is I would have said for x in [1,2,3,4,5].
Which is valid, right.
I mean, so which is just like writing for x in [1,2,3,4,5].
It is kind of like that.
I just sort of ended the colon here.
Then I say do this for every X,
which is exponential to the power two.
Now I could have also done the same thing like this.
So let's list.append, I could have done this as well.
But, this is the equivalent of this.
For line 29 and 31 to 33 will give the same desired.
It's just that this is a bit neater to write
and call list this comprehension and Python developers
love using it.
Just because it's in a single line as compared
to three lines of code.
Okay, we already did append,
so it adds an item to the end of the list.
Extend is used with another list, so where you see that here
extend this list right, by adding all these items
at the end of the list.
So let me show you an example.
If I have this and I have this list over here
and then I run this, with one, two, three, g, h.
Append is for a single element.
Extend is when you have two lists
and you want to join them together.
This is concatenation.
Insert, as the name suggests, it will insert one item
at that particular position in the list, right.
So it inserted.
If you had to do this yourself, you could imagine,
it is doable, but it would have been very troublesome.
So it will take the index and it'll say that okay,
at this index you want this value.
At this index, which is one, you want this value.
So let me do that for you.
Because had you done this like this instead,
it would have overwritten two here, instead of one.
But insert will insert it instead of deleting any of the
overwriting any of the existing element.
You've already looked at remove,
it will remove that element from the list.
So there is another inbuilt function
which is called sorted.
It will take a list and will sort it for you.
Let's try it out.
Though it will automatically sort it in
the alphabetical order here.
Okay, so if you look at the original list and you look
at this one, this one is sorted now.
Another thing is that if you want to reverse the list,
so let's suppose for some reason you want to print it
in reverse order.
You sorted it, you want it sorted for most reasons,
but then just by printing the names, or let's suppose
there's a functionality in your application where the user
can go to A to Z and Z to A.
Course in list four and the syntax is colon colon
and then negative one.
Print course.
So if you look at this, this has printed the list
in the reverse order from here to here.
But the original list is still the same.
It's a neat little trick where you can use the reverse
order of the list without actually reversing the list,
which makes a lot of sense.
I mean, why create copies after copies of the same thing.
So you can just reverse the list and use it just like that.
Now, another interesting thing is that you can have a,
that the list in itself can contain dictionaries,
tuples, other lists.
It can just go on and on and on and to whatever operations
you can just keep chaining it on top of it.
So if you look at this over here, it is selecting the tuple
first and then on the tuple it is doing the slicing.
So whatever it is that you want to do,
you can chain this together.
Tuple can contain more tuple.
There is no sort of limit on the amount
of nesting you can do.
It is really up to your needs.
If you think that okay, this tuple will contain further more
tuples, you can just keep adding square brackets,
one after the other and it will work.
So this is a more visual representation.
Zero element, first element, this has zero, one, two,
this has zero, one.
Now you can add further under it more and more and more
and it can just keep continue doing that repeatedly.
Now the thing is is that even though it's a tuple right,
I mean you can modify the tuple
even though it's inside a list.
So it's not that any mutable item will
suddenly become mutable if it's inside a mutable sequence.
Let's look at tuples.
So a tuple is a sequence of immutable Python objects.
Tuples are sequences just like lists except that they
with start with regular bracket and similarly
they are separated by comma.
So a tuple has slicing, updation, length of the tuple
repetition, deletion, concatenation.
All of those things are available as operations
even on a tuple.
A tuple is somewhat faster than a list
and in one of the areas where you might want to use a tuple
and you might want to also use a tuple when
you have to deal with values which cannot be changed.
So as you talked about last time right,
immutable scientific contents, things that you do not
want to change, the value of pi,
just refresh your memory about that.
If there's a set list of countries in your application
that you do not want to be modified,
that is when you will use a tuple.
So basically if you have a constant set of values
and you want to iterate through them,
that is when you would use a tuple.
Let's open the file that I have ready for this.
Let's run this and let's see the output.
Now length three, makes sense right.
Maximum is Python.
So the way it chooses a maximum is with a starting
of the letter.
It's ascending order and then there is a minimum
which is Hadoop.
Now what if it had another value, three, inside it
or two inside it, what will happen?
Let's see.
What will be the min and the maximum?
Min will become as two.
What if this was a big number like this, what will happen?
So chain will always win.
So it's looking at our sky values and that's how it's
determining the maximum.
So again sorted and reversing works exactly like strings.
Multiplication and membership checking, again,
exactly as you have seen till now.
You can do a repetition, you can do a membership testing,
just exactly the same.
Now of course, you cannot update a tuple.
That will throw and error saying that.
So this will not work at all.
If tuple three was already defined, but of course if tuple
three is not defined you can add the two things together.
Of course, this will throw an error because tuple
is not defined over here.
You cannot delete a variable which has not been defined.
This is something we covered in last chapter as well
and we went through a couple of examples where lists
are mutable, where tuples are not.
Now, as you saw that lists can contain tuples,
a tuple can contain lists as well.
Cuts both ways and then of course you can chain it together.
So the lists in excel can contain multiple lists.
Same visualization again, zero, one, two,
then zero, you have zero, one, two.
Within one you have all these elements.
The list can extend further and can just go on and on
and on like that.
So don't be confused about this.
You're actually updating the list here.
You're not updating the tuple.
The tuple updation would mean that you're doing it on zero.
So let me show you via an example.
Let's suppose you have a tuple here, zero, one, two,
one, two, three
and four, five, six.
If I try to do this, let me run this.
This will not work.
However, if I try to do this, this will work.
Do not be confused about this because what I've done,
this thing has been done on this particular element,
which is mutable.
This is not mutable.
This is mutable.
This is basically a tuple element,
what I've selected right now.
Now this is a tuple element.
The tuple element in itself cannot be changed,
but what is inside the tuple element is mutable
by itself, then it can be changed.
So you can convert your tuple into a list as well
by using the list keyword or the tuple key word.
You just pass it, the tuple or the list,
and it will just convert it and give it back to you.
So I can do this where I can say in tuple, print tuple item
and print item.
Notice the difference?
Just that of brackets, but of course, this one is not,
one is immutable, one is the other way around.
I can of course always do this.
I can do this and then I can run it
and now this entire thing is a list.
So it's just helpful to know that these conversions
are possible, not that you will necessary need to use these.
Next, let's look at strings.
As you look at the strings, you can create them by
enclosing them within single quotes, double quotes,
or even triple quotes.
The operations available on strings are again,
slicing, updating, concatenation,
repetition, membership and then reversing a string.
I know a string, it's a little difficult to sort of imagine
a string like a sequence, but you really need to think of it
pretty much like a tuple, where they're individual elements
which cannot be updated.
Start thinking of a string somewhere like a tuple where,
but of course the difference here is that the members
of the string cannot be a list.
So it cannot contain another list or it cannot contain
another tuple, naturally,
but it can contain any alphanumeric characters.
You need to think of it really like a tuple,
where each of the individual elements are not mutable.
Of course, strings are the most popular types in Python.
Single, double or triple quotes.
Let's try out a couple operations on strings then.
So you have a string, you have a length on the string,
you have string one, two, three,
and this is membership checking.
Let's run this.
Okay, so string one gets printed, string two, string three,
pretty standard stuff.
Length of string here is six.
String one, two, three is y, t, which is starting from
one, two, doesn't go up to the third one.
T in string is true.
If it was a Z in string, then it's false.
We can check further individual characters.
Now if you notice, this behavior yeah, I mean of course
now it seems like a sequence.
It seems like a tuple.
I can do this.
So if I was to give you, let's suppose I do this
and I replace the string with this.
Let's see what will be the outcome.
Of course this one is going to not represented
like a string.
The length will be the same.
Yeah, but the representation here,
the slicing representation is going to be
somewhat different.
It's just in principle, like it's a similar, you can use it
as a clothes analogy, it's similar but not the same.
Because when it slices right, it is giving you y, t,
and over there you were getting what is again, a tuple
in return, even with the slice.
Next is place holders in Python.
Now these are very, very powerful.
What they allow you to do is,
think of them like fill in the blanks.
This is like a blank and you're filling it
with a string on the outside.
Now this is a better way to do this would be like this,
where you have a variable.
Now what is the alternative to this?
The alternative to this is welcome to plus course name.
Now this is messy, I'll tell you why.
If I have to write this, you're getting it that it's
right in the middle of it.
There's a space here, I have to take care of the period
to end the sentence, have to give another space.
It's not very readable.
If I would do the same thing here, I would just do this,
which is more readable.
Compare it for yourself.
Now you know what you're going to print.
Over here under the place where it even more helpful
is when you have different datatypes.
Because if you are to do it without using this,
only way to get it done,
you have to use string to convert it
because otherwise this won't add.
Python wouldn't add a string and a number.
It cannot concatenate a string and a number.
It can add two numbers, can concatenate two strings.
This is again, very messy.
Compare this and this.
What is more readable to you as a developer?
Which matters, it matters quite a bit.
The output is the same but there's a difference.
This is more readable.
Think of these like fill in the blanks
and these are the values.
Now this way you have a lot of options, %s, %i,
and they are pretty exhaustive.
So %c is any character.
%i is a signed decimal integer and the list just goes on.
There's of course, a decimal as well, %f.
I would suggest that you go and explore,
but most of the time you'll be using the %s, %d or %f.
Now there are some certain built-in string methods.
One is a capitalize, it will of course as it sounds,
it will capitalize the first letter of the string,
not the other ones.
Then there is string.count, it will consider the sequence
of letters or a single letter that you give to it,
start counting the number of times it occurs in the string.
Next is encoding.
Encoding is basically the process of putting it
in a certain format so that different computers
can support it.
The most popular in coding right now is utf-8.
In case you don't know about it,
go ahead and please explore it.
It's not exactly within the scope of the course,
but just know it's the way that different computers
across the world represent characters.
Because if you look at outside the English language,
you look at the European languages, you have the accents
and look at the Indian languages, which have all sorts
of characters right, how is it that different computers
sitting across the world are able to transmit
that correctly, are able to represent
that correctly on the screen?
Or the Chinese characters?
That is through utf-8.
So whenever you type something, I mean,
as you're transmitting over the internet,
it gets encoded and decoded.
That's what encode is all about.
Max and min will basically work as per the ASCII values.
So max here would give you the value of U,
which is the highest ASCII value and min will give you
the value of A, which is the lowest in edureka.
So what does replace do?
Replace will search for a particular letter and replace
all of its occurrences with the replacement given
as a second parameter.
This is the first one and this is the second one.
This is the number of times it should replace.
If you notice the output, it did not replace the second e,
it just replaced the first one.
Str.upper will capitalize all the letters.
So capitalize is capitalizing only the first one,
but upper will capitalize all of them.
Next is index, now same as find, but there is an exception
if index is not found.
So it will find you the index of the letter k
in the entire sequence.
It's kind of like search operation in a list or a tuple
where you are trying to find which is the position
of a particular element or a particular item
in the sequence.
So if you look at k, zero, one, two, three, four, five, k.
That's where k occurs in edureka.
For the more string methods, of course reversal,
slicing, you've already done this.
Now find is similar to string, however if it doesn't find
anything it will not give you an error.
So let's see, it gives you a negative one,
it did not find anything.
Whereas what the other one will do, which is index,
it will throw an error that not found.
So index basically works under the assumption that you,
the developer or whoever it is in the logic,
has already checked whether it exists or not.
It is not going to help you out in checking.
So find is helping you determine if it is present or not
and if present, where.
Index is just going to tell you where.
It is not going to, it is going to throw an error
if it doesn't exist.
String concatenation, this is something that you have
already seen and string one start two,
that'll just multiply it.
So happy learning, happy learning.
If you look, notice this, right.
It gets multiplied and prints the string.
It gives you the string, just two times over.
Next is sets.
Set is basically an unordered collection of unique items.
Please notice that the items have to be unique
and set is defined by values separated
by commas inside braces.
You can also create a set by calling the inbuilt function,
set, pretty much like how we saw with lists and tuples.
Quickly run this and see what we will get.
If you see that we have got in a set here,
containing the various values.
They've automatically been ordered.
So the question is when would you use sets?
The answer is that if you want to collect unique strings
or integer values from a sequence,
that's when you will, because it doesn't repeat them.
If you notice over here it is not repeating any
of the elements, even though e appeared multiple times,
we only see a small e and a capital e.
So if there are multiple duplicates,
it will sort of give you the uniques.
One example is let's suppose that college administration
is faced problems because during information feeling
students are entering same password and ID.
Now, sets support unique elements and we can convert
the list of IDs and passwords into sets and get only
the unique ones because it will just collect
the unique ones and make sure that the duplicate ones
aren't entered into the database.
Now there are various operations available in set.
One of them is the union operation.
It is one way using this pipe operator,
so it's available on your keyboard.
It's available on different locations on different keyboards
but usually it is available on the backslash key
and you need to use it with the sift.
Now, if you have this sort of a set, which is one, two,
three, four, and then another set which is
three, four, five, six,
and you do print A union B, you'll get a union of the two.
But again, no duplications.
It will join the two together without any duplication.
Look at the example.
So it joined them together.
Similarly, you have A and B.
It is the intersection.
It will give you common ones between the two.
So common between these two are three and four.
That is what it is giving you, the common ones,
three and four.
That's the intersection, so you could also imagine
it like a Venn diagram, that it is a common intersection
between the two.
Then there is the difference as well, which is A minus B.
All of the values in A,
it should not the values inside of B.
So let's try this out as well.
Just try to think of what the answer might be
and let me run it.
So A minus B is one, two, three,
four, five, six, seven, eight.
This is B, the intersection is B.
You can also think of A as like this.
A minus B is also like, both of these are going to be same.
The common ones between A and B gets removed.
Something very similar to what have been seeing
in sequences, membership testing.
You can also do a subset check where you can check
if a set is within another set.
There you need to imagine that B contains A.
So A is a subset of B.
Then of course, with membership checking you can also
do the opposite, not in.
You can do super set.
So for set and super set let's try it using this.
Let's see if set one is a subset of S or not.
The answer is true, it is a subset.
But if you add a value of C, then it is not a subset.
Subset is that it will basically all the elements
of set one are found within all the elements of set S.
The next check is that if S is a super set of set one,
where it contains all the elements of set one as well,
which is true.
Now if we mark it as C and now we run it,
then it is false because S doesn't contain all the elements
of set one.
However, if we set it as C and now we run it,
then it's true again because now, the S set contains
all the elements that are contained in set one.
Now, similar to append, set has something called add,
where you can add an element.
So, let's try it out here.
You have added an element and it automatically ordered it.
If you want to remove an element a certain index
you can do that as well.
This is we'll try removing one from the set S.
Then you have discard as well, again you provide an index.
The difference between remove and discard is that
remove will remove the element from the set.
Discard will remove it only if it is present,
which means that it will not throw an error
if the element is not present or will remove to an error.
Pop works similarly to the list.pop, where it will remove
the element and give it back to you.
S.clear will remove all the elements from the set.
This can create an empty set.
So let's try s.pop as well.
What you have done now is .pop.
So it will basically remove one of the elements,
which is the first one over here in this case.
So you can do a print as well.
Right, it will remove day and then the set was
one, two, three, B.
Next is dictionaries, let's go through it in depth.
Dictionary is an unordered collection of key-value pairs.
It is generally used if you have large amounts of data,
which basically is mixed, if it's not continuous values
that you want to store in an array.
If it's like details of about a person, where the data
about a person can take various forms.
So just think of your facial profile, or think of your
any social media profile.
You can have your name, your last name, location,
you can have your interests.
So you know while and varied data, very hydrogenous.
You could have your age, you could have you birth year,
you could have your jobs, the number of jobs that you're in,
even like your current job.
So all of those things, so the dictionary you will use
in here, hydrogenous data presenting a real life situation
or a real life entity.
When would you use dictionary?
So one example is if you want to create, let's suppose,
any sort of you know, arrangement of data where you want
to store the name, the Aadhar card number,
social security number.
Any Excel sheet that you've ever created, that's kind of
an example of a dictionary where each of the header
is the key and each of the rows are a value.
Name would have multiple values and the key is name.
So when the questions that you ask is what is the name
of this particular person who's at the 10th row,
what is the name on the 10th row
or the social security number of the 10th row.
Then the answer is that okay, social security number
is one of the inputs and that's the key
because you can go row by row but then for checking
which data corresponds to what sequence
of character represents.
That depends on the header.
So it might be filled with data and contextualization
happens through the header.
So very simple, this is how you create a dictionary.
You are given one and two so it can have integer values
as well, but typically it will have string values
because integer values are something that are used
in lists as well.
So it doesn't really help out that much.
The point of a dictionary is for it to immediately make
readable sense.
In the sense that if I were to create a person dictionary,
I would say first name, say last name, age.
I would say something like city and then what dictionary
allows me to do is access the elements by keys
within the square bracket,
which is much more readable than you know, integers.
So if it was an array and I could do something like this
sort, let's suppose same thing you know,
but let's suppose it was like this.
Let's compare the two, is that line 56 is much more readable
to you as a developer, as compared to line 58, 59.
You wouldn't know what person two actually means,
whether it acts as the age or not.
It causes all sorts of problems.
So it's very, very important to understand which data
structure to use this tuples, sets, you can have your way
with it in a way.
Like in the sense that, think about it that if you
have to put a nail in a wall, you can use a rock
but then or you can use so many other things.
You can use a hammer, but then you can use a screw driver.
But if you were using a hammer, so essentially your hammer
can't do what a screw driver does.
A screw driver can't do what a hammer does.
Using the right tool for the right job,
that is where the different kind of sequences exist.
Because they do different kinds of jobs based on
where you're using them and what you are using them for.
Updating is very similar, you just take the index,
you just take the key.
So if I were, for example, update the age.
Let's suppose I do this and then I just do this
and you will see that this will work.
Yeah, now the age is updated.
So it's a mutable data structure.
I can delete using again, the del keyword.
This comes the del keyword where I tell it which particular
key to delete.
So I will say hey, delete city key, I don't need it anymore.
Print person and we'll run it.
That's it, the city disappears, I've deleted it.
Some of the built-in functions, length of the dictionary,
which is number of keys, basically it will go by
the number of keys not by what is against the values.
String will basically give you a string representation
of the dictionary.
This is useful especially for doing web development
and you're dealing with JSON data structures.
Type is just in word methods in Python,
which will give you the type of any variable,
if that's a string or a tuple or a list
or a set or a dictionary, in this case,
or an integer or a decimal or something else.
Given a class object it will just give you the type of it.
There is one thing, if I try to print something
which is not present, so let's suppose I deleted person city
and now I'm trying to access it.
Say it will throw an error.
You will say very commonly in Python key error.
Where the string particularly that here this key
doesn't exist.
What are you trying to access?
Like, I don't know what you're talking about.
Now, in case you don't want errors to come in
in this instance, it's not a way to avoid errors
but it's a safer approach.
So what will happen when you do get
it will return null or none.
You can still put checks against this.
You can say if person.get city, you live in
and just use this, person get city.
However, the problem is that so you can't do the same
thing with this because the moment it checks this line
it will throw and error.
But this is just ignore it.
It can do an else, print, city not found.
City for this person was not found.
Now, there are certain very useful methods, one is .items.
It will return all items in the dictionary.
So if I want to get person.items and then another method,
which is keys, which will return all the keys
in the dictionary.
So let's run this.
So person.items will basically return a list of tuples
with the keys and values inside it.
Keys will just return the keys.
This is first name, last name, and age.
That will present inside the tuple.
Then there's of course, dict.copy where it can create
a copy of the dictionary and dict.clear,
which will empty all elements.
Copy's basically just you can always do create a variable
but copy's just like a safer way of doing it,
other than just assigning to another variable
because it does a deep copy.
So the thing to note here is that what happens
with dictionary sometimes is that because dictionaries
in Python, okay, they are not ordered.
So you cannot predict for sure if I create word,
it'll come in a certain order.
So let's just see what we'll get here.
So of course, it iterates over the keys,
it doesn't iterate over the values.
But there's no order here, if you look at it.
It's city, first name, last name, age.
There's no discernible order in which it has done this.
But if you wanted to do it in order,
you wanted to iterate it in the order of the keys there,
you want it to follow a certain order.
This is how you would do it.
You would get the keys, you would sort it
and then you would iterate over it.
So this is when you want to go the dictionary
in a sorted fashion.
Your dictionary can contain dictionaries as values.
It can contain tuples.
It can contain lists.
It can just go on and on and on like that.
The way that you will access it is of course,
by a square bracket notation and keep accessing
till you get what you want to get.
(pleasant instrumental music)
Why do we need function?
Let's suppose you need to write a program which calculates
the factorial of a number.
That is, if you're given a number four,
then it just calculates the four factorial right.
Just four into three into two into one.
So you keep multiplying the same number by itself
unless the number is one.
Four minus one is three, three minus one is two,
two minus one is one.
You do four into three into two into one.
That's a four factorial.
Now the logic implement the factorial is given here.
You just input a number from the user.
You set factorial that you want to calculate for
and if the number is positive, you keep repeating
the factorial, you keep multiplying it over here,
as shown in this particular line.
So you go from range L to comma and num plus L
and you keep calculating over this again and again
and again.
Now, think about it in this way.
Let's suppose you need to factorial multiple places
in your program.
You want to calculate the factorial of one, two, three,
four, five, like different number.
It won't be a fixed number.
You have to do it in multiple places,
not just in one file, you have 10 files.
What should you do?
Should you copy, paste the code again and again and again?
You can do that but the thing is that it will not be
the most optimal approach.
As a developer you job is to write code or logic.
Your job is not to copy paste same thing again and again.
I mean, that's the job of a clerk.
So keep that in mind.
That way, if you start thinking about in this way,
why am I writing similar kinds of things or sentences
or directions to my computer again and again,
see if you can combine them together.
What are functions?
A function is a block of organized, reusable code.
As I said, you have to use the function again and again.
That's where this definition comes from.
The function is a block or organized, reusable code.
So typically, when you create a function,
the intention should be doing one thing and one thing only.
So as you can see here, that if you just sort of
put the logic to implement factorial in a box
and then use this box for an input and then an output,
like we did in math.
So y=x+2, where y is a function in x.
So in a similar way, you give input you get an output.
Now this is a very powerful idea.
As I said, wherever you think that you're writing
the same kind of code again, multiple times,
you're repeating yourself, put it in a function.
How do you define call a function?
Let's dive into some code.
So the way to define a function is the def key
written in the name of the function.
Here we want to create a function called add.
Now what happens next is that it can pass parameters
through this function, which is the input value.
So A is an input value, B's an input value.
In this function what it does it that it calculates the sum.
So of course you could have done it like this as well.
A is equal one, B is equal to two,
and you could have said something like A+B.
That's it, you have sum.
But why wrap it up in a function?
Because if you do it like this, the next time that you have
to calculate the value of A or the value of B,
calculate for a new value of B
you have to write it again.
So it would be better to put it inside a function like this.
Then you can just call it for the different values
that you need it for.
Now this is the function definition.
The last statement is the written statement
returns whatever is the output.
You have to explicitly ask it to return something
or whatever it is that you need.
It will not assume by itself.
The reason being, that let's suppose you have another
variable inside here.
Now, if we didn't know by itself that they are the same
sum or dif.
Because instructions can be a lot in a function.
So you wouldn't know which one you wanted to say.
So you have to be explicit about it that you want
to return sum.
Now to call a function, all we need to do it
you need to write the name of the function,
follow the parentheses and then the values of the input.
So you see, I'm reusing this logic
again and again and again.
Let's run this.
As you just saw, I can simply type print add(1,3)
and then print add(4,5).
I just asked to print the output of that.
Let's run this.
See if without parameter.
Let me try with a different number.
We get the output again.
So this is how you call a function.
Now why do you need to use the return statement?
So as I was talking about it,
terminates the execution of the function.
It also returns a output.
So the other thing that when you're looking at the control
of the function, where is the code,
where is the interpreter?
You need to think about it in the way that you called
print add, this one.
The control passes to the add function.
With the add function and then executing one by one,
one by one.
Then it returns the control to line number seven
and then the program goes to the next line.
Then it goes in again, then comes out again.
But use the return statement.
Return is really calling kind of like, think about it,
where do you return somebody?
It's not just returning a value, it's also returning
the control to the main part of the function,
where it needs to execute even after line seven.
It needs to go on line eight.
So there's this concept of main.
Let's assume that you have two files.
One of them is one.py and another one is two.py.
Let's suppose one has this function and you are printing
out function top level in one.py.
It has this condition,
which is checking for something very weird.
But underscore underscore name underscore underscore,
it's like an invisible variable present in the file.
Okay, invisible variables sort of is within the file,
which is checking that if the value of this variable
is main, then this needs to have printed,
otherwise this needs to have printed.
Let's take this for a run.
So as you can see, top level in one.py.
This gets printed, of course.
One.py is being run directly when we execute one.py.
So the name wasn't in main.
Now let's see with two.
In two what you're doing is that you're importing one
as as module.
Things that you're doing over here is that you're printing
the top level in two.py and one.py will be run directly.
So with two.py let's see what can get.
Top level in two.py.
So instead of this line number four when we executed two,
we first get that top level in one.py execute.
Now how is this?
This is because as soon as you import a file to another
Python file, whatever code is present
inside it gets executed.
If there is one file that's something written inside it,
it will get executed once you're importing it.
That means not that it will necessarily show that it'll
cause any sort of problems on it.
The reason why this happens is because certain variables
need to be defined.
Certain variables and functions, they need to come
into the computer's memory for that file to be
sort of present.
I could then load those things and that the programs
is contained this feature to be used of Python
is that people will declare certain defaults.
They will want certain things to happen
if a file is imported.
So it is by choice that the design is existing.
It's not a flaw that if you import a file
it will get executed, so importing file is a bad feature.
So file gets executed.
Now this gets executed and then if you see that
this time one.py is being imported while
another module gets called.
Well, it doesn't mean module when once you are running,
might be to the main module, which is calling it.
But since you've imported it from two.py
is not the main module.
It is the two module presented two.py, which is calling it.
Now, then finally we print top level in two.py,
then we call the function, one.function,
function in py and then finally, for two.py to get
that two.py run directly.
So same behavior as that over here
we have imported something.
So whatever is getting execute and the other file
gets printed.
So as it says, if one.py file returns the statements
from this only get executed.
Then two.py file runs, first the statements from one.py
get executed then statements from two.py.
Rather than thinking about it in this way,
the statement from one.py, you have other statements
from two.py.
Think about it in the way of, what order is the input
statement in?
Then put is at the very top, then grab the file,
and then the rest of the lines it imports first,
some in the middle, then a few initial ends in two.py
and then the lines from the other file.
So it's kind of like whenever the importing happens.
It's about which place in the code
does the importing happen in.
Next let's look at some of the built-in functions.
So built-in functions are functions that are given to you
by Python out of the box.
These are just some common utilities or functions
that you will use no matter what sort of programming
language you are in.
Almost all programming languages provide for this
and so does Python.
So built-in functions are the functions which are built-in
to already available Python and can be accessed
by any users.
Now, these functions are available to you as soon
as you load Python for the very first.
So for example, you saw the print statement.
Actually, the print is also built-in function.
You did not have to encode print or start something else.
It was just available to you as soon
as you installed Python.
Some of the common built-in functions are given here.
Sorted, absolute, all.
We will look at these one by one now.
So talking about sorted, it returns a new sorted list
from an items in an iterable,
which is to say that any array, tuple or list,
any tuple, list or sequences.
That's what it will return.
Let me show you an example here.
So you see, it gave us a sorted list.
So it checks that if the iterable, which is again,
the sequence, the list or whatever it is,
if all the elements are true.
So it checks them for being true.
So it's not that it means we all like, true comma,
true comma, true.
It can be true comma, one comma, two comma, three.
Even one, two, three are true values, they're true.
So it's just mode of how Python labels them to be true.
What is considers to be as a truth value.
This could be another datatype.
It could be strings, anything which is not false
or none or zero.
These are true values.
Suppose I change this to none, to none true values.
It gives off two only if everything is true t is how
Python stores it.
That's the true t value where it is not the truth value
but it is kind of like a positive value.
So think about it in a positive versus a negative.
Let's suppose I change this to zero, again false.
I change this to false, of course false.
Any, on the other hand, is a converse of all functions.
That is, it will give you true t for all the opposite.
So even if one of them is true, it will return a true value.
If you go to the very first example,
even then it will return a true value 'cause at least
one of them is true.
But if we give it as false and none,
then it will give a false value.
Next is a Boolean function.
It converts the value to a Boolean using the Python
truth testing procedure.
The truth testing procedure is the same on which kind of
gets applied to all of these iterables in any.
Think about it internally calling bool on zero or none.
Our bool of one is positive.
Bool of bool is also positive.
In the sense that it is a string.
So as you can see, this is like any or all.
They were definitely use something like bool to determine
if the value that is being passed to them is true or false.
So this is really the atomic version of any or all.
Next is chr.
So it returns a string representing a character
whose Unicode point is the integer.
Let's see what it does.
So it basically takes an integer value and it returns
ACII character out of it or the Unicode character out of it.
It goes to 255.
Now what it does is that it opens a file and returns
a corresponding file object.
This is on that we will do in file handling.
Then there is an abs function, which returns the absolute
value of an integer, floating point, or complex number.
So it takes the negative value of an integer
or a float or a complex number.
Python does have complex numbers, by the way,
and it will return the absolute value of it.
Enumerate returns an enumerate object with items
and index values.
We will check enumerate in a little while.
It's further along in the slides, we'll come back to it.
Then there is an int function, which returns an integer
object constructed from a number or string x.
So what int will do is that it will take the string
of one, two, three and turn it into this.
ABC, it will not do anything.
So only when numeral are returned as strings.
Then it does something to them, otherwise it will not work
on any of the special characters or whatever.
It will only work on this kind of thing.
So sometimes you might need to convert
a string into integer.
Then len returns a length of an object.
Length of an object means length as in array
or enumerate.
Length of this, a length of ABCD or length of a tuple.
All of these.
Globals is something we are going to cover in a little while
so let's leave it for that point of time.
Bin converts a number into a binary string prefixed
with a 0b.
If you take this, you get a binary string,
which is prefixed by zero and B, still zero and B.
Now eval is a very interesting thing.
It executes whatever you have returned within strings
and passes it to as a parameter
or executes it like its Python code.
So in the sense that if I type eval print four plus five,
it corrects like Python code.
So it's throwing invalid syntaxes
because it's a security flaw, security problem.
So right now in my interpreter has the securities turned on
where it will not let me execute anything which is
a little more than arithmetic.
That is how usually you'll find the setting of Python to be,
that sort of (mumbles).
Now the thing is that eval is used to execute Python code
and the reason can be dangerous is that
let's suppose eval something and it deletes all the files
from your machine because you just allowed a user
to put in an input that was fed into eval.
The user was a malicious user.
They end up making it execute their code on your server.
Whatever they want to do from your Python code.
Next is sum.
Basically iterable that you give it,
which is an array or a list, and it sums up all the numbers.
So if I do a sum of one, two, three, four, five.
Reverse, on the other hand, will as the name suggests,
just reverse it.
So you have to do this with a loop.
You can't do this without the loop.
It does use an iterable object.
It doesn't give you an actual list over here.
Let's look at the enumerate function.
What the enumerate function does is that it returns in
object to the value that, once you pass through it,
should be in iterable again.
So you pass it in iterable and then it returns a list
of tuples and the tuples contain the index values
and items from the list.
If you notice over here, what you have done in that
we have passed the list grocery, bread, milk and butter.
What enumerate will do is that it will print
the enumerate objects over here.
If you convert it into a list as zero comma bread
one comma milk two comma butter.
Let's take it for a run.
So first is that they enumerate.
This thing returns and object, enumerate grocery,
the type of which is enumerate.
This is basically a function enumerate, which is returning
an object with a type enumerate.
Then you can convert it into a list using the list function.
The list function takes an enumerate object
and it will give you then a list of tuples.
Each of it contain the index and the value of the index.
Let's look at a couple of more examples of functions.
So when it comes to function definition, you define it
by written name, where you do some operations,
take the input, do some operations,
and you return an output.
Call a function, you just need to call it,
just use the student's name followed by bracket.
Finally, with the result in whatever form you want.
Let's look at lambda functions.
Lambdas are basically very small functions
for a quick or small operation where you do not want
to define an entire function.
So it is a shorthand notation.
It is the shorthand way of writing a particular function
which are not more than one expression in line.
So they cannot contain commands and they cannot contain
more than one expression.
So once you look at the syntax,
things will become a little more clear.
They can take any number of arguments and returns the value
of a single expression.
So the syntax goes like this, lambda is a keyword
followed by the input.
I know that there are no braces around this over here.
So the way you need to look at this expression is that
it's lambda then zed.
This is not the correct syntax,
but I'm just trying to show it to you.
Then, what it must do to zed and return at the same time.
This part, or this part, is what would be returned
by this lambda function.
This is the input and then output, where we are saying
that for this input, give this output.
Now notice that how you've written this in one single line.
You could have also return it like this.
So even this would work.
You could still call line seven, nothing changes.
But the difference here is that this way is just a concise
way of writing it.
Then the variable that lambda's assigned to.
So basically what we're doing is that we're taking
a function and we are assigning it to a variable,
like we will do even otherwise.
If you were to define a function answer and then it would
basically be used from here on afterwards.
So like if you had to call this function,
you would call answer in this particular way.
Even to call a lambda, what I would have done is taken
the expression of lambda and I would assign it to variable.
Now let's suppose you need to print the answer
and this is what we do to execute it.
So pretty much calling it like a function again.
We got out answer as 28.
We could do and run it again, the answer would be eight,
two multiplied by four.
Now this is still a named lambda function,
but then one of the biggest uses of lambda,
the reason why it exists, apart from it being concise,
is that it allows you to pass out other function,
a function as a parameter.
So if you had, let's suppose, these typical functions
that exist in any programming language,
map for alternative use.
So what map does it that it takes a list of items
and it maps them to certain thing.
So for example, to show it to you,
first let's look at the result of this and then work
map is doing.
Map, what it has done, is that it has taken each value
in one, two, three, four, five.
If you can just sort of compare.
It seems that it has multiplied them by themselves.
One multiplied by one.
Two multiplied by two.
Three by three and four by four and five by five.
So it's squared them.
Well, this was mapped to this, where you can see that
these set of outputs, they came from each of the individual
elements in this particular item's list.
You can then the same index.
So because they're in the same index in both the lists,
you can sort of map them.
Now this is what the map function essentially does.
Now, what we can do is that in map function the first
parameter is what do you want the current list
to be mapped to?
In the sense that how do you define the mapping?
Does it have to be a square every time?
What if you wanted a cube?
So what lambda allows you to do is that it allows
you to do these very neat thing.
If you notice, this is very concise as compared to writing
a function def squared, x, it'll take items.
Writing for item in items.
So one is that you write all of this
or you write it like this.
There's a definite neatness over here,
but the reason why it takes a function is an object.
Now doing it through lambda is way better than writing
a function like this every time.
The next of these sort of functions are filter.
Filter will check each item in your particular list
and apply a filter to it.
It will remove some items from the list which do not
match a certain condition.
So we took a range of negative five to five,
which is the numbers are from negative to plus five,
ten numbers.
We passed it to filter.
Now what filter does is that it takes the list,
applies this to each and every individual one.
Each and every individual element is applied to go through
this lambda function.
Now this lambda will either return a two or a four.
What filter does is that it keeps the true values.
So for whatever items in this list,
this expression turns out to be true.
This filter function will take those elements only.
If x is less than zero, only for negative five,
negative four, negative three, negative two,
and negative one, where x are these values.
That's what it going to return.
Final is reduce.
So what reduce does is that it takes a list,
applies some operation to it and it basically gives you
a final, single output out of it.
So if you keep applying the same output to again and again
it's like decrement operator.
Think about it like a decrement operator, which takes,
let's suppose I do four minus minus and I keep
calling it again and again and again.
So what did you do that it will reduce the number
from four to three to two to one?
So where it takes the previous output as an input
and keep applying it to the time the list it reaches
the end of the list.
So let's see what this is going to do.
24, so how did this happen?
It took one and initially this was one.
So one comma one.
One comma one is one.
Then one comma two came in.
One was multiplied by two.
Now the value of y is two.
Three comes in, the value of three's turned to a six.
Now six comes input and four comes as an input.
So basically multiplied all of them by themselves.
Now we can do an interesting thing.
Let's see what is the result of this.
We're going to square the first number and we're gonna
multiply it by y.
So let's see what is the value of y because our output
will depend on that.
It tells me what is being squared.
So when we square this you get this value.
But if you look at this one, the value will be different.
So as you can see that you can take two inputs as well
for a lambda.
So any number of inputs are possible but only one expiration
and only one output.
Let's look at scope of variables in Python.
So Python has two scope of variables, global and local.
Global variables are defined the file layer.
They get assigned as soon as the file is run
for the first time.
Local variables on the other hand, are defined within
the scope of functions are (voice cuts out).
So if you look at this example over here,
a is present everywhere in the code.
So a can be used even by this function and turn it to the
global, because it it outside the function.
But if I also use b outside the function
it will throw an error.
As the slide says, the local variable present over here
is b, which is inside the function and can be only used
within the given function.
Here's another example, so when you print a, it is outside.
But if you try to print c, which is inside add,
it is not available to you because it is not
a global variable, it is a local variable.
So everything which is within add is local to add.
Next is memory address.
If you have two variables, a and b, you can know the memory
address of either one of them through this way.
So if you want to know the memory address for variable
in Python, all you need to do is call the in-built
id function, which will tell you the memory location
of that particular variable.
So let's try it out in the terminal.
Let's say I have a = 1, b = 23,
and then I do id(a) and id(b).
As you see, the numbers are different.
In terms of where would you use this, it would really depend
on what you are working on or what you're building.
Usually it is not something that you need to deal with
on a day to day basis because these things,
like id(a) and id(b), I mean, this is relevant more when
you're dealing with lower level things in Python,
but not if you are doing data science or stuff like that.
This is not really something that you would end up
dealing with unless and until you're making something
to memory management in particular,
which is any already done for you.
Next is function arguments.
So the number of arguments in the function
that should exactly match.
These are the required arguments.
So if you look at this function over here,
string has to be passed to the function for it to run.
Otherwise, it will throw an error that print name
missing one required positional argument str.
Name of the argument that it is expecting and that in
the middle call, it is missing.
This is basically saying that it needed input
which you did not provide the same and so it cannot function
just like that.
Next we have something called a keyword argument.
When you use a keyword argument in a function
the caller identifies the argument by the parameter name.
So keyword argument works like this, where I can pass this
particular argument that I'm saying that Annie
is the value of string.
Now, it doesn't really hit you over here,
but let me show you another example on why this is useful.
Suppose you have a function add,
which takes full parameters.
Now, then suppose it is not just a simple function.
There's something that's the sum.
Instead, define a new function called random function.
A, B, C, and D.
It does a sum of A**B, A**D,
plus C**B and it returns the sum.
When you're calling the random function,
you might need to remember hey, okay, what do I need to do?
Do I need to erase the path five?
So you're gonna have to map it,
see what is being used where.
Instead if we just label it, where you see that d = to this.
Consider c as three, consider a as two,
consider b as five.
Now you're telling Python that this is what needs to map it.
So it will not then be dependent
on the original sequence only.
It will take it as you have passed.
This is super useful because as you can see,
now I can call the function in any fashion,
as long as I have used the function parameters clearly.
The order doesn't matter anymore and in fact,
I would recommend this to you that rather than calling
a function sort of, let's suppose, this particular way,
where we could call it like this as well, five.
But I would not recommend calling this.
I think everybody should strictly just follow this
particular approach.
Much cleaner, lot of benefits.
Makes your life as a developer much easier.
Now, the default argument.
I only wanted d to have a value of three,
even if it is not supplied.
So I could just do this and I could remove the value
of d all together.
So I can pass the value of d like this.
This is also correct, but what I've done now in signature
is that I have given it a default value of three.
That d should always have the value three,
even if it is not passed,
complicity should have a value of three.
So another example here, where age has been predefined.
Unless they provide data explicitly,
particularly for a value of 50,
which is being supplied when you was creating the functions.
So you're setting the default value of a particular
function, naturally when you're defining the function.
So another example is of salary.
So where there is a basic salary, let's suppose,
10,000 rupees that is given out any which way,
irrespective of who the person is.
What if the salary's not defined?
The reason these default argument exists is either
because there are particular parameters is going to have
a default value, or in most cases,
it is going to have a set value of a sort.
What you need to do is that only in certain scenarios
you will change the value.
So really for various reasons, people could be using
something of this sort.
So sometimes you may not know how many parameters
are required, how many arguments are required
and you might need variable length of argument,
where you can pass it in number of arguments
because you just don't know how many arguments
will be needed.
So let's suppose you're building a very general
mathematical calculator in Python.
If you think about multiplication, you don't know if the
user's going to multiply three numbers or four numbers
or five numbers at a time.
They might just input seven cross five cross
four cross three or like the factorial example,
where we had to repeatedly multiply.
Any operation that typically be done repeatedly
or have a repeated output, the function receiving the data
in this way can probably not define that it needs
what all number of arguments it needs.
So to handle those cases, we have this start operator.
So as soon as you place it in front of a variable,
in front of a parameter over here, you can pass a number
of values to this function.
See what could happen is that you might think that okay,
this second one is a weird kind of way to call it.
But no, even the first one is correct.
Where this can be zero or and arguments,
there is this option of info Annie Dave.
It could also have a third parameter and that second
and third parameter would have been accepted here
as a list that you can sort of iterate over.
So let me show you a real quick example.
So I've defined a function print users,
but it's similar to one that is all here.
Let's suppose edureka defines one of these.
Then the other users are, let's suppose, admin, ceo,
cto, manager.
Let's call this.
Getting back to this once you run it, as you can see,
the first user argument is edureka and then you sort of see
variable length argument is ceo, cto, admin,
so on and so forth.
It is received in the same order with here.
Now another way is passing keyword argument.
These are star arguments then there are keyword arguments
where you can pass the same thing
but with a key value place.
To look at this at this arg1, arg2, arg3.
One, two, three, four, five, six seven ar args
and then you have these kwargs,
which are named sort of like name parameters,
but cause they are received after args,
they would be available as a dictionary over here.
If you look at non key word argument,
they're available as a tuple.
Now Python doesn't internally.
Again, the purpose of this is that to make the functions
flexible to receive different kinds of inputs
where there's a lot more flexibility in this mechanism
as compared to fixing what the function can be like.
So this also, this are the OOPS concept of polymorphism,
where you can call a function in multiple ways.
It's just about the way you're calling it that the output
will be manipulated.
Now let's look at the difference be procedural oriented
programming and object oriented programming.
So in procedure oriented programming,
the program is divided into functions.
That's the functions is the central concept
when it comes to procedure oriented programming
or functional programming.
In object oriented programming,
program is divided into object.
Now POP follows the top down approach in that it views
entire problems from the very top and it builds downwards.
Where it says that okay, I need something which can
give me an output of this sort, which can then further
be divided into okay, this thing needs to do these five
things, so let's make functions out of those.
You start from the very top of it and you keep adding
blocks smaller and smaller and smaller in size.
OOP follows the bottom up approach, where it looks at
the different building blocks that are needed,
defines them as objects, defines the relationship between
the different entities, and then starts building a system.
Now in procedural programming, data needs to move freely
from function to function in the system,
as definitely is the case.
In OOPS, objects need to move and communicate
with each other through member function.
Some examples of POP are C, Visual Basic,
FORTRAN and Pascal.
Examples of OOP are C++, Java and Python.
But the thing to notice here is that even Python,
for example, through its lambda functions
and through sort of other ways and mechanisms,
allow procedural programming.
It is not that it doesn't allow procedural programming,
it is more for style.
So ultimately, these are the styles of how you program.
No language enforces these, but it rather gives you features
to do those things with it.
So think about it in this way that a car has many features
right, but you might not use all the features.
So the car can drive fast, it can drive slow,
it can drive in reverse, but you may or may not
use all the features.
So driving fast, so some people don't like
to drive fast at all.
They will never hit the maximum speed,
but the car can actually do that.
In a similar way, programming languages have features
built in.
If they allow for you to do a certain thing in a more
easier way or they're more oriented towards it.
That doesn't mean, however,
you cannot make it do something else.
Of course you need to be experienced enough to make
it work like that.
Let's look at a few object oriented concepts.
So Python is very again, object oriented.
It supports it, it has a extensive class mechanism
and structure that we're going to look at
in just a few minutes.
What object oriented does is that
it makes your code reusable and makes it possible for you
to write less and less code.
It also relates very well to the real world scenario,
where you can sort of say that images when you think about
the word object, a number of things come to mind.
So everything is an object.
Like in the sense that a car is an object,
a jet is an object, a table is an object.
Now given that everything is an object,
it makes it very easy for you to kind of imagine
or model the real world in your application.
Naturally, if the code is reusable and you know,
all of these nice features are there,
the programmers, they will produce faster, more accurate,
and better written applications.
However, Python OOPS is different than other languages.
Naturally, I mean, that's why it's a different language
to start with.
For example, in Java you cannot create a file
without defining a class.
So like there are differences and Python is an interpreted
language as we know.
It executes code line by line.
That's no the case with Java.
So Java is a compile language,
where Python is a interpreted language.
So Python OOP is different than other OOP.
Let's look at a use case.
SRT is a multinational company.
This company wants to create employee information sheet.
It should include the name, employee ID, and the progress.
Now let's look at an employee, as weekly, monthly,
and his progress column.
But it will be difficult to create separate sheets
for every employee.
So instead of that, they decided to create one class
of employee's information, create object of every employee,
call class for the new employees.
So this is basically the archetype, where the defined,
hey this is a blueprint of a class.
This is what an employee has.
He has a name, he has a first name, he has a last name,
he has a salary, he has an age, he has a gender,
and this is his role, this is the department.
So, all of these things are called his attributes.
If you look at it right, that every employee typically
would have those attributes in the real world as well.
So a class is nothing but a blueprint of a real world
object, where we try to model that object by using classes.
So when I say model, I mean we try to describe it in code.
When it is the word model comes in, men actually try to be
a little formal with things, especially in the context
of science and programming.
When you're modeling a certain scenario
or a certain situation or a person or an individual
or an object, in terms of English, it is more called
as description.
So when you talk about languages, it's like describe
a ball to me, but then you talk about programming,
they are fancy people, so we say let's model the ball.
So modeling the ball really involve describing it,
during its height, its weight, its diameter, sorry,
color, its shape, its material.
The same thing goes here,
when you're trying to model an employee.
What all is it about an employee that is relevant
to us as an organization?
So similarly right, another thing is that employee
can be one particular thing, but they're defined
by certain parameters but the attributes will change
from person to person.
Because at the end of the day, every person is unique.
Similarly, class is just a blueprint.
So if you look at a cycle, right, or a bicycle
or a two wheeler.
A two wheeler's basically the blueprint
of having two wheels and having a brake.
That is the basic blueprint, right.
And from that it can various objects,
whether it is an automated vehicle like one of these,
or it is not automated vehicle like this.
It is red color or green color or pink or red color,
like it can keep changing, but doesn't change the fact
that this is still a particular implementation
of a particular instance of the cycle object.
So like a human being for example,
the anatomy of a human being is the same across all
the seven billion people in the world.
They're not me, it's the same, that is a blueprint
that you will have certain number of muscles
and certain number of bones in your body.
But then still, each and every person is different.
So this is what, this is the anatomy of a human being.
This is a variation that comes from this anatomy.
Now let's talk about relationship between class and objects.
A class is a template for objects.
It contains all the code for the object's method.
A class describes the abstract characteristic
of a real life thing.
So they are saying that it creates something,
which we see in the real world but in programming terms.
An instance is an object of a class created at run time.
So when you create or use that blueprint,
so just creating the blueprint does nothing.
So you need to, you know, declare a function,
you need to call the function for it to do anything,
similarly just declaring the blueprint of a class
does nothing in Python.
You need to actually create an object using that blueprint.
You need to say that okay, let me bring it to life.
This is the blueprint, let me actually construct it,
let me actually set it up in memory.
I just have the blueprint with me right now.
Doesn't have that actual value.
Of course, they could be multiple instances of a same class.
So to clear the class, what you do is that you use
a class keyword.
Class keyword followed by the name of the class
and parentheses.
Of course a class, at the time of creation,
can take certain arguments inside it and we'll see
what it does with those arguments
but just a very basic definition.
You declare a class, the number, the number being
the name of the class and then statements within that class.
To create instance of a class, what you need to do is
that you need to take a variable, you need to call
the name of the class, and follow the pattern,
which will create the class for you
or create the instance of a class for you.
If you look at this they have print(x),
which says that it's a object.
At so and so memory location.
So then it's like a variable.
This is nothing more than a variable then,
at the end of the day.
Now, definition of a method.
So self points to the class object.
Instead of self.hello, we write ob.hello.
Now right, what is happening over here is
the first parameter of a method inside, so a class
can contain method inside it.
First, parameter is that the class automatically supplies
to itself is hello.
So when you call this ob.hello, internally Python will call
this hello method and it will pass the self object to it.
What is self?
Self is this object, ob, that you define.
Next let's talk about scope of variables when it comes
to classes.
The variables are not declared in the class
within the scope of the class.
They are global variables, likely solve its function,
where certain things which where certain variables
are defined inside the function.
They were local to the function or the enclosure
created by the function.
Similarly, for classes, the same concept applies.
Variables which are defined inside the function
are local to it and variables which are defined outside
are global.
The global variables, as usual, are usable everywhere.
Now if the variables are defined inside the class
they're local and they cannot be accessed
without using the class.
B = 60, if you try to print b, theoretically,
it will not work.
Let me show you how.
I'll just comment this.
Suppose I declare a class demo and it has a variable
to variable b inside it.
Outside I have variable b, which is 45.
I can do this anywhere.
So even this class, I can print b like inside a function
or whatever.
But if I try to do this it will not work.
Even if you notice over here right, let's try to run this.
It says name is a not defined, even though the variable
has been defined over here.
However, if we do this, let's see if we can do this.
This works.
So table .a where this variable has been defined
inside the class.
It is not available unless and until you
refer it to in this way.
So you really need to see many of the variables are defined
and you get any of this error that variable is not defined
but you can actually see that it is present in the code
with the self not spelling state.
You need to check if the scoping is correct or not.
Please be very careful.
These are some of the initial mistakes beginning make.
So don't be demotivated whenever you see this.
It can be very confusing at times, but just look if you can
access the variable from that particular location or not.
Let's talk about attributes.
So as we say that classes are nothing but blueprints
of real world object.
Now when comes to real world object,
it will have certain property.
So for an employee right, an employee is property such
as name, employee ID.
Let's suppose their progress percentage in a certain project
or something else.
That is what really defines the blueprint.
Where when we model the object from the real world
into the programming world, you would be defining its
attributes or its property because that is how
we represent it.
Now, you may say that hey, this is not a complete
representation of an employee.
In the sense that okay, I know something about this person
now but I don't know all the thing.
Which is fine, right.
Hey you know those things about a particular person
that are relevant to you.
Like if there's a colleague, you may know a lot about
their professional life, but you may know little to nothing
about their personal life.
Or you might know about their personal life as well
because they're your friend or something else.
So in a similar way, depending on what you're trying to do
with that object in your software, in your code,
you define the attributes that you need,
not all that is present.
There are two types of attributes, built-in attributes
and user defined attributes.
Built-in attributes, like built-in functions,
are made available on a Python class object
by Python itself.
So there are certain properties,
and you need to think of these.
So one question could be that hey, if the blueprint
of the class is defined according to me as an individual,
according to my needs as a software developer,
why is it that built-in attributes exist?
Like how can Python know anything about?
So if I am declaring an employee list, how is it possible
that the developers of Python would know anything
about my employee class?
How is it that they could imagine built-in attributes?
The answer is that these built-in attributes usually
are there for naming a class and contain meta information
about that class or that object.
They're not really something that what you see the class as.
So these are more things related to how more tools
are information that Python needs, rather than what Python
needs to run that class or to execute that piece of class
or code rather than what you would want to describe.
Then naturally, there are user defined attributes.
So user being you, as a developer.
So you can replace the word user here
with the word developer.
Developer defined attributes that you know,
you are using to model the class.
First, let's look at built-in class attributes.
So I did nothing but I've defined class edureka
and there are certain built-in attributes available.
Now, don't be confused with this.
Why is it underscore underscore?
It doesn't mean anything special.
We'll cover this in a little bit.
It's just a way Python writes certain special attributes.
So special in the sense that it's just a visual recognition.
So it's not that this really can be a string or a number
in it, the value would be somewhere along those lines
or an object.
It is just that as a Python developer, the moment you see
this, you will know that hey,
this is not your usual attribute, it is either a built-in
or it is something else.
So it just creates a context where you can just look
at this and recognize it as something different
from a user defined attribute.
Let's run this, see what the output is going to be.
So edureka.__dict__.
What does this do?
It contains the name space of the class.
So the name space is very important Python concept.
When it comes to finding files, in terms of Python files,
and code execution, you need to know this if you're really
architecting a big piece of code with Python.
For now, we don't need to get into the details
of what the name space is, how exactly name spaces work,
but just something that good to know that we can
sort of get the name space of the current class.
The next one is .__doc.
Now every Python class can contain a doc string,
like a common string explaining the class.
If you want to print that for some reason,
okay you want to print the documentation,
you need to use this method call.
So it needs to be in a particular actually.
The doc string needs to be within single quote.
So if you notice over here,
it has printed this particular doc string.
Now please notice that if you place this doc string
anywhere else, if you place it outside, this will not work.
Let me just show it to you.
There's no doc string, but the moment I place it inside
over here and I run it, the doc string is present.
Next is pretty simple.
It gives you the name of the class.
So __name and edureka class name is edureka.
So what I want is the name of the class here.
So edureka name, I can do the same thing.
I need to change it over here as well.
Right, so if you're wondering where you might use this,
this will be used if you are really building some sort
of framework or library with Python.
So in the sense that this is used by large sort of
systems that are being built because that is when
these things are sort of needed way more than they're needed
if you're just trying to build a website
or if you're just trying to do data prediction
or data visualization or analysis.
It is not exactly that you will use all of this,
but this is required if you are, especially architecting
a big solution using Python.
Next is the module that we are in, which is the main module.
So main is something that we covered earlier as well,
if you might remember.
If name is equal to main the beginning of this class
and we are calling the function from two different files.
It is just the module which is calling the particular file
and we know the main module always calls the file.
Let's look at bases.
So what are bases?
So if the class is deriving from another class,
then bases has a value.
It's just something that if a class can essentially derive
from another class, it's called the concept of inheritance,
much like either one of us inherits from our parents.
So in a similar way, if the class is inheriting from
another class, that is where you use to get the list
of these classes, you get and can inherit from
multiple classes as well.
That's where you get this edureka in this code
and this code, bases__.
Okay, let's come to attributes to find the users.
Attributes are created inside the class definition.
Pretty obvious right?
We can dynamically create new attributes for existing
instances of a class.
So you can also create dynamic attributes
for an existing class.
There's no restriction on that.
You can keep attaching that as you go along.
There are three kinds of attributes.
There's a private attribute which can be accessed
only inside the class definition.
Then a public attribute which can be and should be
freely used.
Then there are protected attributes, which are accessible
from within the class or its subclass.
Subclasses, what are subclasses?
Again, it related to inheritance
or basically derivation bit.
Now, we will cover that soon enough.
But just know that there are private, public and protected
attributes available inside of class.
The way this works is that if you want to declare
a variable as public, you don't need to do anything.
It will just declare a normal variable on self.
The self.pub, this is the attribute on the class over here.
If you want to define it as protected, it needs to start
with an underscore and if it has to be private,
then it has to start with a double underscore.
Now, here is the thing, if you want to access it,
you can try accessing it like this.
Now the thing here is that ob.pub and ob._pro,
let me just show it to you.
Let's see what the output of this is going to be.
So I'm public ran.
I am protected ran, but this one I'm private did not run.
This one did not run even though it is declared.
Python threw an error at us and that's it.
That's all you need to do to make it a private variable.
Now, it's underscore protected, but how is that working out?
Because you told me that the protected attribute can be
only accessed from a child class, that can only be accessed
within the class.
Here's the thing, Python is a little fluid or relaxed
with these rules.
This is more for convention.
So as a Python developer, this is the fact that the variable
can only be accessed by a deriving class or within the class
it's more for convention than a very strict rule
that is enforced.
You will find that to be true even for the private one,
as I will show you that there is a way to even access
the private variable of a class.
So some people find it very weird and odd that why is it
that you can access all three essentially,
but why is it that there the distinction.
It's more on an honor system of each other,
where Python developers just sort of trying to indicate
to others that hey, don't touch this thing,
but if you want to you can essentially,
but if you're doing that then people will notice it.
So experienced Python developers immediately notice
that hey, this person is caught making this particular
call and they're going to question you.
So in case you're working with team developers,
they will ask you hey, why are you doing it like this.
So you better have a good reason for using it then.
Right, so as you saw, Python uses two underscores
to hide a method and two underscores can also be used
to hide a variable.
So we saw it for the variable and the same purposes
applies to a method as well.
So I can declare a method over here.
And if I try to do this where I try to, ob.__privatemethod.
See, Python has already highlighted this.
There's a problem with it.
It's trying to tell me that hey, you can try and run this,
but this will not work.
This will most probably not work.
So let's try that out.
See it did not work.
So yeah, now the big reveal.
Where you can access the private method.
You can access the private method, but you need to do this.
You need to use this really, really weird syntax
of appending edureka the name of the class.
So this portion you need to append it to the name
of the variable or the attribute of the method
and then call it.
So let's try and call it like this.
See this runs.
So even private methods or attributes, they are accessible.
If I try to do this now, ob._Edureka and the casing
has to be exact.
So _Edureka, as it is, you should copy paste in fact
for if you're just beginning out with this.
Then pri and then you run it.
I'm a private, I'm a private, private method.
Both get executed.
Now class variable and instance variable.
So we've already done this where we have defined a class.
We have set the blueprint for a class.
We are defining an instance of class, Edureka over here.
You can set the course and you can print.
Class variable is shared by both instances,
object one and object two.
So in the sense that there are two things here
where if there is a variable like this domain,
which is defined at the class level itself.
It will remain same across different instances.
If you look at this one right, ob1.domain and ob2.domain.
Both of them are data science, but the name, ob1 and ob2
can be different.
Think of this as a common shared property,
a class different instances of a particular class
or particular instance of objects to the class.
Human beings, if the attribute is number five.
If the class variable is number five, the default value
will always be like two, right.
So there is no human being which is born,
which is genetically a single eyed human being.
Yes, there are people who are blind.
There are people who are born with neither of the eyes
working, but genetically, all human beings are supposed
to have two eyes, not three, not one.
That is going to be fixed.
So think of it in that way.
It is that particular class variable which is being shared
as an attribute across different instances
no matter what instance is.
Okay, next talk about constructor and destructor.
The constructor is implemented using the ___init__ function.
You can define parameters that will follow the self object.
The destructor is defined using the __del__, third name.
It is called when the object is deleted.
So think of constructed as building constructor really,
that does the construction of your object.
You labeled a class which basically is the blueprint
of your object.
Now to create instance of that object,
the constructor will need to swing into action to actually
make that object.
Similarly, the destructor will basically come into play
when you delete that object.
So think of like a skyscraper which is being constructed
and which is construction would need certain things
to happen, right.
Destructor would need certain things
to be cleaned up and deleted.
Now, in it basically sample initialization.
That's initialization and this is deletion.
So let's look at an example.
What we have here for example, this is constructor,
which is the private protected and public methods.
Public attributes being set on it.
Similarly, I can define a destructor,
where I can say, let's suppose, I can set it to none.
Let's print.
Now let's talk about multiple constructors.
Multiple constructors are basically used in case you want
the same kind of blueprint of an object that is the same
kind of class in different ways.
You only initialize it different ways.
The way you do it is by using a decorator, such as add to it
class method on a particular object.
What this decorator does is that it passes the CLS
of the self object to this particular thing as well.
So for example, let's suppose I wanted to create
another constructor.
I would declare a class method.
This basically take CLS, which is pretty much like self,
and let's suppose this time I want to...
So if you notice over here right, what has been done
is that you need to call the particular method
at the time of creating the instance.
What I'll do is that I'll create an object
ob = Edureka.myCustomConstructor and pass it a value of 400.
Normally I would have created an object, if you talk about
a default object, create it in this way.
If you need a custom object,
it would be created in this way.
Now, certain key concepts within object oriented programming
are abstraction, encapsulation,
inheritance, overriding and polymorphism.
Let's look into these one by one.
Abstraction, abstraction is simplifying complex reality
by modeling classes.
So what this means is that if you look at a fan, right,
you don't know how it works internally.
Most of you wouldn't know how a car works internally
or how an engine works or how the horn works,
but you just know how to interact with it, make use of it.
If I talk about something as simple as this statement
of print hello world.
Does it work?
How does it work?
How does it print something on the console here?
I'm pretty sure most of you will not be able to answer it
and even I would not be able to answer.
In the sense that it would require for me to look at how
this has been executed.
So the details of how this is happening have been sort of
abstracted away from me.
I need to really open up the Python libraries
and the Python code base to understand how this is
making something print over here.
So when it comes to learning Python right,
if we were to start doing that for each and every thing
we would never end up finishing learning Python.
We would never be able to learn anything for that matter
or be able to learn to use anything.
So abstraction needs to happen and it happens in every
programming language where certain things are working
in a certain way but you don't know.
You don't know till the point you need to do something
out of the ordinary with that particular method or object
or whatever it is that you're presented with.
So in case you want to repair a fan because there is no
repair person around and you know,
or it's just too expensive to repair a fan or a car,
that is when you will end up learning about it.
Not when you just need to turn on the fan.
Encapsulation refers to combining the code into interfaces.
This includes writing methods, creating variables,
using classes, any sort of enumerates, creating a list.
We are encapsulation.
We are collecting it in a more transferrable way
for it to be reused in multiple scenarios where
it is flexible in that sort of way.
Inheritance refers to the fact that there is a relationship,
there is a parent child relationship between most
of the things that you see around there
if you look at the vehicle right, there's a parent child
relationship where bikes belong to the category vehicle
of cars belong to the category vehicle,
buses belong to the category vehicle.
Probably cars, buses and trucks could have another
subcategory after vehicle, which could have been
four-wheeled vehicle.
The subcategories underneath.
Same goes for bikes.
Inheritance is something very naturally occurring
and same is available to us to use in our programming
to model the real world, because that's what software
is all about, after all.
Now there are multiple types of inheritance.
One is a single inheritance in that there is a parent
class, there's a child and there could be multiple
inheritances that a class sees as inheriting from A and B.
Then there could be multi-level inheritance.
This is like a parent and a father and a mother.
This is like a grandfather, father, grandson.
So B is the father of C, C is the son of B,
or B is the son of A and A is the father of B.
Let's talk about single inheritance.
Basically what inheritance does is that it allows us to
extend the properties and methods of one class
to another class.
It is if one blueprint has certain properties
and certain things which have modeling a certain,
which have modeled the world around us.
Where an object, for example let's suppose and object
definitely has a weight in the real world.
Doesn't exist an object that you would call an object
in the real world that doesn't have a weight,
or doesn't have a breadth.
Those are common properties, but then there are properties
that are unique to a certain object.
Such as, let's suppose the color, the color of the object
might be different for different objects
and may or may not exist for certain objects.
For example, if we consider even water, like a mass of water
as an object, then the water will not have any color,
'cause water is essentially colorless.
The color property doesn't (voice cuts out).
The other properties still exist for this particular object.
The way this works is that you can define a base class
and you can define its attributes and its method.
Then you can define another class,
which inherits from the base class.
The way inheritance works is that you just need to supply
the name of the class to the child class
now, when you create an instance of the child class
and you call the function fun, even though it is not defined
over here, but it is defined over here, this gets executed.
So let's try this out and see if this works or not.
So in class base one.
Now I can always have another fun class over here
and it can run this one instead.
Now I'm in class sub.
So the way this works is that it will first look
for the class name within the image of class
and then it will go up the chain of finding it
for the parent classes.
Let's talk about multiple inheritance, where you have one
class which is being inherited from two classes.
So let's look at this one first.
There is the first object, there is a second object.
Then there is a third object which is inheriting
from this first, second, and the first one.
When the inherit method is called,
let's wee what gets executed.
Right, so when we are doing this we are also calling
the super method, which is calling this one.
So this is basically saying that call my superior.
Call the class which is sort of, which I'm inheriting from.
Because it is inheriting from both second and first,
both of these methods get called.
First gets called and then second gets called.
Of course, even the third one is printed.
Then of course there is multilevel inheritance.
Multilevel inheritance, the way it works,
is that a single class, as in the grandfather,
the father and the child.
So you have an animal class which can obviously eat.
You have a dog class, which will bark.
Then you have a baby dog, which will beep,
which has a beepclass.
So you can create an instance of the baby dog class.
You can call d.eat on it, d.bark or d.beep.
If you see we wrote less code, otherwise we could have had
to define these pieces of code again and again.
This is a very, verypowerful concept that is available
to you in programming that you can reuse pieces of code
again and again through creative editing,
thereby creating assumptions by wrapping them within classes
and creating these sort of relationships between them.
Okay, so method overriding.
Now this is something that we did even for
in the earlier example when I showed you classes, right.
I had shown this to you where I could override a method
but let's look at another example.
So if you look at this, this is a parameter which is
myMethod and then there's a child,
which has another myMethod.
Now I can create an instance of to child variable
and I can call myMethod on it.
Now even though myMethod is present in parent,
the child one will get called.
Let me run this for you.
So child method was called.
See how we did that?
This is another limited example.
You have a class rectangle and you have a class square,
which is inheriting from rectangle.
Now when you call on get area, recall the one which is
present inside the square because we want to print it
in a certain way.
A is the area of square, so this will not naturally
recall the get area of rectangle.
It will print a of rectangle, which is not incorrect,
because every square is also a rectangle.
But then we don't want to do that, right.
We want to call get area of square and that is where
a bit of overloading or overriding will help us out.
We have the child class overrides a parent class method.
Polymorphism is the ability to leverage the same interface
for different underlying forms, such as data types or class.
Now in the sense that a millionaire and a student
both have to pay the bill, but maybe the way they pay
the bill is going to be different and paying the bill
changes based on what is being called on.
So polymorphism basically says that the same thing
or same object can behave differently
in different scenarios.
If you are a student, you may pay a bill by taking it from
parents, but if you're a millionaire and you're paying
your bills, you might pay through the credit card
or you might pay through your debit card, you know,
in two different ways.
So since that paying bill is something that is common
to a person, right, but the way it is done it might
change based on who is paying the bill.
Look at this one, right.
The class cat is being derived and the class dog
is also being derived from the animal class.
If you observe that both of them are the top method.
C.talk and d.talk will behave differently.
So polymorphism I mean of course, now you what you have done
is that you have used the concert method overriding
as well earlier, but animal.talk is present on all animals.
However, and cat essentially is also an instance
of the animal object.
This is polymorphism is because essentially both cat and dog
are animals, but when call talk on it,
both of them behave differently.
So same kind of attribute are being called,
but the behavior is different and that is polymorphism,
where you have multiple forms of the same object.
So there is also called the getter and setter methods.
These are more or less in a programming convention.
So you can directly access the attribute so (mumbling)
of a class but then you can also use getters and setters,
where you set values and you get values.
This is basically more of a conventional way
that these are used, these methods are prevalent or used
is because a lot of times you may need to modify you data
before you're using it.
You may need to you know, add symbols to it,
or do some modification before you're setting it
and you just don't want anybody to be able to do it.
You want it to be done in a safe and secure way.
Maybe this object is being set by the database
and you don't want just anybody to access the database
just like that.
You want the access to happen through a proper defined
channel because it is complicated otherwise.
That is where you will use setters and getters.
So let's have a look at an example.
Class edureka, we are setting the course name.
We are creating an object of edureka.
We're gonna print, get course name,
then we're going to set the course name.
So we could have also done this, by the way.
The alternate could have been this right,
but let's suppose you had to use the upper case,
you had to upper case it.
Every single time you had to print only the upper case one.
This is a transformation that you have to do on the data.
When you're updating it, if this is to be done every
single time, it would be cumbersome to write .upper
every single time and make it (garbled speaking).
Or the same thing would apply to this,
where when you return the course name, you're always
setting it to upper case.
So that is why you would use getters and setters.
See how we did that.
(pleasant instrumental music)
Now we are done with loops.
We have had a basic introduction to loops.
Let's move over to standard libraries
that are available in Python.
Standard library is a collection of tools
that come with Python.
It includes the following, it includes the built-in
functions, the modules, the packages.
So some of them we have already been using.
Let's look at some others.
Modules used in Python.
So module basically allows you to logically
organize the code.
It could contain classes, functions.
One simple thing is that I have a utilities module.
Or you can have a DV module, which does all the DV
interactions for you or something else.
So any sort of logical wrapping of code,
logical being not ones and zeroes, but logical for you.
Okay, this chunk of code should go in this file.
This file deals a database and this file deals with,
let's suppose PDF and this file deals with file handling
and this file deals with this.
Let's even divide up the code into logical breaks
and then you call it a module.
Finally, they're nothing but Python files
with a .py extension.
So it's not that they're different from any of the other
Python, it is just that they're called modules
because they're organized in a certain way.
And as I said, a module is a file containing Python
definitions and statements.
So when we use import, whenever you have been using import,
you tell Python to load a module.
When the Python interpreter encounters an import statement
it imports the module if the module is present
in the first part.
So it can locate the module where it is in the same
data tree and the path has been given.
Same data tree, same folder as the current file.
It will end up importing that module.
Now see, there are two things, one is that you have
your own file or your own module that you have written,
which you're importing.
Or you're importing something from the Python
standard library, which will always be available
in the search path.
Third is a built-in function which returns a list
of strings containing the name defined by the module.
It gives the list of variables functions
defined in that module.
So every file in this way is a module and it has
to what dir does it that it gives you all the variables,
everything that is available in that module.
So certain hidden things, such as underscore underscore doc,
underscore loaded, all of these are also
by default available.
Let's have a look at this.
You can close all of these.
So let's suppose I do import math.
One, b is equal to list.
C is equal to and I call the dir.
Let's see what the result will be.
See, a, b, c, and math are present.
If you want to particularly check what all is available
inside math, you can this.
This sort gives you a lot of functions that are available
in the math module.
A lot of functions, lot of variables that might be
and these can be a little confusing.
What are these underscore underscore kind of variables?
But you just need to know that they're internal to Python
and Python uses them to organize and run code.
It is metadata about code that Python uses.
The import statement allows you to import
specific attributes from specific module
into the current name space.
It works like import but it allows you to import
specific things.
Where if you look at this right, we imported so many
of these functions.
What if you do not need all of the functions?
What if you only need cosine or 10 or square root?
Why wouldn't, can we do that?
So what you can do is that you can import from math,
import, suppose we import square root
or let's import cosine.
Now let's do this.
Dir of this particular level.
Let's see what the result is.
So if you look at this now, it's not imported
the entire math module.
It just imported square root.
If you want to use any other function that's available
in the math module, you won't be able to.
If you want to use all the functions in the math module
we need to use import math, which will import the entire
module, but here we are being specific.
We are saying that hey, go to the library and get this book.
Or we are saying that from this book,
get this particular page.
Import math is like importing the entire book of math,
which is very, let's suppose, big and fat.
So you don't want to import everything.
But this is more efficient, where you want to import
specific things.
So it is recommended, this is actually a better practice
that you import specific things,
only the ones that you need.
Because what happens is that they get loaded into
the computer's memory and if you have a lot of unnecessary
functions lying over there, it can cause a problem
and trouble dealing with them.
There is also the import*.
So you can also do this.
From math import*, which is pretty much same as import math.
However, this is not recommended.
This is not recommended because, I mean,
it's pretty much the same but you're importing everything.
Consider the scenario where let's suppose a variable
has been defined in the math module,
which is the same name as the variable you are using.
Now I know the chances are less, but then the more number
of files, the more number of libraries you're reading with,
the chances of collision where math module,
which is another file, has a variable with the same name
as you are using.
So you really want to be careful about what you're importing
as it might cause unexpected errors.
Now, these are not errors that you will see immediately.
These are errors that you will find at run time
because the output will not be correct.
So just be careful about how you use the import statement.
The most responsible way of using it is through
this particular way.
The reload function basically is a very nifty tool.
What it does is that it reloads a particular module.
So let's suppose you're working on two files
and you have made modifications to a certain file
and the other file is already running.
You want to just reload the module that it runs
the fresh code.
It will just reimport the particular module that it's
being asked to reload.
Now, these are some of the very, very important modules
available in Python.
The sys module, the Os module, math module,
datetime, random or dual enter, JSON module.
Let's learn about some important modules in Python now.
The Sys module, so the Sys module basically what it does
is that it avoids the access to some variables
used to maintain by the interpreter and to functions
that interact strongly with the interpreter.
Strongly being that they're closely related
to the Python interpreter.
Now, the way it works is that you import Sys
and some very common things that you can do with it
is that sys.argv.
It stores any command line argument that are passed
when you start Python.
So let me show you an example.
Now let's suppose I go to the terminal
and I execute this file.
I pass an argument.
If you notice, this sys.argv, it takes in the file name
that has been executed.
One, two and three are strings.
So any of the parameters that have been passed has strings
so it can also do this like.
So this is whenever you are executing a Python file
through the command line and you want to pass some
arguments to it.
What sys.exit does is that it basically shuts down
the terminal.
It shuts down the execution.
So nothing different, I mean that happens anywhere,
but syst.exit might be something that you use
when you are looking to exit from the execution.
Let's suppose you are running Python file through
the command line and you want to end execution
after a certain point based on some error conditions
that have happened.
That's where it will tell the interpreter to quit.
Another thing I can sort of show you is if I do this,
when I say import.sys and syst.exit.
See, it's closed the terminal.
It closed the Python interpreter.
Now some other important things in Sys, this is .winver.
So this will not work on my machine because I'm in a MacBook
but if you were not in a MacBook, if you're in a Windows
system, this will give you the version of Windows.
Let's run and see what output I'll get for this.
Yeah, it will throw an error for me because I am not
in a system which is Windows.
So this will not work, but you can try out the other one.
Let's see what sys.flags is all about.
It gives us some flags.
What are these flags and like, these flags are set
to zero or one.
It exposes the status of command line flags
that are available.
Now these flags are essentially how command line
sort of works and it basically gives you those values
of the command line.
Let's look at sys.prefix.
Okay, syst.prefix basically gives you where is it
that the work environment or the Python path lies.
If you have set up which environment or if you have
the Python installed, it will basically tell you
where the Python is installed in your system
that is being used to currently run this program.
Okay, now the thing is that sys module is more useful
again, when you're interacting with the operating system
or you're interacting with the command line.
You will not use it a lot if you're using it
for data science purposes, or if you're using it for
web development purposes.
This is more useful if you're dealing with
the operating system that you're working on.
Let's go to the os module, so os is also related
to the operating system.
It let's you do certain things such as directory
file manipulation.
So it allows you to create a directory.
It allows you to make a directory.
It allows you to remove a directory.
So let's try these things out.
Okay, so what you'll do is that you will start
with some of the things available to us.
First is os.name.
I'm in a posix system,
which is basically what MacBooks are on.
The name of the operating system is not MacBook,
I mean that's the marketing name, but it is posix.
Then the os environ, what it does is that there are
certain environment variables that are stored
on any computer.
This is true for even Windows, where certain environment
variables, environment being the entire operating system,
it has certain variables set.
This gives an entire keymap or a dictionary
of the environment variables.
Then there's something called get login and then there is
a get ppid function, which gives the process ID
of the current function.
If you look at this, it tells me the name of the operating
system, dependent module imported,
returns the mapping of object representing the string
environment, returns the name of the user logged in
on the controlling terminal of the process.
The user which is logged on this terminal is _mysql
has got nothing to do with DVS, it is just the name
of the user on my system.
It returns a parent's process ID.
With os.getppid it returns the ID of the test switches
running this particular Python file.
Now the other ones that you saw on the previous slide,
so we can also say get the current working directory.
Let's do that.
As you can see that I'm on this particular directory
right now, Python Edureka, DS mod three, DS mod three.
I can also make a directory using this.
I don't need to exactly print it, so I know this is shown
as print, but let me make a directory without using print.
I'll just use a directory over here with the name test.
If you notice, a test directory has been created.
I can change the directory now to test.
I'll just comment it out so that we don't create
it on the directory.
We have changed the directory.
I can print os.getcwd.
Cwd is current working directory.
Python knows where which directory it is on
at any given point.
Now that you see the current working directory
is this one.
Now if I try to, let's suppose, import any of these files
over here, it will not work because I'm in
a different directory.
Finally, I can remove a directory as well.
For that what I'll do is that I'll just give it
the name of test.
If you notice, this particular directory disappeared.
Right, so the os path module is very, veryhelpful.
It allows us to do a couple of things to the current path
where we are on.
Why is this needed in Python?
One thing that Python uses the os module or the path module
internally is to find the files.
Whenever you're doing any sort of module import
and you know, you're trying to import functions,
in-built functions or anything of that sort.
Python relies on these methods internally.
It definitely is sort of good to have an idea about
how Python is dealing with these things internally,
that it's not just magic.
Just code which is written by other developers
and internally it is finding out
and doing all of those things one by one.
So let's look at os.path.join.
What it does it that it takes one or more paths,
joins them by using the current operating system's
path separator, whatever it is, whatever the path separator
is, and it basically gives you the return of it.
So let's try this out.
I'll just change it to users apple and users apple shared.
Let's run it once again.
Right, so it joined the two paths together.
Let's suppose they were completely different.
Let's see what the output will be then.
Okay, I hope this is clear to everyone.
Let's look at what absolute path does.
It takes a relative path name and returns a corresponding
absolute path.
What we can do it that we can just give it DS_mod.
So what where we are is DS_mod3.
Let's see what we get by running.
If you notice that it took DS mod three, which was present
within its domain, and it figured out that hey,
this is the backward part of it.
This is the absolute path of it.
You can also try it on a file name.
Let's suppose we try it on this file, condr.py.
We run it.
It automatically found where this file,
the full part of this file.
Let's write out for something which is not within.
Let's try it out for something over here,
where we use info.py.
What absolute path does is that it takes a current
working directory.
So if we take the current working directory
and add the string that you are passing inside over here.
That's all it does.
It will get the current working directory like we did
over here, just going to with this path,
and it's going to concatenate it with whatever
you're passing to it.
Now, normpath, it converts path names from a nonstandard
format to a standard format.
So what is the standard format?
Let's have a look.
Let's again reference the os module py.
Okay, this is the normalized path.
Now let's take this.
This is the normalized path for this.
Now let's suppose we give it this,
let's see the output then.
Okay, this is the output.
This one is for this.
Os.path.split, what it does it that it takes a path name
and returns it in two parts, the directory part
and the filename path.
This is very useful.
Let's try this out.
Let's take the entire path, give it to os.path.split.
You could see that the first part of this is the directory
and the second part is the actual file name.
First part is the entire directory
and the second is the file name.
So with os.path.exists, very simply tells you if the path
exists or not, the particular folder or path exists or not.
We can try it on this and the value will be false
because I don't have a folder of this sort.
But if I try it out on this particular thing right,
which does exist.
So let me try it with this, class demos, right.
This should exist and the value is true that yeah,
it actually exists on this system.
The next thing is that it checks whether there are certain
things that did exist or not.
So if I pass it a file name,
it should say that it's not a directory.
It is not directory, because it's a Python file.
Now if I say this and I ran it, yeah,
then this is a directory, over here.
Now os will not work, what it does it that it walks
or traverses the path that give to it.
It gives you, let's see what it gives in return.
So it gives you generator or it gives you an iterator.
So let's actually use it in a better way
and see what it actually gives us.
Right, so it gives us a bunch of things.
It gives us a bunch of files actually.
It lists the entire set of directories that is present
in class demos/python/edureka.
So probably it's a bit too much so let's go back a little
and try over here and see what we can get out of this one.
Just be only in DSmod3, what is available in this directory.
It should only be the files because only what happens
why you are getting so much output over here
is because there are a bunch of things under Python edureka.
This folder is huge.
So instead, let's try this.
Now, this is a little better.
There is .idea, which is just a system file, ignore it.
There's something called a pycache.
That's related to Python caching something.
But yeah, if you look at this, this is the list of all
the files represented over here.
So what os.walk allows you to do is it allows you to
go into folders and navigate or navigate
to the different folders present on your
system Python files.
Let's talk about the math function.
It provides access to the mathematical functions.
Let's look at a couple of important math functions.
Okay, so math dot ceiling, let's run this one.
Right, so ceiling is it rounds
to the highest number possible, which is 11 in this case.
That's what ceiling does.
Copysign what it does is that it copies a sign or whatever
is on the right inside to the left inside variable.
This is still better way of doing it rather than
just multiplying it by a negative one.
And of course it takes of complex cases,
such as complex numbers or any other kind of system.
Copying the sign is non-trivial.
This will end in a positive value of the negative number
that has been given.
If the number is positive, let's see what is output.
The output does not change and if it's this, an integer,
then so this is f absolute, which is floating absolute.
This is not, this convert even the 19 to 19.0,
as you can see.
Then we have math.exp, which basically is e to the power
gives you the result of e is to the power of whatever
you pass to it.
Then we have expml, which returns e to the power x minus one
where x is two in this example.
This is log, so math.log 10 of 10 return the logarithm
of base 10 to 10.
Now in case you're not aware of e or log,
it's just like sort of mathematics that you need to divide,
but these are nothing complicated at all.
You can just give in the input and get the power
and logarithmic values that you want.
Similarly, we have sine and cosine.
You can get acos, in degree of an angle degree.
Then you get asin, atan, cos.
You can get all the values available here.
So as you said that Python has a wide variety of use cases
which include scientific use cases.
This is why this is in-built in Python,
where you know, as compared to let's suppose C++,
where you need to import it and you need to have it
or some other languages where, you know,
you might need a special module.
All of this comes in-built in Python.
There's in-built support for all of this because Python
is used in domains such as science, particularly.
A lot of scientists use it so that it why these things
exist out of the box in Python.
Similarly, there are Python math, there are angular,
hyperbolic functions, you can convert math to degree,
math.radians, you can get the value of pi,
the constant value of pi,
you can get the value of e as well.
Okay, let's talk about the random module.
Random modules, sometimes you might need a random value
in a certain range or you might need a random number
for whatever reason.
This is where the random module comes in.
So let me show you an example.
You import the module and randrange.
What randrange does it that it gives you a number
within the range of zero to 100.
So let's run this.
Okay, it gave you a value of 52.
Every time you run it we'll get a different number.
What randrange does is that it gives you a number
between zero to 100 but with steps of 20 over there.
So start, stop and step.
It'll start from zero and with steps of right.
So if I need to get better results I should increase
this upper limit.
Now all of these are going to be sort of multiple of 20.
It's not that it is going to give us multiples of 20,
but it is moving in steps of 20 every time.
Then there is randint, between zero to 30.
So let's try this out as well.
Which one to use depends on really the use case.
You can just go ahead with either one of them.
So what does random.getstate do?
It's another method available to us.
It returns an object capturing the current internal
state of the generator.
So what happens is that Python somehow maintains
this concept of a generator, which maintains what random
values have been generated and can actually truly
randomize the next value.
It gets the state of the current generator in its
current form, which is just an object.
Next is random.uniform, it returns a floating point number
such that N is between A and B.
A and B are the values that you are passing to it.
So this number would be between three and six.
So again, it's like saying that any decimal value
between three and six, generate that and give it back to us.
What are the real life applications
of generating random numbers?
It could be for generating a random pass phrase
for a banking system on the internet.
It could be for video games that you developing in Python
that are, that randomly the monster should appear
or the spaceships should appear for the person
to shoot them down.
Or it could be for simulation of any random event
or seemingly random event like a rainfall.
This is where it is going to be.
There are similarly several other similar real life
applications where you might need a random.
But I mean, if you're building a casino website,
let's suppose, and there's a random number to be thrown,
there's a dice to be thrown.
You are building a roulette.
A roulette is basically one of those things that you see
in the casino where you can just throw a ball
and it will land on a particular number.
Now, if you're building any sort of a chance sort of game
with Python, you will need a random variable.
Or even if you're building another probabilistic theorem.
So let's suppose you are building machine learning
you're implementing an algorithm.
Often machine learning relies on random numbers
in terms of a certain language, in terms (mumbling)
that's where you might need the random module.
Next is the date time module.
For date time module includes tools for working
with dates, times and combinations.
So let's have a look at some of the methods available.
Datetime.maxyear, what does it do?
Let's run it and see.
It gives us the maximum year that is available to us,
the minimum year that is available to us on the system.
Datetime.time basically gives us the time object.
The max year is double nine double nine
and the min year is one.
It returns the time object with r minute,
second, and microsecond.
This one returns a time zone object.
But this might not work on all systems,
as you have seen just now, that it did not work
on my particular system.
That module has no attribute time zone.
Just be careful with where you're using this.
It works on certain systems.
It does not work on certain other systems.
Next is a Json module.
The Jason module provides an easy way to encode and decode
data in Json format.
Let's have a look at this.
So Json is basically, it looks like a dictionary in Python
but essentially it stands for Java script object notation.
It is something that is used a lot on the internet
these days through transfer data out, to store data.
It's very, verypopular these days.
It's not that you need to know Java script before using it,
it's just a format for data.
Like MP3 is a format for storing music.
Similarly, Json is a format for storing data in.
Yeah, let me just find the file.
Okay, and just write it here.
So first you need to import the module for Json.
Then let's suppose we create the data,
which is a dictionary, and then I can create a Json string,
which is json.dumps is the method available.
I need to pass the data to it.
Then I can print the Json string.
Let's run this.
So you might say hey, this looks nothing different
than what I printed over here.
But trust me, this is actually different.
This is different because if I do the way Python looks
at it is going to be very different.
Sorry.
Right, so if you look at this, the type is string.
This is not a dictionary.
It can look like a dictionary but it's not a dictionary.
If I try to do this, yeah, this will not run
because string object does not support item assignment.
Okay, so it's a strong object.
Let's talk about regular expressions next.
Let's suppose you want to find a particular data
in a large text of data.
So you want to find, let's suppose, all the email IDs
that exist in this entire text of data.
You want to find the name of certain colors.
You want to find certain words.
You want to find how many times does the word flag appear
over here and replace it with the word flags.
So any of these things, right, where you need to match
for certain patterns or words and either replace them
or find the number of occurrences can be handled
very easily with regular expressions.
Or let's suppose you want to verify email addresses.
So this is something that you see very commonly, right,
that when signing up for a website, it tells you
automatically whether the email ID that you have provided
looks like an email ID or not.
So the thing is that there are so many patterns,
so many ways in which email ID exists
because there are like, websites which end with .in,
.co.in, so on and so forth.
So how do you make sure that an email ID actually looks
like an email?
How did the company figure that out?
That's where regular expressions help us.
What are regular expressions?
A regular expression is a special text string
for describing a search pattern.
So it will describe a search pattern, the pattern by which
we will look for a match in a given string.
You can do that as a human being very well,
but how do we tell the computer to do it?
The way we tell the computer to do it is by
telling it for certain expression and asking it
to match it to a given string.
So you find a word in a string, you generate an iterator,
match one or several of the letters.
This is done internally, thereby generating an iterator,
and then it matches one or any of the several letters
based on the input that you have given.
Or it matches a series or a range of characters.
Then you can probably replace the string or you can match
a single character.
So Python has the re module available for this.
It has several methods available on it.
One of them is sub.
Let's have a look at this.
Okay, so what happens over here is that read or sub
now this over here is what is called a regular,
this is what is called as a regular expression.
This is what we meant by describing a search pattern.
This is a search pattern and what it says is that
match A or D.
So whenever the way it is written it that you write
a, whenever it's something between square brackets,
and let's suppose you give it the characters to match.
Within square brackets, anything is an optional one.
So if you're going to match for A or D
within this given string.
So it's going to find A or D.
It is going to replace them by stars.
So let's run it and see if we get the right output or not.
Okay, so if you look at this one right,
all the A's got replaced by star and all the D's
also got replaced by a star.
Let's do another one.
Now you want to match the pattern abc.
So this is not a or b or c, this is, this one says that
it has to be abc in that sequence and replace all the abc's.
So it replaced abc with a star here and here.
Let's compare it to the first one.
You see the difference?
If you did, let's suppose A and D, and let's see then.
Nothing changes because A D is not in a sequence.
But if we put in square brackets, then here we get
nearly the same output, right.
Now this is another one.
This says A or B or C and then one or two or three.
So let's try this out and see what output we get.
A one gets matched, B two gets matched,
B four has no match because there is neither D near
things are four.
Let's suppose we had D.
Even then it would not be matched because there is no four.
But if you add a four over here, yes then you get a match.
So first, it says that first character should be
either an A or B or C or D.
The second character in the string should be either
one, one or two or three or four.
Let's do another one.
The thing is that this is again,
like one of the simpler ones but then there are
complicated ones as well.
There is this dot, the regular expression,
has the o notation.
Has an o notation of how you describe a pattern.
Let's see what we get with A.B.
Okay what is A.B doing?
It says that basically match any sequence.
A, which starts with A, capital A, then add any character.
So dot stands for any character on the keyboard
and then B.
So A to B gets matched.
AXB gets matched and gets replaced by a star.
AXXB does not get matched.
A dollar B gets matched.
If I were to add another dot, then what will happen?
Then none of the first three match, only this one matches.
This could literally be any character.
It could AB in itself as well.
But it could be dollar in this one and it will still match.
Square brackets, dots, all of these are notations
and now what is AB+?
So it says A and then B in number of times.
Let's run this.
It matches A and then B once.
So that gets replaced by a star.
Then A and all of these Bs get replaced by a single star.
Then A, see, this is left untouched
because it doesn't match.
Getting it, so this says hey, find for a string
within substring within this one.
Sub being that find any matching patterns which are
the first character is an A and the second character is B+.
What does this do?
This says AB but B between three to six times.
So when it is curly braces, then it is saying that B
has to be three to six times.
If it is less than three times, then it will not replace.
So ABB doesn't get replaced.
If it greater than six times, it will not replace.
It will replace only till this point, to the first six
characters then the last three B characters get left out.
It matches this and it matches this naturally.
This is a very important one.
This is for matching anything with starts with ABC.
The starting portion of the string which starts
with an ABC gets matched.
Notice that this is not matched.
Whenever you have this upper cap, this caps operator
over here, it will look for the starting of the string
and it will start replacing it from there.
It will not look at the end of the string.
If you want to match at the end of the string
you need to do it as ABC dollar.
Naturally, if you want to match either way,
then you just keep it as ABC and then it will replace
both of the starting and the end.
But only at the end you need to put in a dollar,
where to start looking from the back of the string.
Upper cap, it will start doing from the front of the string.
Similarly, there are many more of this sort.
Like there is something as complicated at this one.
This is a little big in terms of how, what all is present.
The best way to get better at it is basically by finding
more and more regular expressions
and practicing more and more on it.
Then we just did a substitute but there is also
search and match, which tells you whether it can find
a certain thing or not.
It just gives you a match object.
If this was, let's suppose abc, and you did this,
it would give you none, because it could not find abc
within the string.
So you just confirm the search object to find a match,
but do nothing to it.
You just confirm that okay,
this string exists within this string.
Then there is a match object.
Okay, so what does the match object do?
Match looks for a match starting
at the beginning of the string.
Search looks anywhere but match looks at the beginning.
So this is similar to in fact,
I'll prove it is similar to saying that.
It depends on you how you want to write it,
whether you want to write it using re.search or re.match.
Re.search is usually used when you expect multiple
for a word to occur multiple times.
Re.match is more at the starting of the string.
So if you were matching an email ID, you would use re.match
but if you're looking for the word blue in a particular
text document, then you would use re.search.
So you can use it to match emails IDs or mobile numbers.
This is like a very, very typical application
of regular expressions.
Let's talk about packages in Python.
So a package is a collection of Python modules.
We learned about modules, which was nothing but Python file.
But packages are basically a bunch of modules
logically combined together.
Python package allows you to break down large systems
and organize their modules in a consistent way that you
and other people can use and reuse efficiently.
Again, if you look at programming, there is a lot of stress
on re-usability of things.
Whether it's functions or classes or modules
or packages, a lot of things are about building blocks
and combining them together and making them work together.
So we have done this, right.
We have from backpack, import, pen, papers, calculator.
We have sort of covered this particular bit
where we have learned about the import function
and the found one.
Let's talk about exception handling.
Exception is a signal occurs when an error or unusual
condition has occurred.
A very classic example of an exception is when
you try to divide by zero.
Now, mathematically, the scenario is that programmatically
you haven't written anything incorrect,
but mathematically this can't be done.
So what Python does or any other programming language does
is that it throws an error saying that zero division
is error, it is not allowed,
which is an exceptional scenario.
It is not something which is incorrect,
but it is an exception which has occurred
because of something failing at a really fundamental level.
Fundamental level being something which is just impossible
or unexpected scenario.
Unexpected scenario in the sense that an earthquake
is an exceptional scenario, right.
Think about exceptional scenarios when you think about
exception handling.
Now, you need to kind of take care
of these exceptional scenarios.
You need to anticipate them as a programmer and make sure
that you are handling them within your code base.
This is very important for a graceful degradation
of the software.
This is very important for showing the current kind
of error to the user and it is an overall,
a very, veryrequired code programming practice.
So the syntax and Python is try.
To try says that so what happens the try block is that
you write whatever you are trying to do.
Let's suppose you're trying to divide.
So it will come under the try block as defined over here,
with an indentation, of course.
Then you will handle certain exceptions.
Let's suppose when you're trying to divide two numbers,
you expect three kinds of exceptions.
One exception could be that the user might try to divide
by zero or let's suppose the user might try to divide
very large numbers or the denominator might be so small
that the number is so large that it cannot fit
in your computer's memory.
That is possible.
Like, computers have their limits.
Like probably our computers could write up to in hundreds
of thousands of trillions, but it will reach a limit
where it cannot store a number larger than a given number.
Now, all of these things are different exceptions.
It is better than you deal with different exceptions
in a different manner.
For example, dealing with a very large number would be
very different from dealing with the user
about a zero input.
So probably if the number is too large, it could still,
you know try and figure out how to represent it
in a different way and still give the user a solution.
But division by zero, you might just tell the user
hey, that you gave a bad input.
So don't confuse this with a if else at all
because in the sense that I have seen people use this
instead of if else at times when things could have done
through an if else statement,
but these things, this is to be used more when
you are expecting something unexpected.
Yeah, so I know it's a head spinner, but you are expecting,
you're anticipating.
So rather than expecting, I think I should be using
the word anticipating.
We are anticipating that hey, this thing might go wrong.
I might get a bad input so let me just make my program
malleable enough to kind of handle that scenario.
So a try clause can have any number of except clauses
to handle them differently but only one will be executed
in case an exception occurs.
So we'll raise the value error and only one will
be executed, the first one that it finds.
You can also do something of this sort with
you have to find multiple errors which have been handled
at the same time.
The typeError and zerodivisionerror are being handled
in the same way.
Let's suppose the user inputs a string and you want
in both cases, the user to be told that hey,
you have given a wrong input as compared to a value error,
where something else has gone wrong, right.
As we discussed, so let's suppose this is more about
the large number error and this is the type error
of the zero division error.
Then there is except, which is for handling any unexpected
or unanticipated exceptions.
This is even one that will be on when you can't even
anticipate, but you still know that this might break
because it's a context is that you're doing this
for mission critical applications.
So let's quickly look at an example.
So we're trying to get a input from a number.
Now Python gives us the raise keyword,
which we can use to raise an exception.
What we are artificially raising an exception.
This is not how it is usually done, but we're artificially
raising an exception just so that it can go into this block.
I think we should look at this example instead.
Let's suppose for entry and randomness,
we are trying to divide.
As I said, you can get an expected input.
So let's see what happens when we run this.
Right, so if you look at this, the first time it executes
one divided by A.
The entry's A and an exception occurs.
Oops, this gives you the info next entry
and it goes to the next entry.
It does a zero, nothing can be done.
When it is true, then you get some result out of it.
Then you get some result out of it
but if you notice that what has happened now
is that you have otherwise, your program would have stopped
at the very first step but executed to the end
because you were handling the exceptions
as they were occurring, so some way or the other.
Imagine that you're getting random input
and you're iterating over it and this is how you are
sort going to deal with it.
There is also the finally block in the try statement.
It can have an option finally clause.
This clause is executed no matter what and is generally used
to release external resources.
This is in case, let's suppose you're reading a file,
and some exception occurs and you're not able to handle
the exception but still, you want that regardless
of whatever after the file has been dealt with,
it should be closed because it's a very, verylarge file.
You just do not want to leave it chance, to leave it in
the memory because that will cause a lot of problems.
So that is where you will do any sort of cleanup.
Finally is mostly used for any sort of cleanup
whether a DV connection needs to be severed
or the file needs to be closed.
That's why you will use the finally block.
So this is an example.
We are trying to write to a file.
Let's suppose an error happens
that you can't write to a file.
Ultimately, you need to close the file.
You do not have permission to write to the file
and an exception occurred.
Finally you need to handle it gracefully,
where you need to actually close it.
There are also something called user defined exceptions.
So you can define your own exceptions as well.
It will be logging t error and name error
and value error and zero division error.
These are in-built exceptions.
But as a programmer, it's a fit to your application.
You can define your own nature of exception.
The reason why you would do this is because it would
basically throw errors which are relevant
to your application.
This is something which large companies or large things do
usually, where it's a very, verylarge system
and because what does an error do?
So think about it in this way, that when you see an error
on the screen, it tells essentially a piece of information
to you, which gives you a clue that hey,
something has gone wrong and this is how
you can probably fix it.
Errors, if you look at errors,
they're supposed to be informative.
The laptop or the Python in the printer doesn't say that
hey, something is wrong figure it out.
It tells you, tries to tell you as specifically as possible.
It can understand and what kind of error has occurred.
Otherwise, it would be very very difficult to find problems
in your code.
So similarly, if you're working a very complicated system
right, and you know an exception has occurred
and you know as a developer when you're creating
that piece of code that why the exception will occur.
Why the exception will occur, which is very dependent
on your own use case.
That is where you will define your own user exceptions.
Now how do you do this?
The way to do this is that there is something called
an exception class
you create a class, error class,
that derives from the exception class.
Either directly from it or you can have this sort of
multi-level instance as well.
Really up to you.
And then you just use it.
So you, over here, the class is declared over here itself,
but if you wanted to import it, you could import it
and you could accept, you could raise it as an error.
Now of course, you can add multiple things within the class
in itself and that will help you do certain things.
The stacked way is the way it gets printed.
All of that, but what I would say is that
that should be left for later when you're actually
working on bigger systems.
But for now, you can just keep it in mind that
hey, there is something as creating my own exceptions
as well that I can do.
You can implement methods, override methods that are
relevant in the exception class so that certain thing gets
printed every time or some explanation gets printed.
All of those things as possible.
You can make it as customized as possible for yourself.
That's what programming languages tend to be a lot about.
So good programming languages allow you to customize them
in the way that you want to use them.
(pleasant instrumental music)
Let's suppose we get data of unemployed youth
across the globe from 1947 to 2014.
This has various information about regions,
which have high unemployment rate.
For example, Afghanistan and the region of in that time
period whatever, unemployment rate.
But again, it's data from all over the world.
This data size is going to be huge.
There are seven billion people in the world right now.
Even if you collect only 10% of that data,
it's a huge number.
That's 700 million people then.
Even if 1%, that's a huge number.
Then that is 70 million.
So this data size is huge.
How do you deal with it?
How do you draw insights from it, from such a large dataset?
Excel will not work out at all.
Let's look at use case number three.
We want to look at geographical distribution of posts.
Let's suppose there is a protest going on at Wall Street,
Occupy Wall Street or there's something going on in Syria
or something going on in Libya, where either a revolt
is happening, a revolution is happening or let's suppose
any event is happening.
Let's suppose multiple sporting events are happening
across the globe and you want to kind of figure out
the geographical distribution of a post
with a particular hashtag.
Around the world a marathon run is happening
and everywhere that leads or the supporters are using
the same hashtag and you want to see which geography
is the post coming from because that sort of,
let's suppose, dictates that where is the moment
in terms of if you know where the geography of the post is.
This is the location where it is and the momentum
is changing from location to location to location.
Let's suppose it's all in pictures happening
and within a country it is happening in multiple stadiums
at multiple times.
So then you know where the action is based on
the geography of the post.
Now, large amount of data, lots of insights to be drawn.
How do you do it without using programming?
What is data analysis?
All of us have heard this term over and over again.
There are so many jobs out there which say business
analyst, data analyst, and all the titles in between.
But what really is data analysis?
Data analysis is a process of inspecting, cleaning,
transforming and modeling data with the goal of discovering
useful information, suggesting conclusions,
and supporting decision making.
Let's break it down piece by piece.
You have your raw data here.
This raw data can be even something as simplistic
as the name, age and salary of a person.
Even that is your raw data.
But what happens is that when you collect this raw data
from the world, a number of times it might have
discrepancies in it.
Discrepancies such as the supposed last name
is the same in certain cases.
Let's suppose the age is missing in certain cases.
The salary is missing or something else is going wrong.
There are various things that can be wrong with the data
or not exactly in the format that you want.
So let's suppose the salary is given only in floats.
The decimal values have been included but you don't care
about the decimal values.
Or you need to transform age into integers perfectly.
Whether age is missing you don't want a null value
you want a zero value because null would throw off
your calculations, it would give you errors.
So all of this comes into data pre-processing,
where you transform data into desired format
and you clean the transformed data.
Transform data in the desired format would also be
the row and column arranged,
where maybe you don't want certain columns.
Maybe you want to label certain columns a certain way.
So any of these things where you arrange the data
in a clean cut way that before you can even start
with any sort of analysis on it.
That way you know the predictably, okay there are no null
values, there are no empty values,
there are no extreme values of data.
Extreme being, let's suppose, salaries are given
and you're supposed to figure out the CEO of the company
has a very large salary.
He will drive the average salary to a particular point.
Let's suppose every employee in the company had a salary
of $50,000.00 but the CEO has a salary of $500,000.00.
What will happen ultimately is that you might get
an average salary of $125,000.00 or $150,000.00.
But that is not correct.
That is, what that does for you, is that it gives you
a false impression that this is the average salary
because that is not the average salary.
The average salary might very well be 50, 55, 60 thousand.
So you know, those kind of things are required
to kind of decide how to manipulate the data to get into
proper shape before you can even start anything with it.
Then, the final step, which is modeling the data.
Now, the way you model the data, so modeling is basically
drawing insights from it, which could be just simply
taking an average, which could be take a variance,
something else where you say that oh,
the variance is so much it gives you a trend.
The average age is this much.
For an average age of 24 the average salary if 60 thousand,
which can enable decision making.
Now decision making is not done by a computer,
it is done by you as a human being where you know
the context, where you know whether this is correct
or incorrect.
Not correct or incorrect, but basically what to draw from it
whether to take any action or not and what action to take
in what direction.
So that comes from analysis of strains.
Strains are brought forward by the model
that you prepare on the data.
Honestly, these are the things that you do
in your daily life.
So in the sense that let's suppose you see a discount
on a website right.
You see a discount on Amazon,
you just see a discount on Flicker.
You would compare it right.
That's data.
Sometimes what we do is that when we are comparing items,
on one website it is published as 799.
On another website it is published as 800.
So what we do is that we say, you know let's consider it
to be 800 on both.
One rupee doesn't make a difference.
That's transforming and cleaning the data.
That's pre-processing the data in your head.
Then you prepare a model.
You prepare a model that will give one website gives
10% off and another website gives 20% off if I buy
another item with it.
Then the model is prepared where you do the math
and you figure out what is the total discount
you will get on both the things and whether you want
the second thing or not.
How much money do you have, how much do you want to spend,
and you analyze and you then make a decision.
Again, so there you do not involve a computer
because the dataset was very small in size
and you just were comparing two items.
But just make that into 2,000 items or 3,000 items.
So if it is 3,000 items, then the dataset becomes
difficult to do that analysis mentally.
But the device, when it comes to data analysis,
it is something that we do on a daily basis.
Next which is why Python for data analysis?
Python provides various methods for data analysis
manipulation and visualization.
For data analysis and manipulation we have NumPy
and Pandas and for data visualization we have matplotib.
We will cover all of these in the subsequent classes,
in this module and the next module.
But as I've already told you that this is,
Python is very, veryuseful in this respect
and widely accepted, widely used.
Now let's do a short introduction to NumPy.
NumPy is a package for scientific and I would say,
even data science computing.
It has certain features, like multi dimensional arrays.
So till now we've just run a single dimension array
in Python, a is equal to one, two, three
within square brackets.
But what if you need a multi dimensional array?
Multi dimensional array, imagine a matrix.
A row and a column set up like an Excel sheet.
That's a multi dimensional array.
Then once you have the multi dimensional array,
naturally you want to do certain things to it.
You want to read from it, you want to write to it,
you want to update it, you want to delete it,
you know the typical crud operation.
You want to calculate the sum of the column,
sum of the rows, the average of the rows.
That's the method for processing arrays.
You might want to do an element by element operation,
which is something make a sum.
You might want to do mathematical operations
like in linear algebra.
That depends on, so now of course,
which things you do or things you don't do
depends on what you're trying to do,
but is has multiple features which enable you to do
multiple things.
In NumPy you can basically divide the operations into
three categories.
One is mathematical and logical.
Average, sum, median, variance, all of that.
Second is more scientific.
So if you're familiar with something called
the Fourier transform, if you're not from an engineering
background and taking this course, it's okay.
It's not that you need to know that to be able to deal
with NumPy but it's just one of those things
that is available in generating
that it's a mathematical function.
It's used in signal analysis and so on and so forth.
Something of that sort and linear algebra.
A lot of use cases, even for things like social media
posts and everything.
If you end up doing machine learning after this course,
you will see that linear algebra, the one that you studied
in 7th or 8th grade, has a lot of real world applications.
It's just not theory.
So installing NumPy, very simple.
You just need to go to the terminal and I hope you
already have it installed.
You need to pip and install NumPy.
I can't show this to you because I already have it
installed, but this is the command, pip install NumPy.
As simple as that, nothing more.
It's all small, by the way, no caps and that's it.
You will end up installing NumPy.
Let's look at the NumPy array.
Now, NumPy has something called ndarray.
It's a multi dimensional array object consisting
of two parts, the actual data, some metadata,
which describes the stored data.
This is two key differences here.
First and foremost from a normal Python add in
that it is typically multi dimensional.
The treatment always on an ndarray it is set up
like a multi dimension.
But the clincher, the big big difference is the metadata,
which describes the stored data.
Now, if I have to go to Python and let's suppose
I have to describe a multi dimensional array.
If I had to describe multi dimensional array
without using NumPy, I could do it like this.
How is it a multi dimensional array?
Because I can do this.
This is me saying row zero, this is the first row
and the second column.
Two is in the second column so you can imagine it being
arranged like this.
If you look at this, this is multi dimensional right.
So if I can do this in Python why do I need NumPy?
Because apart from a lot of there things,
a lot of other functionality that you'll see,
one key difference here is the metadata.
So what is metadata?
Metadata is information or data about data.
So let me give you an example.
When you store a file on your computer, any file.
You can right click and you can go to properties
or details or something of that sort,
which gives you extra information about the file,
such as when was it created, when was it last modified,
what is the length of the file in terms of minutes.
Let's suppose it's a song file.
It tells you about the format of the file,
it tells you about certain other things.
It tells you what the author, if there is a description.
You have ever used iTunes or any other media player,
it automatically shows the name of the album and everything.
Where does that come from?
Where does the name of the album or the artist come from?
It comes f rom the metadata that is stored within that file.
So the file in itself is data.
It is song data, right.
It contains ones and zeroes, which make up a song
for us to listen to.
But then this data has information about itself.
Even its name is a metadata.
So one is the song in itself and then the song and format
of a file, but then more information
like when was it created, who created it,
what user on the system created it, who the original author,
what is the copyright on this.
Several, several metadata points about it.
That is one of the key differences here with the way
NumPy arrays are created.
They're not like regular arrays.
They can look like regular arrays as you see,
but keep this in mind that there is a lot of additional
methods available on it,
additional functionality available on them.
Plus they have metadata about themselves,
which can be very, veryuseful and which is very needed.
Once you're, when you're dealing with large amounts of data.
Because it tells you about the data and unless and until
you know about the data you can't do anything to it.
Next is that each element in an ndarray is an object
of data type object called dtype.
So every dtype is something that you will see
even in Pandas and all across NumPy documentation
and even in these slides.
Dtype basically tells the type of the data.
Now ndarray contains a header.
This is an entire ndarray over here.
It contains a header, which is what information eventually.
The first piece of meta information that it contains
about the data that is stored inside these cells,
so think of these individual cells as the array items.
It contains the data type.
Now when you extract an item out of an ndarray,
okay, where you select a particular item to read it
or to manipulate it or whatever,
the data type gets attached and it becomes an array scaler.
So this is ndarray, which is an actual computer skill,
there's nothing but just one item from that entire array.
Now even this one item in this extracted,
it is not just the data.
The data type comes in and attaches itself to it internally.
This is not something that you have to do explicitly.
This is just for your understanding sake.
Where there is a header again, which contains information
about this array scaler.
So okay, why, what is scaler?
Basically just the raw data.
This might be four, but the head might contain that it's
a float, or it's an integer, or it's a complex number.
You might have values as one, two, three,
four, five, six, seven, but is it a float?
Is it an integer?
Is it a scaler?
Sorry, is it a complex number?
That can be determined by the data type
and whenever you detach one of the elements
that data type gets attached to it as well.
Why is this important?
Because even if it's one, two, three, four, five,
six, seven, when if you're applying some sort
of mathematics to it, which involves complex numbers
or you are using it for division,
you need to treat it like a float.
Because a division by a float can give you a different
result than a division by an integer in Python.
If you divide by a float, I recommend that you go back
and try doing that.
If you divide by a float, you will get a float
in the result, but if you divide by an integer
you might get a zero result.
Definitely not get the float value unless and until
you convert it.
So that is why all of these things are very very important
to know the data type, especially when dealing with
this large amount of data.
So let's create a first NumPy array.
The first thing is that you import NumPy
and this over here is an alias.
This over here is an alias.
Import NumPy as np and then you have a method available.
The reason is that NumPy in itself is a large name
so typically it is, when you see code online,
is where you will see a lot of people
use np as a short form.
I suggest that you become familiar with it
because this is something that you're going to see
in a lot of places.
Now, you can do np.array and you pass it
a normal Python array.
That's it.
This is the simplest way to do this.
Np.array(1, 2, 3) and print(a).
So let's run this.
Now the output doesn't look much different
but it looks like just like an array
but if you notice there is one thing.
Let me show it to you.
Let's suppose maybe some of you have already noticed it.
Maybe some of you haven't so I'm gonna pass it
as original array and I'm gonna pass it
and I'm gonna print both.
Okay wait.
Right, notice the difference?
This is not with commas.
This one is with commas.
For this general array, like you have known it
all this time.
Not that it makes too much of a difference for you
when it comes to accessing it, like once you will go down
that road, but just notice that there are subtle differences
and a is the type is going to be like, if we do this.
See?
And this a has metadata about it.
Okay, so this is the simplest way in which you can
create a NumPy array.
Let's look at creating a multi dimensional NumPy array.
This is quite simple.
We do nothing but one comma two comma three comma four.
Now one thing is that this needs to come within
square brackets as well.
Don't make this mistake of missing the square brackets.
In the beginning people tend to miss it.
There are items, number of these items within this array
will decide the number of rows.
Let's print it.
Again, see the commas are missing and this is written
in a certain way and even the output is coming
in a certain way.
Whereas if we had a multi dimensional array with list
inside it it would come with commas and it would come
in a flag structure.
Let's see what happens if we add an other value to this
or if we add another column.
Then it is not able to represent it like earlier.
But this converts it into an array.
The columns are the same, it will just treat it
like a single dimensional array.
This is a single dimension.
If they are same, then it will make a multi dimensional
array out of it.
It will not assume that right, it doesn't know what
the value of five is going to be so it's just going to
create it like a single dimensional case.
It will not create a multi dimensional.
This something about reissuing it, right.
I pass it a list, it just created an ndarray artifact.
Another way to create an array over here is I can
use the arrange function.
The arrange function.
I say zero to 1,000.
And I run this.
See again, no commas.
This is slack, this is not indexed like this.
This is not a multi dimensional array.
This is a single dimensional array.
It would exclude the last value, 1,000 is not included.
Starts from zero first is included.
The second one is not included.
Another important method over here
is creating an array of zeroes.
Now, some of you may wonder what is the usefulness
of creating an array of zeroes.
Well, the use case is a lot in scientific computing
and machine learning, again.
Sometimes you need things like this.
What it takes is it takes a tuple of the size that you want
and it gives you that.
So it's five plus five.
If you look at it a multi dimensional array.
I can do, let's try with five comma three.
So three columns and five rows.
Three columns, five rows.
Let's try it with something random as five comma 15.
15 columns, five rows.
We can to 15 here as well.
So all of this.
Under the method available is linspace.
Linspace creates a linearly spaced vector.
Now this is relevant if you are familiar with the concept
of a linearly spaces vector, which is sort of a fancy
way of say that okay, these numbers which are separated
in a linear space, in a sense that in this particular
dimension, there are equal steps between them.
What it takes is three parameters to start,
stop, till what point it needs to go and then steps.
So it's kind of like, think of the x-axis that you draw
on a graph, axis that you draw on a graph.
This is one of those things which will help you (mumbles).
This linearly spaced vectors can come of help over there.
So I'm gonna cover np.linspace, zero comma 20 comma five.
Let me run this.
I get this output.
I can do it like this as well.
It'll have to be a shorter one, yeah.
So this is the number of steps.
So don't confuse it with the differences.
It will not be two, four, six, eight.
This is going to be the number of steps
that you want to reach 20 here.
So if it is two steps, then it is going to reach it
in two steps.
If it is going to be 10, that is where you will get
random numbers.
It's going to create 10 values, which are ultimately
going to reach to 20.
In steps of 10, reach 20 in steps of 10.
And steps of 10 being in 10 steps to divide 20,
so take the second number and subtract it from
the first number.
That, in this case, is 20.
It says reach 20 in 10 steps, not exactly 10 steps.
That is what this is saying to it.
Or we do it as a, so it says reach 20 in exactly eight steps
or reach 20 in exactly six steps,
where zero is always going to be the first step
and 20 is always going to be the last step.
So you need to figure out the rest of the four steps.
Okay, so we did this even earlier, right,
where we took an array, a list and converted it
for our use case.
This is pretty much the same, where you can use the as
array method as well to convert an existing sequence
into an nd array.
Let's talk about restructuring a NumPy array.
Restructuring, what do we mean by that?
It converts a linear array of eight elements
into any sort of reshape or resizing that you want
to do to it.
So let's try this out.
Let's take this and let's create an array of size,
two comma.
Let's create an array of eight zeroes, okay simple.
Then let's resize this into two comma two comma two.
We will print A first here
and then we're going to print A here.
First, it falls like this and then we reshape
into this size.
Let's see if it can convert it into four comma four or not.
Or four comma two.
Now why can't I do four comma four?
Cannot reshape the size into four comma four.
So four comma four is basically going to multiply.
Four cross four is 16.
So if you have eight elements, you cannot make 16 elements
out of it, naturally right.
You need to do it in a way where you can make finally
eight elements out of it.
It can be transformed to two comma four.
There is two rows and four columns.
Or you can transform into four comma two
or one comma eight, which is the same right.
Or eight comma one, where eight rows in one column.
Now the use cases again, whenever you're doing any sort of,
so let me just give you a little bit of background
about where you might end up using this.
When you're dealing in machine learning right,
machine learning is ultimately it's nothing but
the mixture of maths, statistics, and programming.
That's the overlap.
There are these three Venn, so think of a Venn diagram
where maths or statistical maths, linear algebra
and probability and programming, all of them meet together.
That is where machine learning happens.
Now when you're talking about linear algebra,
linear algebra they use a lot with matrices.
Matrices are multi dimensional arrays.
So a matrix is nothing but what you see in Excel sheets,
shows in columns and that is what we're doing here
with the NumPy array, where we had a normal array
but we wanted something more.
In that domain, there are a lot of mathematical computations
that need to happen.
Transforms need to happen.
Arrays need to be reshaped to perform certain mathematical
things on it, functions or operations on it.
That is where all of these are used by Python developers
into like, for either making those libraries that are
doing machine learning or for adding their own
machine learning code, where these transformations
are really, really helpful.
Just good to know for now.
If you're wondering where am I going to use it,
eventually when we get into the application part,
there you will see that we're going to reuse these things
again and again and again.
It's very, veryimportant that you know about these.
There is another one called ravel, where so ravel what
it does is that it levels up the array.
It flattens it out.
So if I've gotten a reshape right,
let's reshape this into two comma two comma two.
Then let's print the ravel out of it.
What will it do?
It will basically just give you the same one.
We can do a reshape and then a ravel.
It will take this flatten, this 2D array and flatten it out.
That's all that ravel does.
You can take a flat one and reshape it
or you can take a shaped one and flatten it out.
It's just a two way transformation.
It is a converse of what we did earlier.
Let's talk about the indexing of NumPy arrays.
NumPy array indexing is very very identical to Python index.
If you want to collect an element, so let me remove
all of this and arr = np.arrange(2,20).
Now, I want to get, so earlier as you saw right,
where we were selecting a particular element out of it.
Let's run this.
Okay, now what we have done is simply just access
the sixth element out of it.
Let's look at the type and see what it is,
because over here it looked like an integer.
Is it like an integer?
No, it's not, it's a NumPy n64 type.
This is one of the gotchas, where you know initially
you might get stuck where you are looking at it
like an element but like a simple number eight,
but it is not simply a number eight.
Let's try another thing on it.
This is what we've got and we picked out the sixth element.
Let's see if we can print element plus seven.
We can, okay.
But still, understand that this is enabled by Python
that it is allowing you to add an integer
to a NumPy array element.
This data type and this data type is not same.
Element has a different data type,
seven has a different data type.
This is an object of the belonging to the NumPy libraries,
where seven is the normal seven integer value that we have.
Yes, you typically cannot add, so this is like adding
a cat to a dog.
The real world doesn't give you cat dog, right.
What is happening is that Python is handling it internally.
Part of the same that we need to be conscious about
what we are doing.
Let's try to see if this is a type conversion or not.
What is a type of element plus seven?
Is it giving us a NumPy object or is it giving us
an integer object?
See, so what Python is internally doing is that it is
taking the seven, first converting into this NumPy n64
and then adding it.
Because if this was simply eight, okay,
if this was not the NumPy one you'd get typed as int.
You see, eight this, eight plus seven it's giving us
NumPy n64 and this type is different of eight plus seven.
Okay, what is slicing?
We covered it a little bit when we were covering Python
of the first few classes.
Python's concept of list slicing is extended to NumPy.
The slice object, now over here we have a more sophisticated
way of doing slicing, where we can create something
called the slice object.
That is constructed by providing start, stop and step
parameters to the slice class.
So this is a slice class, this is a constructor.
We covered this in the last class.
I hope you guys remember.
We're creating a slice object and we're just passing it
to the array, the NumPy array.
Let's try this out.
So we have an array of np.arrange, let's make it of 20.
Then, we are going to cover it as np.arrange.
Okay wait, sorry.
Here we slice, slice off one comma 10 comma two,
start stop and that.
Then we are going to print arr.
Let's print arr first as well.
So it started from the first one, went up to the 10
in steps of two.
One, three, five, seven, nine.
One, five and nine.
Now there are various ways of slicing.
So you could have also done it in this way by the way,
where we could have used the similar notation,
so where we wanted to start from two and let's suppose
go to nine.
You could have done this as well.
This is something that we covered in array.
Even that works, but this is a more sophisticated way
of doing it using the slice object and you can not use
a slice object and do it directly like this as well.
You can always use this one where it starts from two,
it starts from 10, the 10th element, and goes up to the end.
Or you can take it to the last element that you wanted
to go to.
If I wanted to go to 12, start from the top,
so it goes to 12 minus one.
So it says 12 first twelve elements.
Now, here is again, now this is something very very cool,
very very nice that we can do with this, where we can
extract specific rows and columns using slicing.
We can slice the first two rows
and the first columns like this.
Let's try it out.
Zero comma two and zero comma two.
You do not have a multi dimensional array.
But let's create a multi dimensional array.
Okay, let's try to cover this, which basically brings in
everything but first is for rows, the rows that we want.
We can want zero to three rows.
Let's do zero to four and see if it gives us an error.
No it won't.
It doesn't give us an error, it just ignores it.
The rows doesn't exist.
The first is rows and then the columns
that you want with it.
Let's see if we give it a new column.
Yeah, it will still work, but not really.
It would be confusing for other developers in terms
of why you have used a higher index, so don't do that.
Just keep in mind the number of rows and columns
that you actually have.
You could also tell it that hey, just select the rows
in this range.
So we could also do it like every row after the first one
or we could tell it hey, ignore the first column
or could tell it that hey, include everything
till the second column but not after that.
It gives us the same result but we can go over it like this.
This one says everything after the first row,
which is the first row will contain one, so it ignores one.
Everything from the first row onwards.
This could have a zero as well, right,
which is the traditional one,
which would include one, four, seven.
So we do it as one then becomes four and seven.
This one says hey, all columns up till the first column,
which is not going to include anything which is like
two, five and eight.
This column gets dropped after this two,
this column, two, three, column five, six,
column and eight, nine column, that gets dropped.
Let's look at few NumPy array attributes.
These are very very helpful.
We can get the shape of the array by printing arr.shape.
Just three comma three, makes sense.
Next we can get the dimensions of the array,
which returns the number of array dimensions.
Because of the 2D array, so it will return two, right.
Next is item size, the number of items in the array.
This is the length of each element of array in bytes.
It will take each element and based on the array type
it will give you the number of bytes,
number of bytes for that particular element.
So one is usually showed in eight bits.
Integers are stored in eight bits and that's where
it will give you the size.
Not very useful, but sort of something good to know.
Right, there is something called NumPy.empty.
It creates an uninitialized array
of specified shape and dtype.
So uninitialized where it doesn't have a value,
it just has been created.
If you do the following constructor where it take the shape.
So you need to give it an array, which defines the shape
of it and the data type.
It will just basically give you something with zeroes
but essentially it's uninitialized, it's just empty.
Let's try this out.
So we give it a shape, let's suppose, three comma five
and we give it a dtype of int.
Let's run this.
Okay, so what it is doing right now, it is picking up
random values and it is just populating those
in the rows and the columns.
But it will look at the size, the size is being maintained.
Let's try with float and see what we get.
So again, the random values, it empty is basically
useful that way.
Now it can also give zeroes, but then that just depends on
whatever values it picks up.
Next is reading and writing from files using NumPy.
NumPy provides the option of importing files
directly into the ndarray using loadtxt function.
The savetxt function can be used to write data
from an array into a text file.
So let's try this out.
Let's savetxt and test.txt
and let's pass it an array.
Sorry I got the spelling wrong.
Let's try this out.
Let's see if a file got created.
So yeah, test.txt got created and this was returned to it.
Of course it writes it in the Python format,
so let's try to create an array using some other ways.
Let's create an array using this particular connotation.
Let's see what the output is then.
Right, so of course if you need to write in a different
way, you will need to specify that in those with
higher options, but for now this is sufficient.
Similarly, you can load it up using np.load.txt
and you can give it the name of the test file.
So you can load it up from the, now notice that what is
there on the file and what is being loaded
is slightly different.
It's just that when it is storing it,
it is storing it in a certain way.
When it is accessing it it is accessing it
in a certain different way and there is minor differences
are ultimately the object here is not to have the file
in a human readable format as such.
It's more about storing the data on the hard drive
and using it in the NumPy array not any other way.
Now, another thing that we could do is that you could
save it so CSV file, using something called the delimiter.
Now delimiter, what is does is that it separates out
the rules of subsequent values that are present
in the column using a comma.
So delimiter is something that you specify.
I'll just show it to you quickly.
First let me save the file in a certain way.
First, let's save the file and we will use the same
save text feature.
Okay, we are not going to try out.
We can use the same method and we can specify a delimiter.
So the first delimiter that we specify
is going to be a comma.
We are going to separate our data using commas.
Let's try this out.
Let's see what is there in CSV.
Now you see the data comma separated.
We could have also separated it using any other character.
Let's suppose you want to separate it using the semicolon.
Let's check the CSV file now.
Now, these have been replaced by the semicolon.
You can also use anything.
We can use something like a dollar, we can use any sort
delimiter that we want.
It is purely up to us if you want a delimiter of this sort.
Of course this is best to use comma because that way
it becomes clear.
A reason that you would not use comma is when your data
in itself contains commas.
So if you have any string, if you have any sentences,
let's suppose a Facebook post or a Twitter post,
which has commas inside it, then probably using commas
is not a good option because the library or your Python
programming, it will get confused between what is
the comma coming from you delimiter or the comma
separating the return or what is the comma
that is coming from the data itself.
So likely it will read, it will break it up into
comma that it will also consider the comma that is coming
from the data in itself
when you're storing it in a .CSV file.
So please be careful about what delimiters you're using,
but typically if it's a numbers kind of data,
if it's just numbers, you can use a comma
and it will work fine for you.
Other thing we can do is we can use the genfromtxt function.
Over here we need to specify the delimiter.
It's just pretty much the same thing as I did earlier
when we're using the low text file.
This low text when we were doing, genfromtext does something
similar and it allows you to again,
deal with data in different formats.
Maybe you need to pass the file to somebody else
and that is why you would use these file operations.
Or you're receiving the file from somebody else,
you're receiving the file from the sales manager.
The sales manager asks you to go through it
or something else and you need to create a report on it.
That is where you would end up dealing with files
because a lot of times, data in companies maintained
in Excel sheets and files and you will export it
to a CSV or something else of that sort,
which is compatible with NumPy and use it.
Now let's go on to Pandas.
Pandas is an open source library providing efficient,
easy to use data structures and data analysis.
Now Pandas is basically built on top of NumPy.
Pandas is built on top of NumPy and the name Pandas
is derived, not from the animal, but from Pandas data,
which is sort of a very technical term,
an Econometrics term from matplot, multi dimensional data.
Pandas is well suited for tabular data which heterogeneously
typed columns, which basically means that any sort of
rows and column based data which contains different
types of columns, which contain different types of data
types, heterogenous.
It is not homogenous, where one could be name, age,
sex, gender, city, different strings, Booleans.
All of that heterogeneous data,
it is very very well suited for that.
It is also suited for ordered
and unordered time series data.
Time series being that anything which is time stamped.
So let's suppose there is a sequence of events,
like you were looking at the geographical use case,
where it's a sequence of events which is marked
by a time stamp.
So any sort of ordered and unordered time series data,
that is also what Pandas is very very helpful for.
Now, the third thing is any sort of arbitrary matrix data
with rows and columns labels.
Which is pretty much similar to the first one.
Matrix data, rows and column labels and I mean,
it's like an Excel sheet.
It's kind of describing an Excel sheet in technical terms.
Any other form of observational statistical dataset.
The data doesn't need to be labeled at first.
Pandas is very powerful.
It gives you labeling tools as well,
where you can label your data.
When I say label your data, I mean give a column name
to certain columns which are not present originally.
It will accept them and allow you to add your own
column names as and when you want.
Now let's look at installing Pandas.
Same command, you need to pip install Pandas.
I'll just have a look at the virtual environment I'm in.
So I'm at...
So this doesn't have Pandas and I'll have to install it.
Okay, so now Pandas is installed.
We should be expecting that it will be available to us.
After the Pandas installation, let's look at the data
structures within Pandas.
Kind of like how ndarray or NumPy array was available to us,
similarly Pandas has some data structures
that are available from itself as well.
The first one is series.
This is labeled homogenous array of immutable size.
Immutable size being that you cannot change the size
of this array after you have created it for the first time.
It is a labeled homogenous array,
where the data is not heterogeneous.
Now of course we looked at Pandas initial thing was
that it could handle heterogeneous data.
For that you need dataframes.
Dataframe is a two dimensional data type available.
It can have heterogeneous types and it is size mutable,
where you can change the size.
So as compared to series, for this one you can change
the size of the tabular data structure.
So it like saying that you can add rows and columns
even after you've created an Excel sheet.
Series are most directly you cannot modify the number
of values inside it after you have created it.
Third one is panels.
It is a labeled, size mutable array
but the dimensions are three in number.
So it's a three dimensional array where even individual
element can be a series or a dataframe in itself.
So that is where the three dimensions come in from.
Panels is the most complicated one out of the three,
because it's three dimensional so it can be a little
difficult to deal with it.
But it's one of the important ones for sure.
One thing to note, notice that all the above data structures
are value mutable.
So what does that mean?
Value mutable means that we can change the individual values
but in case of series, you cannot change the size.
So you can have four elements in that homogeneous array
and you can change all the four elements,
the value of the four elements, but you cannot change
the size of the array in itself.
Please, please notice the slight minor differences
when it comes to mutation.
Mutation being change.
So what kind of changes are allowed
on these data structures?
You need to be sort of familiar with when you're using them
always there's a chance that you might get stuck
when you're programming.
Let's look at series.
Series is a single dimensional array that contains
homogeneous data, that is data for single type.
All the elements of a series are value mutable
and size mutable.
So the elements are size mutable, value mutable
and size immutable.
So you cannot change something which is like a float
to an integer because the size of the element
in itself is fixed.
Size of the element and size of the entire series
array is fixed.
You cannot change it.
So data can be a multiple time, such as ndarray.
So you can create a series which contains ndarrays.
Or which contains lists, constants,
series or dictionaries, et cetera.
The indexes need to be unique, hashable,
and have the same length as data.
It defaults to a certain value.
We'll look at this in a short while.
Data type of each column, if none is mentioned,
it will be inferred automatically.
So the data type needs to be present for each column.
It will determine it by itself
if you do not give it explicitly.
When it comes to copying the data, it deep copies the data.
So what is deep copy?
Deep copy is that it will copy as much as it can about that
thing that it is trying to copy.
It is said to default, false is default,
but it can do deep copy as well.
So deep copy means that any of the references,
so even the memory references, it's not a copy,
it's not a Xerox copy,
it is like the actual document in itself.
So it is kind of creating a duplicate which is very very
valid as the original one as well.
It is not simply a Xerox.
It's like getting a duplicate drivers license,
which is as good as the other one, the original one.
It's is not a Xerox, it's not a simple Xerox okay.
This is very important to know when you are you know,
trying to copy it and pass it around
and you make changes to your series data type,
series variable in one place but it does not reflect
in the other place.
So you need to know where your deep copying
and where you're shallow copying.
The opposite of deep copying is shallow copying.
Let's create a series.
What we need to do is that we need to import Pandas.
We need to create a series.
Let's run this.
If you see that it has given a series, it is an empty one,
as there is nothing present over here.
The dtype is float 64.
Now let's suppose you want to create a series
with some data inside it.
One of the typical ways to do it is by creating
and using an ndarray.
Let's also import NumPy over here.
Let's create an atarray.
Let's pass it to series.
Now let's see what is the output.
Okay, sorry, needs to be one dimensional.
Right, okay this can be a little confusing,
so let me instead give it in the reverse order
and let me give a little bit of random values
so that it doesn't seem like the exact same thing.
Right, so nine has an index of zero.
Eight has an index of one.
Two has an index of eight.
The second eight has an index of two.
This indexing the left, this column zero, one, two,
three, four, five, that is automatically by Pandas.
This is not present in the original array.
Now you can define the indexing by your side as well.
I mean, you need to pass an additional parameter,
but if you do not it will automatically index it.
Indexing being that it will automatically set up an index
next to it to which you can access it as I will show you
later down the slide.
But just note that this column over here, this one,
had been auto generated.
This is something that Pandas series will always do for you.
Now you can also create it using a dict.
So let's suppose you want to create a data dict
and you want to set A and one, B as two, C as three.
We pass the data dict and then we run it.
See now, the index is that it has picked up
has been given by you in a way.
When you give it a dict it will automatically create
indexing using the keys and the second calumn
will be the values.
So when you give it an array, the indexing is done.
You can sort of think it like this that when the array
is given, the NumPy array, the elements are indexed anyway.
So this is zero, one, two, and that is what indexes.
In this case, when it comes to dictionaries,
the index is A, B, and C.
So index being, so think of an index of a book.
You need an index, go to a certain page.
The page here is one, two, and three in this one.
The index is A, B, and C.
If you want to go to one, you have to refer to the index
of index where it is and the index here is A.
That is what the series used
to create an index for A, B and C.
Next is accessing data from a series.
Slicing works, as usual.
In the sense that let's suppose it had more values.
Let me just do that.
It will access two to four minus one.
Of course the last one is not included
any slicing in Python.
So four minus one is three over here,
which is the value of B.
That is how it will work.
We can always do this as well till the end
or you can do it like this.
What happens if we give it a value that doesn't exist?
Just to see, what if we give it seven?
So yeah, it will just stop at F and not proceed further.
So yes definitely, there is a similarity between this
and arrays, but then again, as you proceed with using
Pandas in the future classes and the future modules
in this one, you will figure out that hey,
this is the normal functionalities present,
less extra things are present on all of these libraries
and all of these new data types that we're learning about.
Next is dataframes.
Dataframe is a 2D data structure in which data is aligned
in a tabular fashion, consisting of rows and columns.
Now the constructor for the dataframe object,
Pandas.dataframe, the first value is data,
the second is the index that you want to define.
Third is the data type and the fourth is a copy,
the value of the copy.
Now which can be true or false.
Again, the deep copy concept.
Data can be multiple types, as I said,
the array is this constant,
as you have seen in the series one.
Index can be rows and column labels of the dataframe.
It defaults to np.arrange(n).
Similarly, like here we have a default value.
It will, in this case as well, have a default value
depending on the data that you have passed to it.
Dtype of the data type of each column.
This can be essentially a list here, or any sort of
iteratable or an enumerate which contains the data type
of each and every column,
since this dataframe contains head rows in its data.
It doesn't contain more data.
So data type here will be within it and that is what
this will contain.
Let's create a dataframe.
We'll create a list.
10, 20, 30, 40 and we'll create a data.
Pandas.DateFrame and to this we'll pass the list.
Then we'll print the table.
You see the index was created automatically.
Now the column was given a name by itself.
Notice that the column was given a name by itself.
You did not provide.
So zero, if you're wondering what the zero is,
this is the name of this column.
This is the name of this column where the values are.
Now why did it pick zero?
Because so we had not given anything to it by default.
When we don't give it anything by default
it will automatically assume something and it has its own
rules in which it will give a name to the column.
Also call it the label.
So label and column name are two things that I will
be using interchangeable.
As you go down the next few slides you will see
how we can change the label.
If you're using a dictionary, if you change the data to this
it's a list of dictionaries where we have A as one,
B as two, we have A as 23, we have B as 41.
Then we have A again as 45 and B again as 47.
Now if you look at this, A and B, where did this come from?
It came from these keys.
The way again, don't be confused about
when will the label be worked.
Whenever, so whenever it is considering a dictionary,
even when earlier we saw the dictionary right,
what it does is that the right side of the dictionary
is always going to be the value.
That is going to be the value in your cells.
If you consider this to be sort of an Excel sheet,
these are the values, one, two, 23, 41, 45, 47.
These are the row numbers on the left, zero, one, two.
These are the column names.
Neither column name is not explicitly provided.
It will automatically try to figure out hey,
what can be the column name.
Now A is a column name for these three cases.
B is the column name for the other ones.
Please notice that this is how it will basically infer
if you're not providing it explicitly.
What happens if, let's suppose, one of the dictionaries
has column that has not been defined for the other one?
We have something called as nan.
Nan is sort of a null equivalent.
Equivalent would be equal, but it is similar to how
we have a sort of not applicable or not present as a value
for these other rows.
It goes from zero, which is the first element right,
and says okay, A, B have a value but C doesn't have a value
so it will put nan.
Then it will go to the first one.
Hey, okay A and B have a value but C doesn't have a value
so it will put nan.
Similarly for the third one.
A and B has a value, C has a value, so it will put at us 48.
So please notice how it works when nan is a,
sort of a null, it's not a number, it's stored in areas
where no data is provided.
Now let's suppose you wanted to give it an index.
Till now what it was doing was that it was giving a zero,
one, two, based on the position in the array.
So it was looking at the array.
This is the zero, this is the first row,
this is the second row.
Let's suppose we don't want it like that.
We want to label it in a different way.
We will give it an index.
The index will definitely be a list of items.
We want to be very explicit and say this is row one,
this is row two, and we want to give it a row three as well.
So we have row one, row two, and row three.
Why would you do this?
Because it makes your code very readable.
I mean, it just makes your dataset very readable.
That is why you might want to do it.
Okay, let's suppose for a second these are test results
for three people.
One's name is Jim, another one is Dwight, third one is Pam.
These are the test results for somebody.
Now you're seeing how readable this is as compared to
how it would have been otherwise.
You can sort it like this, you can store it in a dictionary
key value pair as well,
but then this is just way more readable.
If you look at the console right,
when you're going through this and you're looking at this,
it is arranged like an Excel sheet.
That is where it comes in a lot because when it comes
to data analysis, a lot of times it's just looking
at the data.
Now we have Python and it does store the data really well
in dictionaries and lists but then viewing it is a pain
unless and until you arrange it
in a very particular fashion.
But this does that for you where it loads the data
in a certain format and it makes your life way more easier.
Okay, so there is another way to create the dataframe
using Pandas.series.
You can convert a dictionary or series into a dataframe
and there's a trick to it.
This is another, so let's suppose
I'm creating a series of 40, 45, and 60.
Index is maths, chemistry, physics.
I create a series two, Pandas.series
and this is 70, 72, and 74.
Imagine a student's maths data.
Again, maths, chemistry, physics.
Now what we can do here is that
we can do the dataframe object.
We can say Jim has series one and Dwight has series two.
We can print the table.
Right, so now we have a dataframe
for marks of Dwight and Jim.
Dwight and Jim are making the columns here.
The indexes are being created by maths,
chemistry and physics.
The values are being populated from over there,
where series two belongs to Dwight for maths, chemistry
and physics and series one belongs to Jim.
So let's look at dataframe addition and deletion
of columns in the dataframe.
A new column can be added to dataframe when the data
is passed as a series.
This is something that we've already seen, right,
where the new column can be added.
In the sense that let's suppose I add C++ over here
and this has a value of 90.
Naturally for Jim, the value will be Nan,
where it is not a number.
Does not give it, it doesn't exist.
Like you saw earlier.
Now, let's suppose you saw this but you want to add
a new column now, third student.
So how can you do that?
Now adding it, creating a series every time is cumbersome.
So let's add it directly.
Let's suppose we have to add the data for Pam
and this contains marks like 90, 91, 92, three
and the index is maths, chemistry, physics, C++.
Let's add even one more column for her for English.
Let's run this and let's see the result.
Okay, so what happened?
Now, English was not present for the earlier ones
when the dataset was created.
It will be ignored here.
The other ones are considered because they were present
at that particular time.
Had we had this like and we were creating it earlier,
then we would have had a fifth column of English,
but if you are adding it like this,
let's see again what we get.
Then you're not getting the English column.
Column like this will be added only when you're adding it
for all three.
So we will come to that as well in terms of how
to add a column, but this is at least you have learned
how to add a row.
A column, so your column is added but the row over here
of English has not been added because of this one.
Now you can also delete a particular column from a table.
So let's suppose I want to add Pam,
but I want to delete Jim.
That's it.
So that's how you delete it by using the del keyword.
Now delete keyword doesn't return anything.
But if you do something to get the data as well as remove it
you have to use something called as pop.
Instead of printing table, let me print Jim series.
This prints the series that basically is for Jim.
This is 40, 45, 60 coming for Jim.
Delete and of course, even in the table,
now Jim doesn't have a record because that has been popped.
So again, to item Pam.
Slight difference between delete and pop.
Delete doesn't return the original element
that is being deleted but pop will return the element.
So pop is like popping, you pop something out,
you take it out.
Delete is that you just delete it.
So if you have a, let's suppose if you have a box of candies
and you pop one out, you're taking it and you're removing
it from the set.
But if you are just deleting it, you're just destroying it,
which means that you're not taking it away
or you're not consuming it.
Let's look at addition and deletion of rows.
So where we saw recently, that you know we could not
add the English row when we added the column later.
So let's see how we can add a new row all together.
Now, first and foremost, you can select the data rows
by passing the .loc function.
Let's use that.
Print table, let's look at the mark maths for everybody.
Sorry, I did not use the loc function.
So you need to use the loc function and you get the results
for Dwight, Jim and Pam.
Now notice that how it has automatically given
in to you in a readable form.
It has not just returned 17, 19, 40 and 92.
It has just given you with the labels as well.
It will give you the result of the labels intact
so that you can make sense out of it
with all the column names.
Now, in case you want to use a mathematical row reference,
where you want to use that I want row number two
instead of the label of the row.
Then you need to use iloc, which is integer location,
which will like this.
So we can try it out for multiple.
Name is given, straight even look.
So it gives you this as well.
So again, these are not simple,
these are not just simple things.
You can actually access the name,
there are ways to access it.
You can check what row name is being accessed iloc.
Now, let's talk about appending to a row.
So to append to a row, what we need to do is,
let's suppose we have and you want to append to it.
So we need to define a new dataframe.
Now the new dataframe needs to have the values.
So let's suppose we need to have the value of
30, 60 and 95.
The columns are Jim, Dwight.
Let's run this.
We have run into an error.
It says shape of paths values one comma three.
This is implied three comma three.
So now you need to sort of visualize the data that you have.
Visualize the data in the sense that there are
three columns and you want to add a new row for the columns.
So how would you do this?
Let's look at what is happening over here in the example
that is given in the slide.
Now, 11 and 13 are being added to two and three.
Okay, now the columns are two and three and they are
being mapped to this one.
That's all we had to do.
We had to kind of pass it as an array and give it the values
within the array instead of giving it without the array.
So in the sense that otherwise, if you look at this right,
it's an array of arrays over here.
It's an array of arrays, where for the first row,
you can add multiple rows in this time
because the dataframe will always contain multiple rows.
Can always contain multiple rows.
So you wanted to add one single row.
Then I was doing it like this, right.
It was implying that I want to add three rows,
but create a dataframe of zero, but I wanted to create
a dataframe of only one row.
I wanted to create a dataframe of only one row
which would contain the marks of the English column.
For doing that, I will use this, where I will append
this dataframe to the row.
Do this, right, and then I can change the table zero index.
I can later do a label change.
Now drop function is used to drop rows
where labels are provided.
So you need to provide the labels.
So let's suppose I want to drop the table now,
drop the row now.
I'll say drop zero.
Then I'll print table.
Okay maybe I need to use the numeric one.
Okay, now let's suppose instead I want to drop the C++ row.
The C++ is dropped.
Similarly I can drop chemistry or I can drop maths.
Let's look at importing and exporting data using Pandas.
Now, similar to how we were using it with NumPy,
you can have a dataset and you can read the CSV
by just giving it the path to the site.
So let's try this out.
Let's just create a new CSV file for us.
I'll just comment all of this out.
Okay, it will read this 10, 20, 40, 50,
just present over here just like we loaded it using NumPy.
But now it's a Pandas dataframe.
The data type is of a Pandas dataframe.
Any table that you have at your end, it can be returned
to a CSV file as well.
In the sense let's do this and...
So if you notice that dataset has been exported to the file.
The table that we had over here has been exported.
Again, makes it easy to you know,
kind of have a prepared dataset.
So it happens often that you know, let's suppose you have
a large dataset and you might process it at once
over one at machine and you might give it to somebody else.
It is very handy that way, where you can just import
it and export it in a very compatible way
to different people and it can just give your data
to different people, let them handle the same.
Now similarly, you can read Excel sheets as well.
You just need to provide it the path and you can write
to Excel sheets as well in case needed.
So it's just like super simple, just like how CSV works.
In case you ever face issues with this,
of course my first recommendation would be convert
it into CSV.
All institutes can be converted into a CSV
so you can do that as well.
Here is a use case or a problem sheet.
So let's suppose Bob wants to track the demographics
of his country by age for every year.
So what he wants to see is that how does a country's
population, the demographic, vary according to age
for every year.
So how many young people are there in every year.
3% of the people are below 30.
40% are above 30.
So he wants to create that sort of a distinction.
Now, he collects all of his data about the country's
population from the census, from the internet,
from numerous websites and how he's trying to look at it.
But if he was to look at it just on an Excel sheet
or even a Pandas dataframe, how easy would it be
for him to digest that information in a single quote.
More than that, if he had to share that with somebody,
would it be easy for that person to look at the same thing
and infer the same thing?
In the sense that he could have it arranged and everything
but it's not immediate.
The solution that we have for ourselves is a Matplot library
provided by Python which helps us in plotting the data.
Now there's a saying that a picture paints a thousand words.
Even when you have very complicated data,
once you plot it on a graph it becomes dramatically easier
for you to understand what might be happening
compared to looking at the raw data in it.
So a short introduction to data visualization in matplotlib.
Matplotlib is a Python library that is specifically
designed for the development of graphs, charts,
et cetera, in order to provide data visualization.
Matplotlib is inspired from the MATLAB software
or the MATLAB programming language
and reproduces many of its features.
This doesn't mean that you need to know about MATLAB
or anything, we will still be writing Python.
But yeah, it is good to know that it is based
on something very strong.
One of those languages that are used by CAD machines
and scholars and universities.
A lot, provides a lot of mathematical operations
straight out of the box and that way a lot of Python,
so a lot of things that you can do in Python
with respect to data science and machine learning
can also be done in matplotlib.
It's just that Python tends to be more favored
because ease of writing as compared to something
like matplotlib.
So installing matplotlib, it's pretty similar to the
previous commands that you have been using for a solution.
This is how the screen should look like.
The command is pip install matplotlib.
All small, no difference from the way
it is written over here.
Pip install matplotlib.
Pip space install space matplotlib.
Let's look at plotting in matplotlib.
So the first plot that we're trying to plot is this one.
Let's head over to the ID.
The first you will import matplotlib.pyplot as plt.
This is very very important that you import it as an alias
because otherwise, this entire name can be quite big.
This is again, a user that you will see as multiple places.
This is similar to when we imported NumPy as an np
and when we imported Pandas as p.
Very simple plot, we're going to plot one, two,
three, four on the plot and you're going to show the plot.
So this first command, this just sort of passes the value.
It is not that the plot has not been created,
but it's just that you haven't asked to show it
or save it.
Unless and until you tell Python explicitly that hey,
I want to look at the plot it won't do anything.
This is also important because sometimes you might want
to generate plots and save them or sometimes
you may be actively looking at plot right there and then.
So let's take this for a spin.
You will notice that something else will appear,
it will not appear over here, whatever machine you're on
it will appear in the tab.
Like in Windows it will appear at the bottom right next
to the start menu.
So this is something that matplotlib generated for us.
It has several features like zoom and everything.
That's really nice, but here is our plot.
It starts at one and goes up to the four.
So we were plotting the y-axis over here.
If you look at this, we plotted the y-axis.
The one, two, three, four.
If I add a value of 10 let's see if axis goes to 10.
So you see that this value has gone to 10.
This is not that value at all.
Now, we haven't given the x-axis, but matplotlib has sort
of figured it out by itself.
It has taken certain default value.
You don't need to worry about it because normally we
wouldn't be plotting it like this.
Normally we would be providing our own x-axis,
but otherwise it's like zero item of one.
The first item is at two.
So it's index spaced.
The way the x-axis has been marked that has been the index
of these items.
So 10 to the 5.0 because it's (mumbles).
List of vertical coordinates are the plotted.
You're importing matplotlib as plt in
the and displays the plot.
The x-axis values are implicit from zero to n minus one
where and is the length of the list.
This is five minus one, four, zero to four.
Now, we can also specify the x values by ourselves.
The way we can do this is let's suppose we collect this
into an array.
So if I am movable to it.
For the sake of demonstration I'm going to create
some mock data for the x-axis.
The second array that I will get with,
and this is list comprehension.
So this I'll print it for you here as well first
so that just in case you forgot what this does.
This will plot the y values and this is going to square
all the y values.
One square, two square, three square,
four square, 10 square
and will collect all of them as x-axis.
Now let's see what we get.
X-axis is two, four, six, eight, 10.
Now you have the y-axis over here.
First set of values was the x-axis and then the y-axis.
Now you can also use NumPy to generate the list of items
and you can use that as well over here.
What we'll do is that we'll use NumPy dot arrange
and will plot that plot.
You want to plot x and we want to square the other value.
Let's see what we get.
Let's compare it to what you wanted to print.
X is going to be one, two, three, four, five.
This is going to be squared up from zero to 25,
which is a maximum square.
Let's print arrange, let's print this just so that
we can confirm that.
So arrange will take you from zero to five in steps of 0.1.
So that's continuous value.
It is plotting all of these values
that are being provided over here.
Let's go to the next one.
Now, often we would require multiline plots.
That is multiple lines on the same plot.
This is again, very commonly used, and couldn't be an easier
way to do this.
There couldn't be an easier way to do this.
All you need to do is you need to call the pot
again and again.
Think of calling this method plot over here
as a way of putting a line on the graph.
That's it.
So every time you're calling plot because putting a line
on the given graph.
So what we'll do is that we'll do is equal to range of five.
It's a simple Python range of five and we'll just one
it would be power of 10, but you can see the difference.
These are done power 10 is that you can spot a difference
between them (voice cuts out).
So naturally this line over here, this is called 10.
This is for 10 and this is for the star two.
X-axis that they have is the same one.
X-axis is a shared from zero to five.
You would call that as range minus one.
Now in case you wanted to plot multiple lines using
the same plot function.
If you think this is a little cumbersome,
you can simply copy paste the same thing over here,
as a single one, and it will run.
We got a similar graph.
Let's try another thing.
Let's try it with three lines.
This time we're gonna do i**100, so that you can really
really see the difference.
It seems like we have gone out of range here.
We can plot multiple figures using a single plot
function as well.
Let's try with this.
One thing that you might have just seen that earlier
when I was showing you the plots, even though we are used
three of them, the lines are not clear enough.
So the reason for that was also because they had sort of
merged together.
The values had increased so much now,
but you have like comparative values
so we can see the difference.
Let's see if we can remove the x to make it work or not
or do we need to give the row every single time.
Seems like that doesn't work.
It's giving a different sort of a plot.
We need to give it the same x-axis.
That needs to be followed everywhere.
That's when you will get the three lines.
So in case you want to show a grid in the background,
this is just a visual thing, so you can just set this
parameter to two, prp.grid.
So we have a grid in the background.
Another thing that we can do is that we can limit the axis.
Rather than limiting the axis, let's suppose it's
a very large graph and you just want to see a certain
portion of it.
Do a demonstration.
The first is for the x-axis.
Suppose only 5, 10.
I want to see the portion of the graph which is between
five and 10 and for the y-axis I want to see
something between which is 15 to 20.
This is a coordinate space.
X can be between five to 10 and y can be between 15 to 20.
Let's run it.
So we got nothing because probably the graph is not
even in this space.
Like there's no point that any of the graphs
have in this space so let's change this and see
if we get something.
Still nothing.
Let's change it to zero comma four, zero comma three,
and zero comma three.
We should definitely get something over here.
We get the three lines.
Let's suppose we start watching it from this point.
So think that we are going to cut a slice of this graph
along this axis and this axis,
where we just want to see the lines.
We don't want to see a single line.
We can set x as one two three
and same goes for the other one.
If you just set it to one two three, one two three.
It's just launching.
You can do another one, we can do one two, one.
Anything beyond 1.5 and we are lost.
Let's do 1.25 and two, 1.25 three
and over here let's to 2.0.
You see how we sliced that portion?
This is the limiting the axes part is really for seeing
a particular section of the graph.
That's how you need to think about using it.
Now you can also use alternate xlim and ylim
in case you think that remembering this is a problem.
So we can do the same thing by using xlim,
which would be 1 point same thing, 1.25, three
and this one it would be two to three.
We'll pretty much get the same results, same kind of graph.
Of course it's up to you which one you want to go for.
I would personally prefer this one because at least
it's explicit.
You know what you're trying to look for
in term of what you're setting.
Let's talk about adding labels to a plot.
Very simply, we can just, okay so let's remove this first.
That's it.
X-axis and y-axis, as exactly as I wrote it in Python.
In case you want to add a title.
Learning matplotlib.
Let's talk about adding a legend to our particular graph.
So (voice cuts out)
the plot statement in case you have multiple plot.
So don't forget to give the call
to the legend function as well.
You need to set the label over here like this
and then you need to call the legend function as well.
Let's run this.
See, now it's looking much more professional,
much neater as typically we see things to be.
Let's look at saving plots.
Very simply, nothing much to do.
We can just give the name of the file.
It will create the file by itself.
So this is created now and this is saved on file.
Savefig, F-I-G, save function.
Next, let's talk about plot type.
Matplotlib provides alto of types of plot formats
for visualizing information.
This includes scatter plot, histograms,
bar graphs, and pie charts and many more apart from these
as well, but these are the primary ones that are typically
used or asked from developers.
Let's have a look at these one by one.
Histogram, so histogram displays the information
of a variable over a range of frequencies or values.
It shows the distribution of a variable
or a range of frequencies or values.
Please notice what it is for and sort of try to understand
that it's not a bar graph.
It displays the distribution of a variable.
Now, what we're going to do here is that we are going
to use this function called random.randn from NumPy
and we haven't covered it earlier, so let me show you
the documentation of this function.
So NumPy.random.randn, so what is does is that
it generates an array of shape d0, d1.
So whatever shapes you give here, that is the shape
that will generate, filled with random float sample
from a univariate normal Gaussian distribution
of mean zero and variance one.
Please know that the input variables over here
are not the range or anything of that sort
that we are looking for.
This is a dimensionality of the matrix.
Let me show you by example.
Two lows in c columns.
Show us in columns that values that are according
to Gaussian distribution,
which is a mathematical expression.
So it's calling some mathematical equation,
that's what you need to understand in case you're not
familiar with Gaussian distribution or you haven't studied
it or like you don't remember it right now.
But it's just over here that we define
the rows and the columns.
We are not doing anything else.
Now you want to print 400 rows and 10 columns.
This is ... where basically Python is obfuscated that
but we can do it with 40 and 10 as well I think.
Or let's try with two.
Yeah, this is much cleaner for two rows and two columns.
So let's plot a histogram with one of these values.
We are going to take y is equal to this
and then we will do .hist, histogram short for histogram.
We're going to give it the data value.
Then we say plot.show.
Let's run this.
So we get a histogram.
Now don't confuse it with a bar graph because it looks
like that but it isn't around that idea.
In case my sample set was larger, which I will just make it,
it will look a lot more like the graph that you just saw
on the slide.
It's showing the distribution.
This is a little heavy, it's stuttering.
Now the histogram groups values into non overlapping
categories called bins.
This is just sort of a categorization that the histogram
can do for you and to get it done we need to call
a second parameter called the bin parameter.
It's just the bin value.
So let's take it for a run.
So the reason why I changed the parameters is because
it sort of becomes heavy with 100 100.
In the sense that with that, the size, the plot size
is pretty huge and so it would take a lot of time
to kind of, that is why the rocket figure
was jumping up and down.
Just keep in mind it's not that it's stuck,
it's just that it is taking a lot more time
and the time would be lesser if you were on a more
powerful laptop or a computer, which has less power
or enough RAM.
So then it would be still, but it's just that the computer
is doing all the calculations to plot this.
It's not exactly trivial in terms of time.
So the output takes a little bit of a time
and there's a little wait time.
Next is the bar chart.
To plot the bar chart we need to provide it two arrays.
The first array is the midpoint of the load face
of every bar.
This basically means this.
So this is one, two, and three, where the midpoint
of the bar graph should be.
This is the height of the successive bar graphs.
So let's try this out.
We'll do set a bar graph as the arbitrary values
and we're going to give them values of 45, 85, 89.
Let's take this for a run.
So the gap came because I gave it a very random hard value.
This is not how we are usually conditioned in school
to kind of make these sort of bar graphs.
It's just that over here I wanted to prove a point that
you can pass any sort of random values and it will take
the midpoint over here.
If you look at the midpoint of this and you look at
the values that are coming over here on the right.
So around 1.5 is when you will see
that's exactly in the middle.
Yeah, this the exact middle where my cursor is right now.
Then 2.6 for this particular bar graph
and the 4.8 for this bar graph.
So you can also plot a dictionary using a bar chart.
I have the code ready for this.
Now what we are doing here is that we are iterating over
a dictionary using the enumerate object.
Let's first have a look at what enumerate does.
Basically it gives an iterator object.
It will just give me the i and the keys.
This is saying that the zero is the first
and this is the second one.
So what enumerate kind of does is that it will put
it into tuples of this sort.
It's returning that and we are taking the for the zero value
but on the zero point with the key value of this.
We are going to plot the bar graph.
The zero here, coming from this i that is being set,
and the one coming from this i and then the two.
Now you can also do it like this.
i+1, doesn't matter, it basically the involvement
of dictionary over here is just doing an enumeration
over here.
As the demonstration that even if you have a dictionary
you can sort of manipulate Python.
So all of this, it's not that it's a matplotlib concept
really, it's more of programming that you can cover it
like this as well in case you have a dictionary
and you want to plot it on a bar graph.
Yet another way to do it is to show xticks.
Xticks, what they do is that they will take the dictionary,
they will take the range that you set over here,
which is zero, one and two, and they will replace them
with the labels that you want.
Plt.xticks, if you look at this, zero, one, two.
I could have gone for the dictionary keys as well,
but I wanted to show you a different demonstration.
A-tick, b-tick, not tick.
Let's just do it as b-bar.
Now if I want to do a similar thing using just
the dictionary, I can do it in this way.
It's because what is d.keys, it's acb.
So earlier I had done it but passing it manually,
passing it and rewriting it by hand.
But you can of course,
use the values provided by d.keys as well.
This is for placing the labels on the x-axis.
Let's look at the pie chart.
Pie chart is relatively very simple to make.
All we need to do is now we need to give the size
initially over here, the size of the pie chart.
We need to give the dimension.
So size of the plot in inches, which is let's set it
as three, three then what we can do is
that we can set the pie plot values
so it will calculate the percentages by itself.
Please note it that you don't have to pass it
the percentages, it will do a sum and calculate
the percentages by itself and you pass the labels.
Should have been labels.
Let's run this.
We got a nice little plot here.
You can similarly increase as many number of labels
as you want and just keep dividing.
So let's suppose I take a very disproportionately sized
value and I say this machine learning.
We get these values on a pie plot.
Next let's look at a scatter plot.
Scatter plots display values of two data sets
visualized as a collection of points.
Now we need to do here is that take this up.
This x is the number of points.
Randn, let's print x and y and see what exactly
this is going to be.
These are two points.
This is one point and this is another point.
These are two values of x and these are two values of y.
When you do a scatter, you're saying that oh,
I want to plot two points.
So there are two points which have been plotted.
Now the values that are generated over here,
these are again Gaussian distribution values.
So think of them as randomly generated values
which are following certain rules.
They're random but they haven't picked randomly
based on certain rules.
Two here is really the number of points that you want.
'Cause this weird of array of sort, like an nd array
of sort of a thousand columns and this one will be
again, one cross thousand.
This is square two do a scatter plot of these.
Let's do the big one.
These are thousand points on this plot over here.
We can do lesser always, we can do 20 and we can 20 again.
So these are 20 numbers.
Let's see if we reduce the number do we get an error?
The numbers (mumbles) what happened?
So x and y must be the same size.
You will get a value error.
Because what happened was that there were x,
there were 10 x coordinates for which it could not
find a match in the y coordinate, in the y section.
So it wouldn't work.
Finally, let's look at styling a plot.
So let's suppose we had a line plot.
Arrange one comma three and then you are doing a plot of y.
What the second parameter is doing is that it is
this finding a color.
Now you may ask which color is y.
Now this is based on a color code provided
by matplotlib in itself.
So C and then we assign plot dot show.
So these are the color code.
B is for blue, c is cyan, g is for green,
k is for black, m is for magenta,
r is for red, w is white, y is for yellow.
So please, please keep this in mind.
This is not just as simply translated always
because black is for k, surprisingly.
You need to refer to the color list.
Name np.find.
You got color lines according to the coloring
that we wanted to give it.
Now sometimes you style your line in a particular fashion.
For that we also have option.
Now you can either do it in a single plot,
as you have seen over here, or you could potentially
do it even in here as well.
So let's try this out on one of the lines first
and see the output.
We made it a dashed line.
Let's see if we can give it a color at the same time or not.
So let's give it m and c is taken.
Let's give it a red color.
This can't happen right.
You can't give it the color and styling at the same time
but you will have to use something else.
But for now, okay if you want to make dotted and a dashed
line and then you want to make a line which is made
out of colon statements.
So this is how you do it.
This is a colon line, this is a dot and dash.
This is just a dash.
Solid line, dashed line, dash and dot and dotted line.
Now, you can also have customized markers and if it's
where the line styling as you've just seen.
Let's try this out for one of them.
Let's see what o does and let's completely remove
the other two for the timing.
Let's try it out on this because the other one had very
equal number of points.
Let's see what happens with d.
D is for diamond, o is for a circle.
This is talking about the points that are being plotted.
What is the line styling for the point?
But that will happen.
The particular point.
So the other one was the entire line was being
styled in a certain way.
This is referring to the control markers,
which is the specific data that you have given.
This is not for any of the...
Let's suppose I was doing this, this is specific
by square values.
Let's try it with the hat on it and you have it with this.
So again, it's just styling.
(pleasant instrumental music)
So introduction to data manipulation.
Using NumPy, Pandas and matplotlib we will analyze,
visualize, manipulate large datasets in order to extract
valuable information and insights from it.
Let's look at the basic functionality of series ndim.
Now, ndim is an attribute available on the dataframe object
from Pandas and it returns a number of dimensions
of the data structure.
So let's try this out.
Here I have a dataframe, which is Pandas series dataset.
Naturally the dimension is one because if we print dfo here,
right, it is a singular one.
This is the index column and this is the actual data column.
Now of course we can do it in this way as well
where we pass it additionally.
Then if you look at the dimension,
the dimension has changed right,
because now there are two columns.
So this is basically the number of columns.
Ndim basically tells you the number of columns
at a present near data stream.
What axes does is that it returns a list of the axes
of the row labels.
Let's have a look at this, what do we get here?
Okay, we got zero, step, 50 and one.
The index is an A and B and the dtype is object.
Let's suppose we change this to one to one zero one
or let's suppose a random number as one zero three
and let's leave it at 51.
Let's see what we get.
Right, so if you look at the range index it to zero.
It starts from zero then it goes to tops one zero two
and is in step one.
So just to kind of reiterate over this,
it is referring to this particular column.
Zero, it starts from zero, goes up till the maximum values.
Don't look at this one, zero.
This is the maximum value.
This is the stop.
So it is a total cumulative value.
These are 102 items, right.
The range index and a step of one.
So in case you want to create a range, you will say
zero to 102, where the last number is not included.
So 102 will not be included, but from zero,
if you go in steps of one, we will reach 101.
So we have to stop at the number which is lesser than 102,
which is this one, correct.
So this is basically again, sort of giving an idea
about the data, part of data analysis to kind of know
your data in depth really well, even before you start
doing anything with it.
Next are values.
As the name implies, pretty simple actually.
So let me remove the df string and run this.
What do we get?
So we get sort of an array of two values,
the two columns, one and 51, two and 52, three.
Each individual row is a collection of arrays.
Not exactly arrays.
You've noticed that these are more like a series.
If you had just one and not two, let's see what we get.
Then we get each individual one.
Let's look at the head command.
Very very important, you will be using this quite a bit
in real world scenarios.
The head, what is does, is that it gives you,
so the first five rows by default.
Just that much, because typically your data will have
lots and lots or like millions and millions of rows,
hundred thousands of rows, and you maybe don't want
to print all of it because it takes too much time.
It doesn't make sense printing all of it.
You just want to look at the first five rows.
Or if you want to look at the first 50 rows.
Or let's suppose you want to look at the first 10 rows.
So you could pass it a value like 10 over here.
Without value, it will take a default value of five.
If you pass it something it will take a default value of 10.
Now, similarly you have something called a tail.
So head and tail.
Tail will give you a value from the back,
the last five values.
You can of course, specify and I want to see
the last 20 values.
Please notice that these are more for
viewing the data yourself.
Yes, you can of course put it another variable
and everything.
So not for use essentially for slicing or dicing.
So I don't prefer doing that.
Well, so slicing and dicing being that if you want to
take out the last six rows,
I might write in a different way, the way that discussed
earlier of slicing a dataframe.
But these are more for quickly accessing parts of the data
which are at the start or at the bottom
and going on from there.
Okay, now let's look at some very,
very useful functionality.
Now we have something called the sum,
a variable on a dataframe.
For that let's set up the dataframe properly.
Let's suppose we have a dataframe with r values
and we will change this to np.arrange and we want to go
from one to a hundred in steps of two
so that we have all values, because it will start with one
and then one, three, five, seven.
Then we have even values.
Similarly we have np.arrange in steps of zero
to a hundred to two.
Let's just first print the dataframe that we will get.
We have got even values to 98 to less than 100.
We've got r values less than 99.
Let's see what do we have.
Something very simple that we have is we can just call
.sum on it and let's see what it does.
Brilliant, right.
So otherwise you would have to write an if else condition
to do this, to check yourself, which can get tricky
as you have complicated data.
But .sum sums up the data for individual attributes
and gives it back to you, but even in forward.
It automatically just looks at it and you know,
it just creates that for you, right.
Now, the std function is for standard deviation.
So if you want to observe the standard deviation
in your data you just call std on it,
dataframe.std and you will get the standard deviation.
Of course it's going to be similar for these,
because these are similar kinds of numbers, similar ranges
and it depends on the ranges of the sample set
rather than the value of the number in itself.
Because standard deviation is basically counts
the deviation from the mean of the numbers.
Because these numbers are very homogeneously spread
from each other, you would expect the standard deviation
to be similar.
Okay, next we are going to cover iterating through
a dataframe.
So let's suppose you want to go row by row
or column by column and that is what we will
cover over here.
So let's first create a dataframe as given on the slide.
Let me remove this.
So we're going to create a dataframe which is going to have
random from five comma four and then columns
are going to be, we have four columns over here.
We're going to add col-1, col-2, col-3, col-4.
Now, kind of like how we learned about iterating
over dictionaries, we can do the same over here.
We can just print the key.
Let's see what .iteritems does.
So you might remember iteritems.
This is even used in case of dictionaries, okay.
This is a variable, this is the same method available
for dictionaries as well, even the dataframe object
has this method available on it.
So let's see what it returns for dataframe
and if it firsts ask for our expectations or not.
Just try to think about what it did for dictionaries.
For dictionaries it returned the keys and values over here.
The key and the value part of the dictionary.
Let's see what it does for the dataframe.
Okay, so for dataframe, what it does is that it picks
it up column by column.
That is the key.
Then it gives you the value.
So in fact, let me separate this out because it might not
be exactly clear which is which.
So this is the key and the this is the value, right.
Let's run this.
The key's column one and this is the value that is being
printed for you.
Let's just look at the type of value that we have a
deeper understanding of what it is returning.
So it's a Pandas series datatype.
It is a Pandas series datatype that it is returning
once you're iterating over it.
So as the slide says, the key value pair iterated over
consists of the column label as the key in the series
object of column values as the value.
Let's look at iterrows, so over here we have another one
called iterrows.
This one is not available on dictionaries
because it doesn't fit with it.
This is a dataframe that's a specific method
that is available to you.
What it does is that it iterates over the row label
as the key and the series object of the row values
as the value.
Till now what you were doing is that we were getting
all the values for column one when we were doing iteritems.
For iterrows we are able to iterate row by row
like you would do in a database.
Like if you were to look at an Excel sheet right,
you would go, what is there under the first row,
what is there under second row.
For column one the value is this,
column two the value is this, column three the value is this
and so on and so forth.
That is the same progression.
Let's see if we can use value.ndim over here.
Right, so let me mute this out.
Let me remove this.
So if you look at the value dimension,
it is one for the series because it returns
a series dataset but label with the column length, right.
So itertuples return an iterator yielding a named tuple
for each row.
So in case you want to named tuple for each row,
that, so this is again something you would use
when the use case presents itself,
but maybe you want it in a different format.
So this is basically a different format than getting
it in the Pandas data series dataset.
But it is still iterating over the rows.
What it does is that it doesn't return two values,
it returns an individual row.
But if we look at the type of the row, let's do both.
Okay, so it's first and foremost, it's a Pandas class
that is being returned.
It looks like a tuple, but it is a Pandas data class.
The second thing over here is that you get it
in a named sort of way.
You know which is which.
So you know which is which.
For index four, you have all these values.
Index three you have all these values.
Index two you have all these values.
It is a named tuple that is given to us.
Now let's look at some more operations
in the Pandas dataset.
Now this is very, very important, please make sure
you are paying attention.
This is the groupby operation.
There is a lot of ways that you can group values by.
Let's try this out.
I think I have the data set up somewhere over here.
Okay, let me quickly set it up.
So if you have a data frame, we have a dictionary first
actually, you don't even have a dataframe,
you have a dictionary, which contains some data
related to the World Cup.
It contains a name of the team.
So we have West Indies, West Indies again,
India, Australia,
then there is Pakistan, then we have Sri Lanka again,
then Australia again and we have Australia again.
Then we have India and then we have Australia again.
Then we have the rand column, which is values 7, 7, 7, 1,
6, 4 and then finally, we have the year.
Maybe some of you have been able to figure it out.
Some of you are already familiar with this dataset,
the real dataset, but some of you are maybe trying
to figure out what it is.
It's actually the winners of the Cricket World Cup
in different years.
That is what this dataset is about.
Who won in which year.
We will simply create a dataframe out of this.
Let's see what we get.
Okay, I missed a comma.
Let's run this.
Okay, so from 1975 to 2015, we have a list of teams.
We have the rank that they initially had
and the fact that they won.
So this is the initial sort of rank before they started.
The year and the team which won the World Cup
in that particular year.
This team won the World Cup, West Indies won in 1975.
Sri Lanka won in 1996 and these are the ranks.
Now, of course this data would usually be,
like if you were to kind of deal with it,
you may think about hey, let's save this in a database.
That's fair enough, or you might be getting it from
an external source, it might be streamed to you
in the sense that it's at the end of API, like a Json API.
Or you're reading it from a file.
But let's assume that you have this data in your code now.
Now, of course there are things for those of you
who are familiar with sequel.
There are things such as grouping everything in sequel
but then it tends to be limited in what it can do.
Plus, with the flexibility of Python and the ability
to easily you know, write conditions and everything,
dataframe having a groupby function,
which allows you have columns.
It's immensely powerful.
It makes life much, much easier.
You just get your dataset into Python.
You don't have to worry about the syntax or sequel.
We can write all sorts of conditions.
You can view your data in different ways,
which sequel might or might now allow for
or which it might allow for but it will make it
very cumbersome.
Naturally, you don't want to be messing around
with the database and variables and everything
and writing your first conditions.
So you can write the conditions, even in sequel,
but then strongly not recommended.
So let's look at what this will do.
If we go ahead and do a df.groupby and you pass it
the value of team and then you as for the group.
Okay, so it has grouped the different teams for us.
The value that is available over here,
so it's a dictionary, by the way.
The value that is available over here are the indexes
where you would find it.
So West Indies is at zero and one.
India is at the occurrence of two and nine.
Australia is at the occurrence of three, six,
seven, eight and 10.
Sri Lanka is at the occurrence of five, which is correct
if you look at this.
Sri Lanka is at zero on two, three, four, five.
That's how it appears even in the dataframe.
So nothing but just the group data present.
Now you can iterate by groups, you can pick up each
group value, look at the number of occurrences
that it has had and you know, make decisions accordingly
and proceed from there.
Now let's suppose you want to group by a multiple columns.
This is that you wanted to group the team by team
and the rank,
where you're creating pairs of the team name and the rank
that the team had.
When India was ranked two, there are two occurrences.
When Australia is ranked one, there are five occurrences.
But let's look at something that is occurring again.
So Australia, as you see rank one, Pakistan ranks six
has a fourth occurrence.
West Indies in rank seven is on zero and one
and Sri Lanka on rank four is on five.
So let's actually create some data that you can
sort of see a duplicate.
I'm just gonna manipulate the rank here.
I'm going to change the rank of Australia to two.
This last one, Australia, I'm going to change the rank
to two to see if we can get a new group
for Australia in two.
Immediately if you look at this, so Australia rank two.
Then Australia rank one, group as well.
So this is one group, with Australia being rank one.
This is another group with Australia being rank two.
Unique combinations of a given country or a given team
and its rank that is provided here.
Now to iterate over groups, what you will do is that
you will simply use something called grouped.
So for name and group, then df.groupby and suppose
we want to iterate by team.
Let's see what we will get row by row.
Okay, we have got the name of the group then we have
the group in itself.
Let me in fact, draw up a separator so that we can
see it clearly.
Okay, group is Australia.
So you have the group name as Australia and this is
the dataset related to that group name.
Group name is India and this is the dataset related to that.
So you can iterate over each of these now.
Let's look at what is the type of the group
that we know what data, what object we will be dealing with.
We will be dealing with a Pandas dataframe.
In fact, have to do something in group.sum,
let's suppose just for the sake of it I can do that.
Doesn't give me insight or I can do group rank.
Then only the rank column is printed.
Or I can group, so I'm basically on row nine
and I'm dealing with a Pandas dataframe.
I'm not dealing with anything else.
So years get displayed.
Now let's suppose you want to get a single group.
I mean, you don't want to iterate over it,
but you just want to get a single group now.
So let's suppose we create a new one called dfgroup
and we go by df.groupby theme and then we want to get
values for India.
The method is get_group and we say hey,
we want the India group.
Naturally, if we have the wrong key, it will throw an error.
It will say that hey, the group doesn't exist.
You can try it with Australia as well.
Next is aggregations.
So aggregated function is a single aggregated value
for each group.
You might want to do a sum of certain things
or you might want to do some sort of aggregation,
an average or a standard deviation.
All of these are aggregating functions which give you
a single reduced value of a particular column
or a set of values.
So let's suppose we need to obtain the group of all numbers
labeled all in the dataframe.
This is one we are aggregating it.
We have aggregated all the groups in the single value
for each group.
This is the group of all odd numbers.
Next is concatenation.
Okay, let's suppose we have a list of of World Cup winners
and we then we have a list of World Cup chokers.
Chokers are basically the teams here that
were supposed to win that were doing really well
in the tournament, but they kind of fluttered
at the last moment and they couldn't complete it.
They couldn't like really win, but they were the favorite
to starts with but they fluttered at the last
when they could have really made a difference
and they could have really won.
So points would be 95, 764 and 656.
Let's add the points here as well.
We did not have them earlier.
874 and 753, 855.
So let's create two dataframes.
We can simply concatenate, we can call pd.concat.
D of one comma d of two.
Let's look at concatenation now.
So concatenation is a process of combining two or more
data structures.
One thing you need to keep in mind while concatenating
two dataframes is that the number of columns
in each of the dataframe should be similar.
So let's try this out.
Let's suppose we have a dataframe with three columns,
E, A, and B.
E has values K0 to K3.
A has values A0 to A3.
B has values B0 to B3.
Similarly year old have a dataframe with D column,
column C and column D.
What you're going to do is you're going to try
and merge the two together.
Let's see what happens.
If you see, we have concatenated this dataframe
to this dataframe.
Now why are the values nan for A0 to A3?
Because they key C doesn't exist.
The key D doesn't exist.
Similarly for values where C and D columns are present,
A and B don't exist so they're appearing as nan.
I hope this is clear to all of you.
We can also use something called append function,
which would append the data
instead of concatenating the data.
I'm call this function, please not that you have to call it
in the dataframe object, not through Pandas.concat.
This is for Pandas.concat, right, when you're concatenating.
If you want to append a dataframe to another one,
so you're going to say left.append right.
Let's see the result.
This basically determines, let's suppose we try to append
a right to left.
Let's see if there is any difference in output.
Yes, there is a slight difference in output.
So when you are appending left to right, it takes the left
dataframe first and appends the right one to it.
This portion is right being appended to left.
This one, right having left appended to it.
This is right and this is left.
Let's see what the axis column would do.
So we type in print then given the axis column,
axis values.
Okay, this doesn't tell us much.
Let's give it a different value and see what the output is.
There's no access name two.
So let's try to look at the axis for this.
Right, so if you concatenate on the axis for zero
you get this sort of an output.
The left array is coming first and the right arrays
and then the other one.
If you have axis of one however,
so axis list terms for the dimensionality.
Axis of zero, so this is basically a 2D array, right.
When it comes to the dimensions, this is a 2D matrix,
left and right.
When you're giving axis as zero what is happening is
that it is concatenating along this vertical.
When you are concatenating across,
this is the default value as well by the way.
When you're giving an axis of one,
it is concatenating across the columns.
So try to imagine an array or a matrix or a two dimensional
space, axis zero is saying that hey, go along the x-axis
and axis one is saying go along the y-axis.
So in a way that in this one, the first one acts as zero.
It is just adding of rows, one by one after the other,
but when axis is one, then it is combining the values.
So A0, B0, K0, so just combining along this particular key.
Let's talk about merging and joining.
So merging is a Pandas operation that merges two
particular dataframes.
So yes, we just did merging as well, but they can be
a different way that it can merge two datas.
So merging along a particular column.
Let's suppose we had to merge on wold championship year
and World Cups played year.
Or better than that, let's merge on the team,
which is common.
So when you call merge you basically remove the duplicate
column and instead give the final result.
So let me show how we can do that.
Let's suppose I'm trying to merge
left to right and I'm going to suggest that hey,
you merge on the value of key.
Let's see what I get.
Okay, my bad this doesn't need to be an array.
This should be sequential values.
Interesting, so we got a result like we did here in fact,
with the axis is equal to one.
Nearly same because when just concatenating it was keeping
both the key columns, but when it merged,
it merged and gave you a single column that it has merged on
because this is a simple concatenation operation.
It will just put them together.
This is joining on a certain column, where you're saying
that hey, for every value of Key0 there is a value
of C0, D0, A0 and B0.
You're merging these two together and the common path
gets is not duplicated.
Now, there are multiple kinds of way which you can merge
or join two dataframes.
One of them is a left join.
Now, think about it again.
This is what makes it very very useful,
where as you say through Pandas,
you can just load a dataset into memory.
Now it is providing almost DB level querying
in the application itself.
Very powerful, very easy to write, reduces like,
in case you are working as an analyst specially
and you are dealing with a database, reduces the amount
of time you have to spend writing database queries
because they can tend to be very long and confusing
as compared to simple Python code.
You know again, because of features like these
that you see it being used a lot in data science,
machine learning and so and so forth.
So let's see what the left kind of join will do to it.
Let's suppose I say merge left.
Okay, we get a sort of output.
Let's do a comparison.
It says merges is on the left object.
The left object is the first object over here.
The left most object, so I should actually say.
The left most object.
So it's right and left.
So don't be confused with the names over here.
Okay, let me just actually rename them.
I'll call them A and I can call this one B.
So first I was doing A comma B, let me now do B comma A
and see if the result changes.
Okay, the result did change.
So left basically says A it means as it is, merge B to it.
So merge B to is, which is this one.
When you just interchange, it gives you the opposite.
When you merge from B to it, so B comes first
and then A comes.
This can be important if you are, you know you have
unequal values or columns or something of that sort.
So yeah, this can be a particular way in which
you arrange your data.
This can matter, especially when you're trying
to analyze the data.
Let's look at the right join then.
What does the right join do?
Let's take this and merge.
Pretty sure that some of you might have already guessed
what it might do.
Right, so if you look at the right join immediately
after this one, okay.
Let's do the right join on this one.
Let's see what we get.
This one is pretty clear, right, what is the difference
between left join and right join.
Let's look at the outer join.
In this particular case, a full union of the two columns
as mirrored on both objects and let's see how we can get
an outer join.
This is, as it says, it merges two objects based
on a full union of the two columns of both objects.
So what it does is that it does a full union,
make sure that both of them are present.
Right, similar output, but this time it's an outer join.
So the reason for similar outputs
is because how the data is.
If the data was a little different then you would have
of course, naturally have a different kind of output.
So let's say the data was kind of like this,
that there was a fourth value, a fifth value.
Let's see what we get on an outer join.
So on an outer join we get both of them.
If we were to do a right join in this case,
what would we get?
So right and outer are similar.
Let's see if right and left are similar.
Okay, so left is not similar.
What left does it says, I'll keep all the values
of the first dataframe.
For B, if there is any extra row, which is in this case,
K4, C4 and D4, I will ignore it.
I get this output, K4, C4, and D4.
Can't ignore there is no fourth row
because it was doing a left joint.
It was giving preference to the A dataframe.
When it is a right join, it will give a preference
to the B dataframe.
When it is an outer join it is not that it is giving
preference to either.
It is basically going to join both of them respective of
different values.
So let's suppose it has a K6 okay,
this was an A6 and this was a B6.
Now you might think that okay, now both of them
are five rows, but the real question is the key matching.
Basically trying to match on the key.
Now when it is an outer join, it is ignoring the fact that
okay, this key, K6 is not present in the right one.
K4 is not present in the left.
So it is giving you this and this row.
However, when it comes to merging on right,
it is only considering the key, all the keys from the right
dataframe, which is B dataframe.
When you're doing the merge on left, it is considering
all the keys from the left dataframe
and making sure there's a row for each one of them.
So if you look at this one right,
this is K0, K1, K2, K3, K6.
The entire A dataframe will definitely come.
If B doesn't have values for a certain key,
they will come as nan.
That's left.
The first one gets preference.
Right is the second one get preference.
Outer is nobody gets preference, both of them are included.
I hope this is clear enough to all of you.
Let's look at the final join, which is the inner join.
What does an inner join do?
Inner join basically is like the simpler thing that you
were doing earlier.
It will just merge on the keys that are common to both.
So an inner join or a normal join, so the default value,
when you do this, in here is the default value.
So line 24 and 25 are same essentially.
24 25 are the same line.
Let's look at a particular use case of Pandas.
Let's suppose you're provided a large dataset
of country wise statistics to extract knowledge from.
Country, landscape, population, GDP,
rural population, a lot of other things.
Let's look at this dataset here.
You have country, you have land area,
population, GDP, rural.
Rural is, I think, a kind of score that the person
is of rural population, whether it has internet or not
or what percentage has internet, what is the birth rate,
death rate, elderly population, life expectancy,
female labor and unemployment.
We have it for about 216 countries, which pretty much
represents the world as of today.
Let's explore the dataset.
First and foremost I'll import Pandas
and then I'll import the dataset.
I need to give it a path of the CSV.
All.
First thing I need to look at is the number of rows
and columns, which is the shape of the dataset.
So it has 215 rows and 12 columns, okay.
First is the header itself, so it has 215 countries
to line 216 and it has 12 columns,
one, two, three, four,
five, six, seven, eight,
nine, ten and twelve, which is correct.
Let's check the kind of data types available
in this dataset.
Okay, so the country is an object in terms of the Pandas
dataframe and everything else is a float64 datatype,
which is also correct, because all of these are numbers.
Let's look at the first five rows.
One thing that it does that whenever the number of columns
are huge, or like they're of a sort of more than
let's suppose these six values, are seven over here,
seven columns it can accommodate.
Otherwise, it will go the next line.
But you can basically see that it is referencing it
as row zero.
So this is just a representation.
It's not that it is broken down or anything,
it is just displaying it by sort of breaking it
to the next row.
It doesn't have the space, even though if you look
at my computer it has the space.
But then, there's just like a limit in terms of
to what line it will print to.
It will just break it up and these columns are,
so this is essentially five rows.
So don't think of these as 10 rows, this is zero
Afghanistan's land area, population, GDP, rural,
internet birth rate and then Afghanistan's death rate,
elderly population, life expectancy,
female labor and unemployment value.
So you can check the first (mumbling).
Let's pick a random number 16, run it
and there we have it.
Okay, let us look at a statistical summary.
You might remember that we did this earlier.
Very intuitive function if you look at,
I'm just calling dataset.describe.
This is like one of those things that I definitely
like and really admire about Pandas
or even Python in general.
It's very descriptive.
You can just sort of read it and understand it
there and then.
So dataset.describe.
Now, for land area, count, mean, standard, minimum,
25%, 50%, 75% max values.
So we can immediately see that okay, the GDP mean is 14333.
Rural or internet all over the world it's 43%.
Female labor all over the world is at an average 58%.
Life expectancy has an average of 70 years.
Elderly population is that of 7.9% all over the world.
Death rate is 8%.
Birth rate is 21% and unemployment overall the mean is 9%.
The count tells us that hey, these are the number
of countries you have caught the value for.
Then you have the standard deviation,
you have the minimum values.
The minimum value for both is 10%, death rate is 3%,
elderly population is 5%, life expectancy is nine,
nine years old, that's pretty low.
Female labor is 17%.
So you know, it tells you something about your dataset.
So maybe you know, after you look at this and you're like,
nine, which are these countries which are sort of driving
this value down to nine in terms of life expectancy.
The minimum value, I'm sorry 45, yeah, 45.
Sorry, my bad, it was not nine.
Instead of the standard deviation, that's the deviation
from the mean.
That's minus 61 plus 61 from 70.
Doing this generally gives you clues as to where
you might want to look to a data scientist,
oh this is interesting.
Which countries have the best internet?
So which country has 96% internet, I'd like to know that.
Right, 96% of land mass is covered in internet.
Is this provided internet, that's great, right.
So that's where usually a lot of data science starts from.
You look at your data, you make some sense out of it,
and then you decide okay, I'm going to use this algorithm
or this analysis, or this analysis process to kind of
distill more insights and opportunities out of it.
Let's look at extracting insights from data.
Can we write a program to find the list of all countries
within a size greater than 2,000 or thousand
square kilometer?
Let's first select only the specified columns
on the dataset.
So to do this, what you would do is that you would
select a make a variable selected data.
I hope you guys remember loc.
We are going to go columnwise.
We will reference the column.
The first is that you want all the rows and but only
the columns country and land area.
Next we're going to write a simple for loop
to iterate over this.
For I in selected_data.itertuple.
Pretty simple, right?
We just wanted to find countries with a size bigger
than 2,000 square kilometer.
We just write the simple for loop and here we go.
See, so easy, the name of a country is greater than.
India, China, Canada, Brazil, Australia, Algeria.
These are land masses.
You can also do with greater than thousand.
The number will definitely be higher.
We can do less than 500, let's see if there are
any countries of that size.
Pretty small, so a lot of countries actually,
a lot of countries of that size.
There's two less than hundred.
I'm pretty sure we might find some.
Okay, a lot of countries less than that size.
Let's see if there's a correlation between the GDP
per capita of countries and their birth rates,
which is to say that hey, if the GDP of a country is high
or low, how does it relate to the birth rate
in that country?
What we're going to do is that we're going to go ahead
and plot it in matplotlib and going to find a,
we're going to plot our findings on matplotlib
so that we can visualize the data.
So if you look at this particular problem statement,
the reason why you would think about using matplotlib
is that it might not be easy to see a correlation
through numbers, through just numbers,
as compared to it being represented on a graph.
So we're going to call plt.figure and we're going
to create a figure instance in memory.
Gonna set the size, that's 50 comma 50.
Next, we're going to read the dataset.
We have already done that.
Now, we're going to use this one and we're going to just
like GDP birthrate.
Okay, so we're going to plot a scatter plot.
What we did here was that we created a NumPy array
using the column, just to see these data.
Then we want to create a scatter plot.
Of course you want to call x limit so that we can just
see between zero to 2,000 portion of it.
Let's go ahead and do that, go ahead and run it
and we have the rocket back with us.
Yeah, so I mean, it gives you a certain level of insights.
These values are, of course, provide coincidentally,
here a little bit didn't map this particular axis,
but you guys should mark the axis.
Now 2,000, that would typically be a value of the GDP.
Birthrate is going to be (mumbles).
So birthrate, not such a strong corelation.
In fact, countries with a higher GDP, if you look at
the countries with the high GDP, a lot of them have values
less than 25.
But if you look at the density over here,
countries with lower GDP, they tend to have,
so look at these values, for example.
GDP is the lowest, but it's one of the highest birthrates.
You look at this one, this is the top right,
nearly 50% birthrate, huge, massive birthrate for
a country to have and look at the GDP, it's tiny.
So the corelation sort of exists, but not in the favor
of the countries with the high GDP countries.
The lower GDP probably indicates inefficiencies,
indicate developing economy, you know kind of struggling
with population crisis, population boom.
Definitely India and China and third world countries,
those would contain.
So yeah, kind of gives you an insight
into what is happening.
Really cool and you can do much more.
You can do similar analysis on so many other things
in this dataset and draw insights from it.
Please go ahead and look at the columns, keep changing them
one by one one by one, keep just running particular thing
and I'm sure you will get a lot of insights out of the data.
So this graph's a little different.
It maybe just that the data that I'm using
and the one that is used for this one are different.
Doesn't matter though, you can just use the data that you
find on the course material or you can use your own.
Let's compare the GDPs of the 10 richest countries
of the world.
This time we will only select the country and the GDP.
We already have the plot.
Let's sort the data.
So it's sorted the data by GDP.
I'm just going to remove this and we're going to print
sorted_data.iloc and we're going to print the first 10 ones.
We are also going to plot a pie graph.
We have the graph, this looks prettier.
Let's select the sorted data from different countries.
The top 10 GDP countries in the world.
(pleasant instrumental music)
Let's look at developing web maps using Folium
and Pandas modules.
What do we mean by developing web map?
Let's look at the problem statement.
Let's suppose there is a person called John and he works
for a disaster management organization.
He's a researcher, current searching on volcanoes in USA
and population in different countries.
So he wants to basically map the population
and the volcanoes in a given region.
What is the benefit?
He would be able to know how many people can be shifted
to other countries or would need to be shifted
to other countries if there's a volcano eruption
happening in a given region.
So he wants to look at the fact that hey,
there's a volcano which goes off in a certain region
and there's a large population
or there's a small population.
Where all could it immediately go to in case there are
there's a national disaster of that sort?
For this he wants to design a map which will give him
an idea about volcanoes and the population
of different countries in a single map.
The single map bit is very important here
and we will look into it in terms of how to get that.
Let's look at the logic to implement this.
The first thing that we need to do is we need to use Folium.
Folium is a Python library that can help us
in dealing with and generating maps.
This is not Google maps that we are talking about
or Apple maps.
We're talking about generating our own map
using the library called Folium.
Next is that we would want to create markers to show
specific locations in a map.
Markers meaning these icons, these pins.
Next we are going to import the Pandas library
for data manipulation.
We have already used this in the previous classes.
Through it you're going to read a data file containing
the list of volcanoes and then use Folium to mark
volcanic location on the map.
Finally, we are going to import something called
world.json and this file is going to allow us to
mark the country of a population and together using
the first part and the second part, where we have created
the markers over here and using this,
we are going to generate something that's going
to look like this, which is showing the population
of the entire world.
Let's look at the steps to design a web map.
First and foremost you need to install Folium.
The command is very simple, pip install folium.
In case you have any issues or doubts with this,
please reach out to support.
Step one, so Folium makes it easy for you to visualize data
that is being manipulated in Python
on a very interactive leaflet map.
Leaflet map being just a small map, not a navigable map
like Google maps.
It's a leaflet map like a big static sort of map
that the physical maps that we're used to,
just the digital representation of it.
Just the digital representation being Google maps
is very interactive.
There's a lot more features, a lot more interaction,
a lot more detail.
But Folium is not going to have that level of detail.
So for example, Google map shows you locations nearby,
interest areas, hospitals or train stations
or metro stations but Folium doesn't do that.
It's just a leaflet map.
In the sense that the level of detailing is not as much.
It's not meant for navigation.
The next is that Folium results are interactive.
So there is some level of interactivity,
but not as interactive as Google map.
But definitely, there are certain features that you will see
soon enough which make it interactive.
Leaflet is the leading open source java script library
for making mobile friendly interactive map.
Now, don't be confused by this, it's just that not that
we are going to learn java script.
It's just that Folium makes use of the leaflet
java script library, which can create these maps.
Once we look at this, you will realize that it will
in turn maybe using some (mumbles) as java script
to generate these maps.
For that, the Python library Folium basically relies on
leaflet, which is a java script.
Let's look at the commands used.
So over here, first and foremost, you import Folium.
Then you have Folium.mapclass.
These are the values that the constructor takes.
It takes a location of arrays.
This is the latitude, longitude pair.
Next, it takes the zoom level when you're starting the map.
So when you're launching the map you need to define
the zoom level.
The values can be anywhere between one to 200 or 300.
I will show you the different results
for the different value.
Then the title of the map.
So this is the title of the map that we will be generating.
Now, for the sake of demonstration, let us try this out.
So I've given it a random latitude longitude.
I've given it a zoom level.
Let's take this for a run.
So now we have Folium.html present for this.
Let's open it in the browser.
We are not seeing anything here but if you see that I keep
zooming out, we do see something.
It's just that the zoom levels were set very high.
So let's set it to 200 and see if we get a better result.
The process is finished, the file would have been refreshed.
Now if you look for the first temp still too deep.
So let's just set it to a very low value now
and see if that works.
Yes, that works.
Zoom level 10 works.
Let's try a deeper zoom level and run this and reloading.
Nothing.
So it seems that the value that this can take are limited
by those numbers, but zoom start essentially
gives you the level of depth.
Let's try even a lower number and see what it shows us.
Okay, now a wider map.
Let's try 12 and see if 10 is the upper limit or not.
10 is the upper limit, even with an 11 I think we should
get a problem.
Okay, 11 apparently works, but 12 just does not.
So now it's one to 11 sort of values.
So the minute I go above 11, it just goes blank.
There is no read here.
This is again, one of the things.
Like it's not like Google maps.
It doesn't have that level of detailing, but yeah,
you can zoom out and you can view the rest of the map.
Now what about the latitude longitude?
That's where you want to place the center of the map at.
So it starts where the center of the map is that coordinate
over here, that is what the center of the map.
So if were to change this to, let's suppose, 11.
Let's see what would be the new center of the map.
Let's reload it.
The center is this much.
Let's change it to five.
This is going to be a drastic shift.
Okay, now if you see, it's in the ocean.
So we moved downward, square of it.
Let's set it to 40 and see what we get.
Okay, we are somewhere, we have almost reached Kurdistan.
We are quite high up.
We have crossed India and we are now in the Chinese
region of sort.
So this is how you can create a simple map using Folium.
Now, notice that this doesn't have any markers
or anything of that sort on it yet.
Now, the next thing is that you can create something
called a feature group.
Now this is another class present through Folium to you.
What happens is that you create different feature groups
for different kinds of things.
So feature group that you want to create right now
is volcanoes.
What it will do is that it will give you an instance
where you can add items to this group of items.
So we will be creating objects.
We have created a feature group object and we can add
a list or a bunch of items to it, not just one of them.
So let's first create this feature group.
Now what we can do is that we can take a set of coordinates,
like this, and call this command call fg.add_child.
So I'm not going to do this exactly, but I'm going to
show you something similar.
So I'm going to do fg.add_child and I'm going to add
a pop up and I'm going to place it
slightly different from where the center of the map is
but within range.
For this, let's set it up as nine.
Next, what you need to do is that you need to add
the child to the map as well.
This is the map, this is the feature group.
Feature group has children
and then the map also has children.
Don't be confused by this, it works like this.
There's map, it is a feature group and feature group
has children.
So map is the father of feature group, so map.add_child
will take the feature group object.
Let's run this.
Let's reload the map.
Let's see where the marker is, here is a marker.
Now if I click on it, it says hello.
Why is the text hello being displayed over here?
Because that is what you have asked in the pop up variable.
So I can say my home and I can set that color as blue.
If we load the page, we are zoomed in, let's zoom out.
And here, now it's in blue color
and it says what I've told it to.
This way, yes you can make the map interactive.
This is a sort interactive map here.
So yes, of course, you can add multiple coordinates as well,
like it is being done over here.
So if I was to create a generalized version of it,
this is how I would do it.
Now you should see two markers, right.
What we did was that we just created an array of coordinates
and we iterated over it.
So it kept adding multiple children to the feature group.
Coming to step two, Pandas is something that you're already
familiar with, now what we have is that we have
the volcanoes USA dataset.
This will be provided to you in your resources section.
Now, we have the volcanoes dataset.
How can we make use of this?
We just import Pandas, we need the volcanoes dataset
and then we can print certain things from it.
So let's just first run this.
It shows us what's in the file.
There's a volcano, there's a number, there's a name
and there's a location and there is a latitude
and there is a longitude.
The status, elevation type, time frame.
So there are different data points about the volcano
and we can see that for every volcano you have
the longitude and the latitude as well.
So let's print this instead.
Just the latitude and the longitude.
So if you notice over here, what you have done is that
in the dataframe you have just accessed the column
lon and lat we got to know from looking at the dataset.
Let's run this again.
Now the first array is the first series that you see
over here on the top.
This is the longitude and second series that you see
over here is the latitude.
There's also something called an elevation of a volcano,
it's also an important data point,
especially when doing considering disaster management,
that's the objective of this exercise.
Because that impacts the overall scale
of the disaster as well.
This is how high the volcano is.
It can have such a deterministic factors in terms of
what would be the strategy that we use by John,
anybody who's planning for this.
So let's come to using Folium.
Now the first thing that you will do over here is that
we have just repeated the code here.
We have got the list of, you got the separate individual
arrays with longitude, latitude and elevation.
Next, we are going to define a method called color producer.
Now what the color producer is going to do is that
it is going to return a string which contains color
vision the elevations.
So we are seeing that for every volcano which is less
than a thousand elevation, it will return the color green.
For thousand to 3,000 it's going to be orange.
For everything else, which is basically volcanoes
above 3,000 meters, we are going to return a color red.
The reason for this is that we will be marking
different volcanoes differently in these colors
so that we know which volcano is what is the elevation
we just created.
Next, we are going to start the map and we will pass
in these coordinates.
Now, why these coordinates?
It's just that the dataset was of that sort.
So the dataset that we have given you here
is USA volcanoes and this basically latitude longitude
pair is set accordingly.
We give it a zoom level, we set a title
and we get a feature group of volcanoes like we did.
Next, we're going to use a zip function,
lat, lon, elevation to create an iteratable
that we can go over one by one.
So this is lat, lon and the elevation, which will merge
lat lon elevation and allow you to access them like this
in a sequence.
Next, nothing much, again, location.
So we pass it a latitude and longitudinal value
over that value.
Then for the pop up we want to show the elevation.
So this is in meters, so the actual elevation we're going
to convert it into string, add meters to it,
and then for icon, Folium.icon, we are going to pass it
the color producer.
The result of the color producer over here.
So let me just show you what will happen over here.
This it the elevation, right.
This is what is going to come up in the pop up.
So let's take this for a spin, ran the other one.
So you see, how for different elevations we got different
results over here and these elevations,
this is converted into a string here, meter was added,
and then we called color producer on the element red,
orange, so and and so forth.
What we would have produced is map two.
Let's open this one.
So we have all of these mapped.
All of these pointers are volcanoes
latitudes and longitudes.
If we click inside it we get the elevation.
As I said, this is a USA based data.
That's why we set the latitude longitude to that pair.
As per this, we have green, orange and red.
That method is being called here.
So Folium.icon basically is another class present
from Folium which tells you what should the icon be like.
It can take various parameters.
Let's look at what all it can take.
So you can pass it the color, the icon color, the icon type,
like it can take different icon types as well.
Right now it is taking this pointer,
but you can pass it different icons as well.
The angle that you want it to be at.
This is for you to explore.
Really depends on how.
What is the requirement for representation?
How are you supposed to represent it?
You can just you know, tackle it accordingly.
Now we have got this.
So as I was saying that this accepts strings only.
So we are converting it into that value.
Next is that now what you want to do is that you want to
take world.jason and we now need to
map a population feature.
So for that what we do is that we create a new variable,
fgp, which is a feature group named as population.
Then we call add child on fgp.
Now this is a different style.
So here what you did was the we created a loop,
but what another thing that you can do is that for adding
a child, you can pass multiple values as well.
So we could have created multiple values out of this as well
or pass a current kind of object.
Over here, in add child, you're passing Folium.geoJson.
This function is used to show geographical
data and maps from Jsons.
So the data that we're going to pass it is going to come
through io.open.
You need to import io if you're in Python 2.x.
So you do an io.open, which opens this file,
which is world.json.
This is world.json.
It's a very heavy file.
It's just a Json file containing
the world data of population.
I open it in a write mode, but then in coding UTF8 sig
and I read the data.
The data is read into the test variable call as data.
So we are basically geo Json, what it does is that
it will mark the population of different countries
and then we are passing it as style function,
which is written using lambda,
where we're going to give different kind of colors
based on the geography.
So green if this is less than this number.
Orange if it is, and if it is less than this, then red.
Like it is at the high one then you're give it a red color.
Now let's add this to the map as well and run this.
Let's see what we got as a result.
So if you see, different countries are marked
with different colors and we we have the volcanoes as well.
So in a single map, what you have been able to do
is we have been able to map different countries
and we have been able to map the location of the volcano.
(pleasant instrumental music)
Next we have a very interesting case study called
the Titanic data analysis.
As some of you might know that there was a ship
called Titanic, which sailed from Southampton in US
to UK and it basically crashed and sank on its way there.
It hit an iceberg and it sunk to the bottom of the ocean.
It was a big disaster at the time because it was the first
voyage of the ship and it was supposed to be really,
really strongly built and one of the best ships
of that time.
So it was a huge disaster and of course,
there's a movie about it as well.
Many of you might have watched it.
Now what we have is data of the passengers,
those who survived and those who did not,
this particular incident, this tragedy.
It has been compiled over the years and published.
Now what you're going to go is you want to look at
this data and analyze which factors would have contributed
most to the chances of a person's survival
on the ship or not.
So whether the person survived or died,
does it have to do with anything common that they shared?
Did the females or did the children survive
better than the males?
Did the rich passengers survive more
than the poor passengers?
Or if the fare amount, the amount of money you paid
to get on the Titanic played any role?
Maybe people who paid more money got evacuated first
and there weren't anything left.
What about the workers?
The workers survive, what are the chances of your survival
if you were a worker on the ship, not just a passenger.
So all of these are very, very interesting questions
and you will basically go into them one by one now.
Now, here is what our data is going to look like
or it looks like.
You have passenger to the passenger ID.
It's nothing but just a serial or increment ID.
There is no inherent meaning to this data,
it's just row numbers in a class.
Next is survived, whether the person survived or died.
Zero is for did not survive and one is for survived.
Next is the name of the passenger, then the gender,
age, then we have something called sibsp.
Number of siblings or spouses aboard the Titanic.
So for this person, if they have any brothers or sisters
traveling with them, what is the number.
Or their better halves, that they had their spouses
traveling with them.
Next we have parch, which is the number of parents
or children aboard the Titanic.
So if I'm a person, are my parents aboard or are my children
aboard the ship?
That counts.
Next is the ticket number.
Inherently again, sort of meaningless as the passenger ID.
Then there's the fare, amount of money the person paid.
Then there's a cabin number, which is basically just
the particular cabin that they were on in the ship
and finally, we have embarked, Cherbourg,
Queenstown or Southamptom.
Embarkation being where did they get on the ship.
So first and foremost, let's load the dataset
and see what we get from it.
I have this here.
First foremost, we import the libraries, import Pandas,
NumPy, matplotlib, math.
We have the Titanic dataset present over here
in the same folder.
We are going to just do a pd.read CSV.
Then, we want to print the number of passengers
in original data, which is nothing but the length
of the index.
Try to recall, just try to jog your memory about
how you got the length of a dataframe.
So you can just, if you have the number of indexes
in a dataframe, which are the number of rows,
then you have this particular data.
So this is df.index is an array.
You take a length and you convert it into strings
so that it can get appended.
Let's run this.
This is basically going to tell us how many passengers
do we have information for.
We have 891 passengers, as we can look at.
That's a decent size.
We can draw insights from a data this size.
Next is let's look at the head list,
look at the type of data we have.
So as we said, passenger ID survived, Pclass,
sex, age, sibsp, parch,
ticket, fare, cabin and embarked.
So this is just a very cursory analysis.
Let's dive a little deeper.
A lot of times what happens is that when you get data set
from outside world and because it's an imperfect world,
a lot of times you will find that the data
has certain missing values.
So think about it in this way that whoever collected
this data of Titanic passengers, they could be missing
information about so many of them.
The person couldn't be found, or some sketchy details
came in or something else, something was told,
something was not told, something was never known.
There are lots of scenarios like that.
So let's just print the sum.
Let's just have a look at how many
of our values are missing.
We call isnull and then we can call sum method on it.
Let's see the output for this.
Age is missing for a lot of them.
Cabin number is missing for a lot of them.
Well, age could have been a very, very important criteria,
but apparently we don't have that for a lot of people.
So let's see how we'll get by without it.
But cabin number is not as important,
so we are good that way.
What you are going to do, so all this issue is that
we are going to remove passengers which have ages null.
So we are going to use the not null method available
in the Pandas library and we are going to pass it
the column for age from the dataframe, where it is not null.
Now here we are using the word wrangle.
Wrangle is any sort of manipulation or any sort of
dealing with the data.
So in this case, wrangle for us basically means that
we have removed certain values and we are just dealing with
a modified version of the original data.
Now the number is 714.
We removed all the passengers from our consideration
which did not have a age provided to us.
Next, we are going to look at gender.
So number of passengers in age embark wrangle data
where age and embark both missing.
This is the embark wrangle data.
Now, what you want to do is that we want to group data
by gender and then let's first do that.
So for doing that, we are going to define new
variable gender data.
We're going to take this dataframe, embark wrangle,
which doesn't have any missing values for age or embarked.
Now notice we did not do this for cabin,
even though there were a lot of arrays missing,
because inherently it is useless to us.
Now of course, this is where your judgment call comes in.
This is where your brain is used to decide okay,
this is not useful and we can do without thinking about it.
For this, we simply call the groupby function,
which is going to groupby on the gender or sex
and then what you're going to do next is very interesting.
We're going to take out the mean.
The mean of the gender data.
So let's see what we get for this.
Now, what we have done first is that we have calculated
the total survival rate,
where for the survived column is .40.
So 40%, only 40% people survived the disaster.
That is what we calculated because we took the column
survived, which was zeroes and ones and we took
the median value.
There were 41 ones of the hundred were zeroes,
then it would be 40 over 140.
That's what this mean will be to figure out the survival
rate, otherwise it wouldn't be the survival rate.
It's just that because it's ones and zeroes.
That's why you have got the mean survival rate here.
Next what we are going to do is you're going to
get the mean data that we have created by gender.
So we're going to look at the mean values for all
the data points.
For a survival rate or chance of 40%, what was the average
gender, what was the age, what was the pclass,
what are the different values for an average case?
So for an average case, for a gender of female.
Notice first and foremost, how this created two rows
for text based data because it automatically realized
that hey, there is no sense of saying that we took
a mean of male and female.
Where male and female are completely different.
They've been marked as such.
If they were not marked as such, if there was no sex column
then of course there would be only one row.
But because we have identified this as a feature,
this has something which is a contributing factor.
We are saying that gender is going to be a contributing
factor to the survival rate.
We can sort of see that how, because for females passengers,
the survival rate is 75%.
For male it's only 20%.
Immediately we know that females were preferred over men.
Women were preferred over men when it came to survival,
which is usually the case in cases of disaster.
Then we have other insights about the average age
of female passenger that got saved and the average age
of a male passenger.
We have the pclass, which is sort of consistent,
that it's greater than two, but still, a lot of even
female passengers from the lower classes were saved.
We also had the sibsp, which indicates that a lot of
female passengers were traveling with children,
as indicated in parch.
71%, a lot of them were returning with the children,
as compared to the men.
It's at 27%.
So again, women and children.
If this 27% of the men who were traveling with children
that adds to the chances of survival.
The way this adds to the chance of survival are
if I'm a man and I'm traveling with a child
and I don't have a sibling or a spouse and I don't have
a parent, then I get the second preference
because I'm traveling with a child and the child cannot
be left alone.
So for women, the preference order comes with children,
then women, then the next one is men with children
because the child needs to survive.
Whenever the child gets thrown into the lifeboat,
the men and women were solely responsible for that person.
The child also get added, like we can build that
as a hypothesis.
All of these are theories.
This is basically hypothesizing as a data scientist
or as a engineer, but you validate all of these hypotheses
through data.
So I may be wrong about what I just said.
Maybe data will tell us a different story,
but that's really the nature of the game,
where you have to be ready to be wrong about things
and you're to constantly still sort, sort and sort
until and unless you are proven wrong definitively
and then you move on to another theory and another theory.
But being proven wrong doesn't mean that you should
stop there and then.
It means that you should keep exploring.
It at least cuts down one of the options in front of you.
That's through data.
We are playing by eliminating choices.
So few other observation, the women tended to be younger,
travel with another person,
have a greater number of siblings or spouse,
say best be feature and children parents feature
again stood out.
So another interesting thing is that women paid more
for the fare, $47.00 as compared to the men.
Now, because the survival rate is so high, then the pclass
socioeconomic status, which is indicated by the pclass
is also lower because it has a lot of mixed people then.
At this high a number, the pclass average would be
a higher number of a lower number because a lot of people
get mixed.
Let's do further investigation on gender.
So now this tells us the number of female passengers
and the number of male passengers present.
Next what we are going to do is that we are going to
set the columns of df because what you are going to do
is that you're going to set the columns as sex and total
instead of sex and passenger ID.
So let's do that and print total df again.
Now it has changed, the data point hasn't.
It's just that we have changed the column name.
Earlier it was mean data and by gender,
but now you change it to sex and total.
Now next you're going to get the gender list
in another variable and we will delete that column
from total dd.
Let me print total df here and let me print gender list.
Why we deleted, I'll come to that in a minute.
So first is that we printed total df
with the sex column deleted.
This is with the sex column deleted in total DF,
this variable.
Then we have gender data,
which is basically grouped by object.
I mean it's a class object.
We can't do anything with it till now.
Next, we do something called as gender survived dataframe.
Let's print and see what we get over here as a value.
So now what we have is that we have gender list.
Earlier we had a total number of people, total number 259,
453, now we have the survived, 195 and 93.
These are not two different numbers that we have.
Let's delete the sex column from this.
So here we have the gender survived df.
Next we're going to combine the two together.
What we're going to do is that we're going to take
total df, from which we deleted the gender column
and we're going to take gender survived df, where again,
we delete the sex column.
We are going to combine them but why?
What will this give us?
So it will give you the survive and the total by gender.
We just did not need it over here, that's the reason
for skip deleting it from elsewhere.
Next we can go ahead and plot this.
We can plot, on a graph, how the survival rate varies.
The first thing is that we call the .plot method bar graph
that is available to us on the dataframe itself.
We pass the color lime green and dodge blue
to give it a title.
We give it a label, X label, y label.
X-axis y-axis.
Now we are going to plot the gender across the x-axis
and the number of people on the y-axis.
We are going to set the text.
Next we are going to set the survival gender list.
This is the survival gender list in zeroes and ones.
Zero is female, one is male.
The total gender list, again, male and female.
Now we have also defined a small function to create
labels on the plot.
What you will do is we'll pass it the survival gender list
over here, which contains the data for the survived,
according to gender.
We will take this plt object present over here
and we are going to plot the x plus x adjust.
It's just a minor adjustment to plot it.
Y plus y adjust and we're going to give it the color
and the font weight.
So how we have done it till now.
You do this for both survival gender list
and total gender list and then next,
we are going to plot this.
This is the effect of gender on survival.
If you notice, effective gender on survival,
the color, x label is gender.
So basically whatever you pass to it gets printed over here
on the plot.
So as it appears, on average women were more than
three times likely to survive than men
and due to the binary nature of the survived field
data type, where we are not considering injured
or anything else, just saying pure survival.
We assume that there was nothing bad that happened to them
afterwards, which would be considered as caused
by the incident or the accident.
We can confidently say that 75.3% of the women from the data
set survived and only 20.5% of the men survived.
Now we can also draw further insights from whatever
we have done till now.
Here are a few additional questions that may help provide
further explanation for why the women survived
three times more likely than the men.
We can look at what is the effect of age and survival rate.
We can look at what is the effect of company,
whether somebody's traveling with a person or not.
We can then also look at the effect of the socioeconomic
status, which class, how much fare did they pay
on the survival rate of the said person.
Let's first look at effect of age on the survival.
So what we're going to do is you're going to take the
embark wrangled dataframe and we are going to group
by survive and then we're going to take a mean of that.
Then you're going to print it.
So the groupby on survive and then we took a mean.
If you look at age for female, the average age is 30.62.
For men the average age is 28.19.
So here we can see that survivors on average are younger,
are presumably from a higher socioeconomic class.
That is a more luxurious class.
Ticket 1.9 versus 2.5 over here.
Pclass, luxurious class ticket and a high ticket fare,
$51.00 versus $23.00.
Traveled with less sibling and spouses and traveled with
more parents and children.
For survivors, you can say that they're younger,
they are traveling with fewer people, at least when it comes
to siblings and spouses.
They have paid a higher amount but class is not as great
as the other one, but there are certain things that we can
ascertain from this data.
So continuing further on this, to figure the effect of age.
First thing we're going to do is we're going to split
the data into children and adults.
The way we are going to do this is that for children data
we are going to set the criteria, age being less than 18.
We are going to create a filter on the existing dataset
using this as well.
This syntax can be confusing to a lot of people.
If it is confusing in terms of how this is working,
this is where Pandas comes into it.
I'm filtering on this Pandas dataframe.
We're using a condition on the dataframe on itself.
This is a very, very common way
of using the Pandas dataframe,
if you're already not familiar with this.
Next we're going to get the adult data.
Again, the condition is of it being greater than 18.
Let's get a count of the children and a count on the adult.
We take the children data, passenger ID dot count,
and take the adult data, passenger ID dot count.
So the reason why we are taking the passenger ID
is because these are going to be unique.
Passenger IDs are not going to be repeated.
We can expect that.
Now let's check out the survived children count.
We take children data on the column survived we do a sum.
Because survive is one and zero, so is number of times
you count one it's just going to add up to be the number
of survived children.
For survived adult counts it can do something similar.
Next, we are just going to put this simply into lists.
We are going to create a children list of survived
children count and total children counts.
So think of this second one as the total children count.
Similarly for adults, we are going to create a list.
Then finally, total.
So let's stop here and print these three lists.
Children, adult list, and then the total list.
So, just to kind of sum it up, what we are trying to do is
that we are going to get the different counts
for survived versus total children and we are also
just accumulating data.
Where is the intuition of, where am I getting the idea from?
It's just because I want to compare by age.
That is the initial idea we're running with.
We're saying that hey, let's split it into children
and adult and then let's look at the data.
So let's run this.
The data is that 70 children survived out of 139.
For adults, 218 survived out of 573.
The total number of children are 139 and 573.
For the total survival, you can just add two items
on the list, survive list.
Let's create a Pandas dataframe out of this.
We will call it CVSA dataframe.
What are the values in this dataframe?
The children list and the adult list.
First row for children and second row for adult,
coming from this data.
The columns are survived and total.
Let's see what we are going to get here.
So you can see, for survived children we get 70 and 139.
For adults you get 218 and 573.
We kept collecting and kept collecting it
and this is where we ended up using it.
This first element is an array, the second element
is also an array, which is making an entire row.
The index is children here and adult here.
We are setting the index.
This is corresponding to this row.
This is corresponding to this row because index is row-wise.
And then the columns are given as survived and total.
Let's create a simple plot out of this.
We are going to call plot.bar and then you are going to
simply, as we have been doing till now,
set the title, set the variable, set the xticks
and we're going to call create value labels as well.
Function as well and finally, after all of that is done,
we're going to plot it.
Let's run this, let's see the graph.
So we can see for adults, this is what is happening
and for children, you can see a ratio of 70 to 139.
We have just taken the data and drawn it up.
Next, we can create another plot where we take
the survival rate for children and adult.
Now this is children against the total children.
Now we're going to compare children to adult.
We create a new list and we call plot.bar
and we change the table, we change the title and the label.
Let's run this.
This is one plot and we get another plot.
This gender of the survival.
We get another one,
survival rate between children and adult.
This is right, children have survived 50% of the time
and adults have survived 38% of the time.
How did this graph come?
Through this, where we children data dot mean survived
and adult data dot mean.
So we took the mean and the mean will be these values,
.50 and .38.
After that we just plotted it, nothing else.
This is survival rate between children and adults.
So what can we conclude from this?
We can conclude that 50.7% of the children from the data set
survived, while 38.1% of the adults survived,
resulting in children being 1.3 times more likely
to survive than the adults, which seems logical,
given that is how human beings tend to respond
in a tragedy where they save the children first
as compared to the adult.
Let's look at the age distribution of all passengers.
Let's see if that gives us any idea.
For this we're going to draw a simple histogram.
For this, what we are going to do is you're gonna take
the embark wrangled data.
In case you've forgotten what this is, we used it earlier.
Let's see what the embark wrangled data is.
This is the embark wrangled data where we have removed
the null values and we're just going to plot the histogram
of age distribution of all passengers
because we have taken the age column and we have called
.plot.his bins with a range of zero to 100.
Because that is the range we expect from zero to 100
that is the human age.
The human age is not in decimals with us.
It can be represented in decimals, but then you just want to
see how many people are of what age.
Just in case you're not able to recall any of the syntax,
how is this drawing a histogram, what all these are doing,
I strongly recommend please go back,
have a look at the module around matplotlib.
This is a histogram of age.
As you can see there are a lot of passengers in this range
between what seems like 18 in here the age for at least 38.
This is 38 or 35 actually, and this is round about 14.
So from 14 till this number, I'm watching the x-axis
that is appearing over here we can just see in histogram.
Let's look at the distribution of survivors in particular.
This was not just for the ones who survived.
Let's look at the ones who survived.
We call the same one for survival.
We have the survivor data.
You can put the age label.
Let's see what we get.
This is for the survivors.
Now, as you saw earlier of course, it is going to favor
the young way more, where for children it's going to be
quite high.
The survivor rate is quite poor after 40.
Let's now look at the effect of age on survival.
First let's look at survive status.
We are trying to do a describe.
Let's look at what the description tells us.
For survive, we have the mean, standard deviation,
minimum, 25%, 50%, 75% values.
The mean is that pretty much as expected,
but these other survived.
Now what we are going to do is that we're going to take
the age data, because we want to check the effect of age
on survival rate.
We are going to group by age
and then we're going to take the mean.
We'll create a list out of it.
Next we're going to use a scatter plot to plot the
effect of age on survival rate.
This is how the survival rate varies with the age.
Some of these numbers you can see the survival rate
just goes down to zero.
For some of these numbers, the survival rate is almost 100%,
where we can say that if the person is of this age,
out of this age, out of this age.
This is not saying that this is a certainty,
but this is more of what the data is telling us,
that this is true.
That if you were of this age, for example,
if you were of 80 years old and on Titanic,
you would definitely survive.
But this is not pattern, these are just probably
in a chances because this is not the norm.
But otherwise, this is a more realistic version of it.
Now, please notice that we have used a little more
complicated scatter plot here to kind of draw it out.
Because you can see the kind of effect we got.
So what I would recommend is that you look at this yourself.
What is alpha, what is C, what Cmap, edgecolors,
the colorbar, which is appearing on the side,
if I can show it to you again.
So this color bar again, how it is coming up.
Things that is really required in programming
that doesn't get talked about as much as the ability
to read documentation and teach yourself.
Because it's a lifelong process.
Everyday will be challenging.
Everyday there will be something new to learn,
something new to do.
Because of that nature, you need to develop
that independence of sort.
What I would recommend is that you look at this piece
of code, go back.
If you can't figure it out, I would suggest that you reach
out to the support because I can tell you right now
what all of this means, but then (voice garbles).
What I would instead recommend is that you change
certain some of the values.
So one thing is that what if we set ss10.
Can we sort of figure out what changed?
Maybe, I don't know, the dots have gotten a little tinier.
Let's change S to a really low value of one.
Let's see if we can discern what changed.
If you notice, now the dots have almost disappeared.
They're present.
S basically determines the size of the dot.
If you said S to, let's suppose, a hundred,
a bigger value than before.
This is purely experimentation.
I encourage you to do it more and more with whatever
code you are getting from us.
Now, this will help you out quite a bit.
Now let's try it with this one as well.
What is vmax?
Now 34.
If you notice, the number of passengers, this color bar
range has gone up from 13, it was earlier 13,
in case you did not notice through 34.
If it set it as 35, even the 35 number will appear
in the bar.
So notice how there is no 35 right now.
Let me run this.
Now there is a 35.
Let's do a little more data analysis before we leave
this data set.
The first five ages only have two or less passengers
in their age group.
The first give ages only have two or less passengers
in their age group.
With such a small sample size, the survival rate may not
be representative as those with larger sample sizes.
So in this sense, what we are saying that let's suppose
there is an age of 12 years old, but it has only two
or less passengers.
Either one or two passengers which are 12 years old.
Now, if you were looking at the survival rate of 12 or one,
versus 22 where you had 100 passengers
or you had 50 passengers, you don't have enough sample size.
The denominator in that equation of the total number
of people of age 12 and the total number of people
of age 50, it is not large enough for you to sort of
be certain that yes, this is the pattern.
Because one of out two or two out two
is not a big thing to happen.
But 50 out of 50 establishes a trend.
If 50 out of 50 people of age 22 survived
or did not survive, it tells you very clearly that hey,
there is something happening with that age group.
Somebody's by the ones, you know who were 22,
they were considered to be too young or too old
or something else or they were helping out in a certain way
that none of them survived.
Let's do one thing where to get more insights into our data,
let's remove passengers with ages in which there are
five or less passengers in the group.
For doing this, what you will do is that we'll get
the total count age.
This is nothing but the total count of passengers in itself.
Then we are going to get count age gd5.
Gd5 stands for greater than five.
So where we are going to do, we're going to take
the count age, the count age of particular column,
your sorry, dataframe, and we are going to count
for where it is greater than five.
Next, we are going to create a list out of it
and we are going to keep only data where,
so we're going to call the embark wrangled dataframe.
I'm going to use that and we're going to filter that
on this list.
This list is going to contain certain ages.
Let's see what ages it contains.
These are the ages it contains.
Now we are simply telling Python or telling others
that hey, just keep the passengers which have ages
as one, two, three, four, nine,
so on and so forth.
Remove all the other ones.
Let's see what we get.
Now the number of passengers has gone down because we have
removed certain age categories where we did not have
enough representatives.
This is only useful when you are looking at things
from a age perspective.
If you're looking at it from a gender perspective,
probably you would have removed people of a gender
if it was not representative.
If there were only two females on board,
there is no point of looking at it from hey, okay, you know,
this is the survival rate in females and males.
Because there were sizeable people in those sample sizes
and this is something you need to be very careful about.
Sometimes you might get datasets where there is
a classification or there's a labeling
for a certain age group but you can get thrown off.
So let's suppose in the previous scatter plot,
if you saw with certainty that okay,
anybody of age 80 would have survived,
how do we know that this isn't one person?
Because this is the survival rate,
this is not a survival count,
how do you know this is not one person?
One out of one doesn't confirm anything.
What we are saying is that no, these are essentially
would be able to consider the ones where there is
enough sample size.
We take the data that we have and we plot a histogram
of the ages.
These are for ages which have enough sample data.
That is more than five people.
You immediately see that there are a lot of gaps.
These gaps weren't present earlier, but now these gaps
are present because we have removed these.
These groups did not have enough number of passengers.
Anything less than five number of passengers is removed.
All of these are greater than five.
You can even discount for these ones.
You might want to consider everything above from 10.
Depends really on the max number of passengers
in a certain age.
Now let's look at effect of age on survival rate
given that you've cleaned the data,
given that you've removed passengers.
Total of less than 31.
So we are going to draw a scatter plot again.
Going to group data by age.
Going to get the mean.
Going to get the number of passengers
in the age greater than five.
We're going to do a count and then we're going to do
a scatter plot.
This is the outcome.
See, immediately that top row gets disappeared.
The top row is no longer available.
If you notice now, those, that top line,
that survival rate of one is completely disappeared.
There is no age group now which has a survival rate
that one of all of the passengers are that.
Because it seems that we had a flawed input.
So if you went by what we saw here, effect of age
on survival rate was effect of age
on survival rate cleaned it up.
These are two different ones and this is a more valid one
if you ask any data scientist because sample size
wasn't filtered out properly earlier.
So here are some other plots that you can look at.
Effect of socioeconomic status on survival rate.
Then we can also look at the ticket class and fare.
It can be indicators of socioeconomic datas.
You can focus on investigating ticket class as that it's
specifically stated as a proxy for SES
by the data dictionary.
We can focus on the ticket class
is what we are trying to say.
That both can be indicators, but you can look at ticket
class because even if fare might have been lower,
but then those days they were very strict on who traveled
in what class.
You had to be a certain gentry type to be able to travel
in first class.
Now, that way we can not just say economic status
but the socioeconomic status of ticket indicate,
where we can say that more prominent, socially prominent
people were given the preference.
For fare rate and ticket class can be expected more.
That is, a first ticket is likely to cost more
than a third class ticket.
You can also consider that for a third class ticket,
most of then would have been bought.
Nobody would have given them away.
Whereas in a first class ticket scenario,
the first class passenger scenario people would have
bought for other passengers, people would have given them
for free, they might have gotten at a discount
because hey, there's a prestigious politician
and he's traveling on the first voyage of the Titanic.
You know, he might have just paid nothing but like
10 pounds or 10 dollars for it to get on that boat.
This is something that we leave for you.
We suggest you go ahead and do it and you should sort of
see this sort of a plot for survival class
of one, two and three and then the number
of passengers in each.
Then you should do non-survivor class ticket distribution
like this is survival and you see naturally,
they are higher at first, but the disparities aren't
much if you take it turn of survival.
But when you look at the non-survivors, number three,
these are the ones who suffered the most.
Overall, the effect of class on survivor rate,
survival rates are better if you're a first class
65%, the second class 47%
and the third class would be 3%.
There's a reason why we haven't shared the code for this
because it is suggested that you go back and do it
by yourself.
There is enough sample code for you to kind of work
on any of these graphs we are producing any given graph.
(pleasant instrumental music)
Let's talk about the first application, web scraping.
Here's a problem statement, Bob is an intern
and he wants to have all the hyperlinks from a webpage.
All the links that are present on a webpage,
he wants to collect all of those.
Let's suppose it's an information website and he needs
that information for because his manager has asked
him for that or his superior has asked for all
the links or all the information about travel website,
let's suppose that.
Now one way of course, that he can go to the website,
copy the link, open them one by one.
Just dull, like copy pasting back and forth.
Yes, you can do that for a hundred websites, let's suppose,
but you can do it a hundred times.
But then there are couple of problems with that.
First, it will be slow.
It will be prone to error.
It's just like apart from other things,
it's just a very boring thing to do, right,
copy paste and copy pasting all the time.
So here is where Bob thinks that he should be sort of smart
and do efficient work rather than clerical work.
That's why he wants to write a program that will save
all the hyperlinks from a specified webpage
and he wants to write in Python.
Web scraping comes to the rescue.
Web scraping is a commonly known technique where you can
extract large amount of data from website.
You can save it to local file in your computer
or to a database.
Now, what kind of data?
This could be text data, this could be image data,
this could be just collection of links, as Bob says.
But usually it is a bunch of things.
So here is the fun fact, Google or any of the search
engines that you use today, it works on web scraping.
So part of it is that the results
are not generated immediately.
In the sense that Google indexes or crawls and scrapes
data from the entire web, website by website,
following link after the other.
It's like think about it like this, it goes to Wikipedia.org
it copies the text over there and gives it as
Wikipedia.org's text.
Then it finds all the 10, 20, 100, 200 links over there,
follows each one of those and keeps doing this repeatedly
over and over and over again.
Just by doing that, it has managed to scrape or crawl.
So think of it like a spider as shown here,
going from link to link.
It's actually called as crawling, crawling and indexing.
So there a couple of libraries available in Python for this
and these libraries are very advanced.
Advanced in terms of they're really ready to use.
You don't have to do a lot of things to kind of
get going with using these libraries.
The first one of these is called Beautiful Soup.
It's a Python library for pulling data out of html
and xml files.
Now what happens is that if you go to any websites
or let's suppose we go to Edureka.co.
If I do a right click and I can do something called
view page source
this will give me all the html and css present on Edureka.
This is html css that makes this page this.
So let's suppose is we search for something like
Python certification training, we can find it in this text.
Of course there is a lot of other html css, a lot of things
present over here, but you can sort of map it
to whatever you see over here.
The images, the amounts and everything and everything.
The amount is present over here, right, all of these things.
Now, what happens in scraping is that you programmatically
download this entire page.
You get all of this data.
So one thing that I could, you know, right now select
all of it, control C and then go to pycharm
and paste it over here.
So just take, it will take a second, there's a lot of text.
See, so it's all text at the end of the day, right.
A website is nothing but text.
One thing is that I could have done that.
The other is that I download using Python
and then I search for data using Python.
So right now if I searched for data and I really read
line by line line by line, and that will take a lot of time.
I mean, it's better to just copy straight from here then.
So what happens scraping is that where Beautiful Soup
comes in handy is that you give it the html text document,
which is nothing but very similar to a text file
like Python file, and you ask it to search
for certain things in it.
How you ask for it to search for those things
we are going to learn that.
One other thing which is going to help us is Requests
library in Python.
It is used to making http requests.
So the downloading part of the webpage, Requests library
is going to help us out with that.
So let's look at the request library.
What we can do is that we can take a webpage.
This is a sample webpage, it's from the Thomas Cook website.
Let's just open it.
It shows holiday packages in a place called Escaraline,
India and it seems to be loading.
So we're going to take this url over here
and request.get.
So the get method is available on this.
Get method simply downloads the entire html file,
the response of this http request, which is a webpage.
It gives you a variable in return that we are storing
in r and r.content is where you can see the contents
of this webpage.
So let's run this file and see what we get.
Okay, seems to be some issue.
Instead, let's try and see if we can make it
work on Edureka.
Okay, so we got Edureka's website and there is some data
or something available over here for sure.
This is what I just did, right.
Now we have it in a variable.
Let's move further.
Next, what we need to do is that we need to import
Beautiful Soup library.
Now and please note, that you don't import from Beautiful
Soup, you import from the caller BS4.
That's just like the conventional for Beautiful Soup
version four.
Now, what you do is that you call something,
you create it on call soup, you pass it the content,
and you specify that you want to use the html.parser,
where you want to use it as an html.
This is a function called as prettify.
This will print a pretty version of this webpage.
So let's try this out.
Okay, so we now have a some pretty version of the webpage.
What this does for us is that it helps us in sort of
analyzing particular thing, how the webpage is structured.
So we can do that in the browser as well,
but maybe in Python as well sometimes, you know,
you might get a response and you want to just have
a look at it in the console.
We can see the structure in a very clean cut way.
We have already done this where I've shown you how to
view the html code of a webpage.
Now, let's suppose you wanted to get all the links.
Very simply what you'll say is for web link
and soup.findall function.
So we are going to use the find all function
and we're going to find all the tags which are in the a tag.
Next, we are just going to print web link
and see what we get.
As you can see, we have gotten all the a links over here.
All the a links.
What we need to do next is that we need to use
the get method on this.
We just want to get the href attribute.
We don't want to get anything else.
We have all the links over here
that are present in the href.
Now let's create an empty array to store all of these links.
So what we'll do is that we will just call link start
append and we start collecting all of these
and then we can just print.
So it's just we're going to collect all of the
neatly in an array and use it for the processing.
So we got all the links in the array.
This is how you can basically scrap the webpage.
Now of course you can get more information
from the webpage as well.
Let's suppose you wanted to get all the prices for a course.
So what you do is that you sort of analyze the webpage.
You see what kind of data present.
So if you look at this one, right, iconinr.
So anything which has a class of iconinr seems to,
you know it always carries the amount.
So let's confirm this.
Let's search for iconinr in a webpage again and again.
So wherever there is an inconinr there is some sort
of a price being shown.
This could be a way to find the prices for the various
courses and store them.
So if you have ever seen any of those price comparison
websites, this is how they work.
What we need to do is that, let's have a look over here.
Or rather we need to select the after discount class
because these are not the discounted prices.
We want to get the non-discounted price,
like a posted price.
Let's try to see if find all will help us out there.
We want to find after discount and because this is
a css class, so let's see how we can select by class
using Beautiful Soup.
It suggests that you use find all and you pass it
the element type and the class type.
What we did not pass is the element type.
So first is span and then you need to give it
class parameter like this, class_.
You're going to see that it's going to be class
after discount.
Let's see if we get any result out of this.
Okay, so we got something.
Now how to get the text out of this
and how to get the course name.
(pleasant instrumental music)
Let's go to application number two,
data visualization on the browser.
Till now we have used matplotlib to visualize data
in the Python window.
Let's look at how we visualize data in the browser.
So Nate is a data analyst who wants to analyze data
and create a visualization on the browser
for presentation purposes.
Bokeh is a Python interactive visualization library
that basically produces presentations and visualizations
for the web browser.
It is used widely as it allows for building complex
statistical plots quickly using some very,
very simple commands.
So let's see what it is all about.
First, you need to install Bokeh, the other command
is pip install bokeh.
Now, let's plot a very simple plot using Bokeh.
I have something ready for you here.
From Bokeh.plotting import output file show and figure.
Now these are methods available from Bokeh.
You create a dataset like this and basically you create
something called is a step using figure,
where you set the width and the height of the plot.
Then you create a circle using the dataset x and y
and then a size.
This would be the size of the circle.
You set out what file is color chart and then you finally
call them show method.
Let's use this.
Okay, so this is scatter charts.
Show method basically opens it up in the browser,
the default browser in your system.
Let's try to change certain things.
So we had the dataset X, which is still one, two,
three, four, five, and we have the dataset five,
which is still one to 25.
We have passed the dataset X and Y circle,
now let's suppose the size of this was 50.
Let' see how it will change the output.
You can see the size of the ball.
This is the size of the circle.
So this is basically telling it hey, draw a circle
at this X and this Y of size this.
So as many points are present to X and Y
it will plot all of them.
This is the plot width and the height.
Now that we're crossing the height,
let's set it to be higher.
We can make it something like this.
Now you see it's a big giant plot.
All of this is being made in the browser using Python.
Let's suppose we wanted to build a scatter plot instead.
Now for this what you are going to do is that
you are going to download a dataset from this particular
source, it's an xlsx.
It's just some sample data.
You can go back and have a look at it.
We're going to just read it through Excel.
You need to install this library called XLRD
before you can do this.
It's something that you should install in case
this is not running, it will probably throw an error
that XLRD was not found.
It's just pip install XLRD in case you run into an issue.
Next thing is that we take this dataset and we take
the temperature column and we divide it by 10,
just to sort of normalize it.
Next again, we create a plot figure.
We give it a text, color, area, all of these things.
These are just markers, okay.
Pretty self explanatory.
Text color, text font.
Because it's the browser, it allows you for a lot
of flexibility that way.
Then you create the circle.
You set the size of the circle.
So let's run this.
This is the result that you will get and while it may seem
like you can't make too much sense out of it,
just notice that all of these are very individual points
and these are being marked to the size of point one.
Now I could regenerate the graph but I think it would
take a lot of time.
But notice that it's a lot of points.
It's not just like you know, 10, 100 points.
These are probably 20, 30,000 points at least
in the very least being plotted, all across this graph.
At least keep that in mind.
(pleasant instrumental music)
let's move on to the next topic.
So for the third application that you're going to learn
today, we need to first start understanding how
a computer reads an image.
You and I see this image, right here okay.
But we perceive it in a certain way,
we look at it like building and sky and that's how
a human being would react.
But this is skyline for a certain city.
There are lights, there are windows, it's evening time.
All of those things.
But here is sort of how a computer sees it.
The computer sees each and every pixel, each and every point
in the image as a combination of RGB values.
The RGB values contribute to what that pixel's
color is like.
So each individual pixel is basically nothing but
a number for the computer.
It's a set of numbers, all the images a set of numbers,
it's a two dimensional matrix that you have been given
by the computer.
Now the size of the image would be B cross A,
which is the height and the width of the image.
So whenever you hear the word resolution,
like one, zero, to four, cross 768.
So it's one zero to four pixel columns and 768 rows.
In two three, why in two three?
Because R, G, and B.
That is how the size of the image comes out
because all these three data points are present
for every pixel.
The R, G, and B value because that's how a computer
shows an image.
So if you have ever noticed how image looks different
on different monitors.
So the same image can look different on different devices
and screens.
It's because pixel in itself, how does the screen
or the device that you're looking at the image on,
how does it treat the R, G and B values?
What is it able to do with those three numbers?
These are numbers, simple numbers, like 255, 237,
and 108, right, for R and for G and for B.
These are numbers.
How does it really manifest them?
How does it paint that color?
That is where all the sharp and contrast and quality
and all of those things come in.
Because even for the same resolution, what a laptop
might show might be very different from a full size
desktop, right, with a big screen or a better screen.
So this is how the computer basically look at an image.
But coming to OpenCV it's a library.
But it's not just a Python library.
Now you will find OpenCV is something that occurs
in C and C++ and then all the popular programming
languages, Python and Matlab and Java
and Dsharp and Dotnet.
They wrote libraries on top of it, which are using
the original CC++ code.
Why?
Because the OpenDV library is super super powerful.
It has a huge community behind it.
There's a lot of things that are developed under OpenCV,
a lot of image processing that you see,
a lot of things you see on your cameras these days
which recognize face detection and everything,
that is essentially, a lot of that is built
on top of OpenCV.
So it's open computer vision essentially.
It's not just Python, other Java, also for example,
has a wrapper or a library built on top of OpenCV.
So let's look at a few basic operations using OpenCV.
First one is to read an image.
The first thing you need to do is first of all,
you will need to install OpenCV on your machine.
The command is pip install opencv-Python.
So let me just show it to you over here.
Rather, let me open a terminal.
So if you had to openCV-Python.
Now I'm not going to show it to you here because I've
already had it installed.
Just one word of caution, this is a heavy library,
so please make sure that you are in a stable
and good internet connection, otherwise it might fail
while you're trying to install it.
It's one of the heavier libraries.
Just make sure that you're on a stable internet connection
before you try to install it.
Okay, going back to this.
Now, what you do is that you import cv2.
So inside Python is called, not called OpenCV,
it's called CV2 and simply, if you want to read an image,
you pass the name of the image, imread,
followed by the the format that you want to read it in.
You can read it either as gray scale, which is basically
reading it in black and white, ignoring the colors,
or you can read it in a color format.
So let's read the image.
Right, so one is for colored and zero is for gray scale.
Gray scale is something that you will hear often.
Gray scale is nothing but converting an image
into black and white that we often see.
Now that you've read the image, let's print the image
as seen by the computer.
Let me anchor all of these.
Okay, let's run this file.
Okay, so nothing but the sequence of numbers.
As we said, RGB, R, G, and B.
Let's look at the axis.
It has three dimensions.
Let's look at the head.
So we basically get a NumPy array and dimensional array.
In this case it's a two dimensional array
because height and width.
then each of the arrays has a particular value.
So let's see what do we have in zero.
What is at the zero value?
Let's look at the first one, zero comma zero.
It's a two dimensional array, right.
Okay, these are the values right here.
This is one pixel being described.
Similarly we can have multiple pixel values.
Different values of R, G, and B.
Now we have something called image.shape,
which tells us of the dimension of the image.
This is available through NumPy, not OpenCV.
We have just read the image into a NumPy array over here
and image.shape is something that is available to us
in NumPy.
Now we know that it's 10681024 and three.
Three being number of points that each particular
position has.
Now we have something called imshow.
What imshow does is that it will just open up a window
and show the image.
You can give it the title that you want, Penguins image.
Now why is this important?
It's important because often you would need to manipulate
the image and then see what was the outcome
of the manipulation.
Before we do any manipulations there is just,
understand how we can show and hide images.
If I don't do this, if you notice this,
it will run but before I can click on it it will disappear.
That's because we did not something called a cv2.wait key.
What this does is that it wait key basically
holds the upper for us, where we are saying that hey,
so don't consider it zero.
We can basically pass it any key that is available
on the keyboard, any letter or key,
and we can say that hey, when this is pressed
then sort of close the window.
So it waits on our key input.
If we don't give that, if we give the value of zero,
then it will wait indefinitely for any key.
Zero is for any key, not for the zero.
So let's run this and see what we get.
So we got the penguin image, but this has now been
opened through Python.
This is not opened by your regular image viewer,
this is something that has been opened through Python.
Next, what we can do is that we can resize this image.
What option we have available to us is that, okay,
let me remove this first.
We can take that original image over here and you can call
the resize method on it.
The second thing needs to be the shape of the image.
So what we're going to do is that we have the image shape
as 768 right.
We just want to half it.
Let's call those two height and the width
and divide it by two here.
Then let's show the image.
Should not show the resized image.
So as you can see, the image is now resized.
Okay, this is very neat feature.
I was telling you that the computer views it as
a pixel values.
If you notice, the more I zoom in to it, because I opened
it in the Python window and not the regular window.
You would not see this in the regular window.
You can actually see what the pixel values are
as you zoom in to a region.
So for white you can see that almost all of them
are equivalent for white regions.
As we go to something really really white over here,
you can even see that it is showing up here.
So this is really, really white, almost clearly white.
Snow white, right or water.
This may be water, but if you look at this,
it's not, so if you look at black, it's almost
like all values are almost going to zero.
You can zoom in there, yeah.
There is also the write operation available to us
where we can write the image permanently to file.
Where I don't need to show this or this or even this.
I can just resize the image and call this
and let's run this.
We have the output here.
This is the resize image now.
It's half the width.
It's not 1024 anymore.
768 has also been halved to 384.
Next, let's look at face detection using OpenCV in Python.
So here are the steps required to do face detection
using OpenCV.
The first thing that you're going to do is we're going
to use something called the cascade classifier.
Now, the cascade classifier is basically your dataset,
which previous researchers, and you know,
programmers have developed before us,
which kind of helps the computer,
helps OpenCV in identifying a face as a face.
Now of course there are multiple kinds of classifiers
depending on the logic but cascade classified is one
of the most common and popular one.
This is going to be the step one.
This is going to train the computer essentially,
to identify face as a face.
You could also think about this as something related
to machine learning because essentially the computer
is going to learn through the cascade classifier
what a human face looks like.
But we are not going to go into the machine learning
part of it.
For now we are just going to understand that this
is an application of machine learning and that is where
the cascade classifier is coming from.
Step number two is that if you're going to use OpenCV
to read the image, you do a NumPy array.
Using the classifier we are going to determine the range
of values, the pixel values, which are forming the face.
So if you look at this step three, we are drawing
the rectangle that use depicting camera allowed
on Facebook, where it will detect the faces
and it would encircle that.
So the way this is going to happen is that the cascade
classifier and the NumPy array.
You're going to use them to detect which pixels
in a given image are sort of approximately can be
the outline of a human face.
Let's look at the code.
The first thing that I'm going to do I'm going to use
the cascade classifier.
This comes straight out of cascade CV2 in itself.
This is present as a class.
It take an argument, which is the path to an xml file,
which is present over here.
So let me just open it for you in a second.
This is a heavy file so it might not make a lot of sense
to you right now, which is okay.
You can think of this more like training data
by which you're teaching the machine that hey,
based on these kind of numbers, you can sort of classify
a particular set of pixels as a face or not a face.
The internals of how they work have been utilized
inside the library, OpenCV.
Of course you can look into it, you can dive deeper into
how this exactly works, but then you really need
to think about if you want to.
This is how it works in the industry where if something
is working out of the box for us, where you know,
we don't have to know the internals of it
because the way it is working out of the box is working
really well, then we don't dive deeper into it
unless and until it stops working well.
So tomorrow if you want to identify not human faces,
but let's suppose faces of monkeys or birds
or something else, then you might start diving deeper
into hey, how is this exactly working.
But if today we talk about it, hey we want to quickly
build something which will identify human faces,
what these libraries do is that I mean,
we could spend to write a classifier by yourself
for example, would take anywhere between six to 12 months.
Now imagine if you had to do that for writing us something
which is so common these days, right.
That is where we don't read in deep.
It has already been done, we just start using it.
This is now saving the instance returned by this class
in the face cascade object.
Next I'm going to read the file,
which is this particular file, photo.jpg.
To it is the same file as is shown on the slide
of this actress.
Let me see if I can zoom out.
As you can see, this is the image.
Next, I'm going to convert this image into a gray scale
image because that is when a classifier works
somewhat better than with a non gray scale image.
Next, we can find the coordinates of the faces
by calling them detect multi-scale method on face cascade.
The first parameter is going to be of the gray scale image.
Another parameter is the scale factor
and then minimum neighbors.
Then we can just print the faces.
Let's see what value we get by printing the faces.
So if you see, I've gotten certain coordinates.
These are essentially pixels.
These are the four corners that have been given
to us by the detect multi-scale method.
Now, the scale factor is basically decreases the shape
value by 5% until the face is found.
The smaller this value the greater is the accuracy.
So what we are doing is that if you look at this image
over here, we are reducing the value of the shape.
The value of the shape being the rectangular coordinates.
You're decreasing the value of the shape by 5%.
The reason to do this is that it gives us
a greater accuracy.
Otherwise, let's suppose if we were not to do this,
let me just note these values down.
We got these coordinates.
Let's try it out without a scale factor.
Okay, I think we need to go scale factor
of greater than zero.
Let's give it a scale factor of zero one.
Now if you look at this, the accuracy's not that great.
It has given us a range of possible values
because it was only 1% reduction.
So the output is different this time.
The numbers are different.
Now it is not so sure about where the face is.
The scale factor greatly, greatly matters
because it will decrease the value of the shape that it is
trying to detect by the percentage to the percentage
5% of pause is coming from 1.05 until the face is found.
The smaller this value, the greater is the accuracy.
But it will also take a larger amount of time.
Next, now that we have gotten the faces, the array,
let's try to add the rectangular block.
For this, very simple.
We just iterate over what faces returns to us
and we use CV2.rectangle method to draw a rectangle.
The way this works is that you pass in the image,
you tell it that x,y is where you want to start off with
and then x + w and x + x.
Basically, these are the other coordinates of the rectangle
and then you pass it the color that you want
and other things.
So RGB value of the rectangular outline
and the width of the rectangle.
These form the x and the y coordinate, the four points
that you want to draw on.
Finally, we will resize the image a little bit
and we will show the image.
So let's take it for a run.
If you notice, we have drawn a, and if you notice
the coordinates, they would be somewhere matching
the ones that you got.
Let's try again with a increased value of the scale factor.
Let's see what we get.
Right, so this time, if you notice that because it was
so high, there's no horizontal bound.
We just got like okay, this is the vertical bound
where the face might be in.
So the accuracy isn't that great with
a larger percentage of reduction.
If we increase this to 1.20,
let's see what the output will be like.
Okay, we got slightly better results, but now there is
no lower bound.
So of course, these are more like picking parameters.
You can try out different values to get the kind of results
that you want, depending on the image.
Of course, not everybody's going to have the same
kind of image set.
The dataset that you're going to get is going to vary
in terms of the problem statement.
You might just be getting very simple data sets
which might be just you know, the typical passport size
photographs, which are not too difficult to deal with
or you might be getting something a little more complicated
like CCTV footage or something which is posted
on social media.
So there are various things to play around with.
So it's not just that you write these five, six lines
of code and you have written something which is that easy.
I know it looks easy right now, but when you start
dealing with the real world scenario of applying it
in actual application, that is where you will be faced
with certain practical challenges, where the image might
not be that great or it might have certain other things.
It might have noise.
Because this time, if you look at this image,
it's a perfect image.
It's a very good image to do this analysis on.
But it's a very front-facing image.
But what is if the image was turned to the side?
Next, let's talk about how capturing video works
and using the computer web cam.
The way video capture works is that it captures
the images one by one and after the other in sequence.
So because the images are put in sequence
and they're continuous one after the other,
they seem like video.
Now if you might remember the principle of physics
called persistence of image, where an image is created
data faster than 1/16 of a second,
they appear to blink together.
So if you look at this right, actually these are four slides
but if I'm repeating them fast enough,
you will get a sense of as that this particular person
is moving, but they're just static images.
This is, for example, slide one, slide two,
slide three, slide four.
But if I curse through it really really fast,
I can make it seem like a moving image
while everything else remains set.
Keeping that principle in mind, you're going to write
a loop which is going to give us each of the individual
frames of the image that we're looking to capture
and using that, you would be able to do some sort of
image processing, where we will process the video
in some format or the other.
The way a captured video works is that we first start
with importing CV to CV.
Next, we call the video capture method on it.
Now, we can either give it the path to a video file
or use numbers.
The numbers specify which web cam we would be using.
So zero stands for the primary webcam that is present
on your laptop or your desktop.
Typically, you can give it a value of zero
or you can give it the path of any of the video files
that you have on your laptop.
Now, the next line of code here is video.release.
This is very important to remember to kind of release
the camera, because when you're capturing the video,
it's sort of a CPU intensive operation.
It takes up a lot of CPU and a lot of computer's RAM.
So please make sure that when you're writing
these programs, you're releasing the video properly.
Otherwise, if any of the script or any of the code
is running on your laptop or in the computer
or in our server, it might just not get released
and that might become a problem.
It might crash your server or your laptop.
Now, zero is for the built in camera,
that is the primary camera.
If your external camera's attached, you can put one
to use that and if you have multiple external cameras
for some reason, you can keep just adding the number
one after the other.
Let's go ahead and execute the code and see what happens.
I have it written over here.
Actually, okay.
So just,
so I imported CV2 and doing a video capture.
Let me comment this and then run this.
See if it can actually start capturing the video.
So nothing was happened because we hadn't actually you know,
done anything with the capture.
One thing that you would notice if you're doing this
by yourself is that the cam light will turn on
if you're capturing the video.
Then it will turn off for a split second.
Do notice if a longer edition, let's go ahead and add
time release in the time module.
This is just to confirm to yourself that hey,
this Python code is actually able to turn on the camera
on your device.
Now of course you cannot see it because you will see
that the camera light will turn on for three seconds now
instead of just immediate,
quickly turning on and turning off.
Now, let's do a little more interesting.
Let's add a window that shows the video.
So for this, what we're going to do is we are going to
call the video.read method.
We're going to call the video.read method.
What it returns to us is it returns two things.
One is a Boolean data type, which is true if Python
is able to read the video capture.
So in case, let's suppose the camera is faulty
or there's some problem with the file that you're reading,
this will of course return false.
This is a quick check to see whether you have got
the frame correctly or not.
The second one is a frame, which is nothing but an image.
So it is a NumPy array, it represents the first image
that the video captures.
Let's print the check.
Time.sleep for three seconds and then let's call
video.release.
Let's run this.
So you can see that we got a NumPy array.
Now, yeah, it's 000, not too bothered by that,
but so we don't care what the exact values.
It would be to present in the image for sure,
that it is captured.
This is the kind of output that we got, correct.
Next, what we can do is that we can show the frame
that is being captured.
How do we do this?
For that, we can call the imshow method
and we will name the windows capturing video frame
and we'll pass it the frame.
Why can't we pass it the frame,
even if it's an image show method?
Because the frame is nothing but a image.
It's a NumPy array we're presenting an image.
You're also going to add the wait key method
so that now it doesn't disappear all by itself.
It waits for us to kind of release it instead of
it getting shut off by itself.
You also need to call the CV2 dot
to destroy all the windows.
Let's run this.
So the very first frame is going to be blank.
That's just how it typically tends to be.
Let's try and see if we can,
let's see if we get a different result placing this
over here and then going into the sleep mode.
Okay, still black, not to bother,
but this is sort of the very first frame
as the video camera is turning on.
So if you notice over here,
there's a slight whitishness, right.
This is just from the camera getting turned on
but once we complete this entire project,
we will be able to actually see the image
that is being captured.
Now, the next question is how to capture the video
instead of the first image frame of the video
because the first image frame is not very interesting.
It's all just black.
Let's try to capture the entire video.
So for this, we are going to utilize the while loop.
We are going to actually do it each and every frame
in this while loop.
So I have already prepared over here.
In the while loop we're going to check for the true
condition, which is going to make it run indefinite.
So why are we making it run indefinitely?
Because we don't want to say that okay,
the video will be three minutes in length
or five minutes or five hours in length, right.
We don't know that.
So we're just going to start with hey, let's just run this
and we are going to come out of the loop
using something else.
We are going to get the frame over here,
as you would till now.
Next, we are going to cover it in two gray scale.
Then we'll start showing this over here.
That is there because it's the entire project
that we are building but let us understand things
piece by piece as well.
So we're going to add a wait key and then if the key
is equal to, we're going to set it to let's suppose, q.
So if you press the q key you just want to break out of it.
But this is how we are going to break out of this loop.
Outside this while loop, you're gonna go ahead
and do video.release and video.cvtrue.destroy all windows.
So let's try and run this.
Okay, now we are able to capture it because, now this is
inside the while loop.
It's running constantly and if you notice this is my finger
and now it's not blank anymore.
So that was because it was a first frame
and first frame is always going to be black
because we are turning it on and like in that split second
it's just an image.
So it turns out to be black and slowly it just starts
capturing, gathering all the light.
That's just how the camera works, right.
Okay, let's quit it.
So I press the q key and it disappeared
and the process finished.
We are going to try and use this to build a motion detector.
Let's look at the problem statement.
Now the task is that given that you have a video stream
from a webcam.
We have to detect motion on any movement in front of it
and we have to return a graph.
It should contain how long the human or the object
was in front of the camera.
Each bar of the graph should represent the time span
and each gap, like this, should represent when there was
nobody in front of the camera.
Let's look at the logic.
What we are going to do is that we're going to save
the initial image in a frame.
We're going to convert into Gaussian blur image,
which is a more manipulable image, it's just a more
convenient way of doing it.
You're going to take the frame of the object
and convert it in to Gaussian blur and you're going
to calculate the difference between the first
and the second frame.
So we're going to do this repeatedly for each and every
frame that we capture.
So each and every frame we will bringing
each and every image.
We're going to do this over and over and over again.
Now, we are going to define a threshold to remove
the shadows and other noises.
I'll show you how we will do this, so in case, you know,
there is a shadow or there is some other thing
like a flickering light, that shouldn't be counted.
Let's suppose you're capturing a video stream
through a CCTV camera and one of the light gets flickered
or it gets turned on or off.
But that shouldn't basically be counted as a motion.
That can be counted as a motion because we are working
with the fact that any changes in the frame.
So we are saying that hey, this is and image and it changes
in this image should be counted as a motion.
That's the simple logic that we are working with.
Now, if there is any change in that due to light
or you know sunlight or sunset or something of that sort,
like sort of image or change, we need to factor that out.
So that is how we define our threshold.
We're going to define the borders of the object.
We're going to add a rectangular box around the object
and calculate the time and object appears and exits
in the frame.
Another thing that we're going to do is that wherever
there is a new object, you're going to identify
any new object that appears in front of the change
in our frame.
We're going to identify it and we're going to add
a rectangular box around the object and we're going to
calculate the time when the object appears and exits
the frame.
So we have already done the part
where we converted into gray scale.
From gray scale we can convert it into Gaussian blur.
Let's do that.
I'm going to add this line over here.
Now convert it into Gaussian blur.
Next we also need to check for the first frame.
So we're going to define a first frame object
outside is none.
So to make sure that we don't trigger a change.
So if you notice, the first frame was blank, right.
If it is a first frame, we might get a motion detection
just for turning the camera on.
Naturally, we don't want that.
Let's add the condition that if it is a first frame,
let's change it to be this current frame and continue.
So we will do not count the first frame.
First frame of when the camera is turned on
and the motion detection, this is going to ignore it.
This is basically us ignoring, ignoring the first black.
Setting it.
Okay, now let's go through it line by line.
Delta frame, now we're going to calculate the absolute
difference between the first frame.
Check the frame is just going to give us the difference
in the two images.
This gives us the difference in the two images.
This is the first frame and there is the current frame.
Gray is the current frame.
Now the first frame will, of course, keep on changing
from time to time.
We're going to, as you will notice, as you go through
the code, you're gonna change this first frame as well,
as you loop over the entire thing.
Next, we are going to build a threshold, as we mentioned.
Now, what this will do is that it will convert
the difference value that isn't 32 black,
which means that any difference value with less than 30
in this dataframe.
So we're going to pass it the dataframe.
Any difference of less than 30 in intensity, it is going
to convert it to black.
The difference greater than 30 will convert
those pixels to white.
After that, we are going to call the dilate method on it.
Dilate as the literal word meaning goes, it expands.
It will expand on the, it will expand the particular areas
that we have found on the threshold frame.
What the threshold frame does it discounts for the
shadows and the lights flickering and all of those
kind of things by minimizing the effect,
by converting them to black.
What this will do is that if there is a shadow it will
basically convert it into black on a threshold frame.
What the next step, this dilate frame is going to do
is that it's going to zoom into those threshold frame,
which has been refactored, which has been cleaned out
of all the noise and the random signals.
Next we want to find out the different contours
in our image we do it using the cv2.findcontours object.
As you can see, cv2 has a lot of these methods
right out of the box.
You may wonder, how you would do it if you were not
so familiar with the library.
Okay, how is it that I just like, from my end,
I just know all of these methods.
It's not something which happens over night,
it's the same thing.
Like, you Google, you search, you read through the library
documentation, you figure out hey, what does this method
do, what does this method do
and you figure your way around.
Now, I would basically say that it is better that I leave
this part up to you, where you explore it out on your own.
In case you get stuck, please reach out to support.
They will definitely help you out in terms of understanding
these lines individually.
But really, the better way to do this is to read
the documentation in depth, in detail,
and try to understand it from there.
Play around with it and see what you can get.
Next is that for any contours which are less than
you know, in the dimension, you will do the move them out.
So let's do that as well.
When we add a condition, we're going to get multiple
contours because it cannot--
Identify multiple things in a given image, right.
We are going to say that hey, if it is less than 1,000,
we just continue on to the next thing.
We just continue on to the next thing.
Otherwise it will do build a rectangle for ourselves,
where the cv2.rectangle is going to build a rectangle
around the object that it effects.
So contour is a object.
Think about cnts as multiple objects.
Any object which isn't thousand, ignore it.
Otherwise, identify it, place a rectangle around it,
as I showed to you in practice as well.
Next we are going to show the measures that we're capturing.
We're going to show the different frames,
so you can see what the dataframe looks like,
what the threshold frame looks like
and what the original frame looks like.
Let me run this and let's see what we get.
So what will happen is that a bunch of frames
will open up, if you notice.
This is the dataframe.
If you notice it has a Gaussian blur to it,
it is looking a little grainy because it just makes it
a little easier to process data that way.
This is just a grayed version of the original frame.
This is the original color frame,
this is what we are capturing.
This is the grayed out version of it
and this is the delta version of it.
Let's look at the threshold frame.
Yeah, here it is.
So yeah, now this is the most interesting one.
So if you look at the threshold frame right here,
this is where it dilates, where we are converting it
into black or white.
So it is just trying to find objects.
If you notice, this over here is an object.
Now, I move my finger in front of the camera.
This is my finger okay, right now.
Let's look at the original color frame
and this one side by side.
So this is my finger or this is my hand.
Okay, this might be a better way to do it.
If you notice, the threshold frame is counting
with certain values, right.
So as we sort of saw over here, it will take the delta
when the motion is not happening.
So when my hand was static it was looking white,
but when I was moving it, if you notice that is when
it was converting it to black or white.
It really is focused on the difference.
It is not about painting the object,
but it's more about looking at the difference
on the contours and taking it from there.
As always, you have the wait key
and we can destroy the windows.
Now, let's try to calculate the time for which the object
was in front of the camera.
For this what we're going to do is that we're going
to define a dataframe with column start and end
at the very top of a program, so let's do that.
Let's take the dataframe over here,
and next we want to set a status.
Status is going to be zero initially, because we are just
starting recording and there is nothing.
The first data's going to be zero.
When the object is detected, that it when it is,
you know the contour is not, the contour is greater
than thousand in length, in height, in the contour area,
you're going say the status has one.
We are also going to use something called status list.
What is a status list going to do?
Status list is initially going to contain,
it's just going to be area of two items, none and none.
We are going to append to the status list
every time after the for loop finishes,
where if it detected, something of a motion in that
particular frame, it will append.
This could be zero or this could be one for that frame.
Next, we're going to take the last two values
of the status list array, because we are just concerned
with the motion that happened.
We are not concerned with the previous motion.
We are just to get the past motion.
So we're collecting all the previous motions separately,
but for this current scenario we are just concerned
with keeping the length of the area to be of two values,
none and none.
We're going to slice it and you're going to capture that.
Next, we're going to add a condition.
The condition would be that yeah, so if the status list
previously was one and now it is zero.
Or if it was previously zero and now it is one.
Then you will take this times array.
We're going to take this times array,
let me define it over here,
which is going to be an empty array first.
You're going to add the t at this particular time
there was a motion that happened, so either one.
This is basically continuing with a status change
from a previous frame to a current frame.
Then finally, we are going to recall all of these changes
over here like this.
What we can do is finally, that we can iterate over the
times that you have collected and we can write it
to a CSU file.
Let me take this for a run, that we have some data
in our CSU file.
Go.
Right, so this is the original color frame.
If you notice, my finger would be recognized as an object.
Okay and yeah.
So this should give us enough data for movement.
Let's see what time.CSV has.
It seems to empty.
Let's sort of run it once again and see what we can get.
Might happen some bug.
Let me try to move it a little more.
Okay, right.
Let's see if we get something now.
So yeah, if you notice, now we have the time over here,
the start and the end time in a times.CSV.
We have two instances of movement that we captured quickly.
One is this and the other one is this.
We have actually detected motion.
Now we can actually go ahead and plot this.
The video part of the capturing part is done.
All we need to do now is just go ahead
and plot this on a plot using Bokeh.
For this, we are going to use Bokeh and we're converting
the time to a string format, first and foremost.
So that is over here.
Let's first import Bokeh and then convert the time.
Just so that's it's pretty because if you notice over here,
it's written like this, right.
It's not very human readable,
it's converted into more readable format.
Next we are going to use the column data source
with a frame to pass it the data stream.
As we have been learning till now, we're just gonna use
the same methods we have been learning about Bokeh
and we're going to plot this.
So we're going to create a figure, x-axis is going to be
datetime, height of 100, width 500.
The title will be motion graph.
So we're going to set the text or the markings,
we're going to set the tooltips, we grab the tools,
and then you're going to take the plot, mark the coordinates
and going to set the output file as graph1.
Let's run this and see what we get.
So once I quite the camera, so now we have got a Bokeh plot,
as expected.
You have to quite the camera, please keep that in mind.
You can actually, I mean, run it in two different files.
You can just you know, because you are mapping it
in time.csv, you could just do it in different files,
just get times.csv first and then run it over here.
But if you notice, now we've got the start and the end
and it says okay, for 30 seconds or 35 seconds
we got this motion.
We have our file over here as well, graph1.html.
Of course you will not get an exactly same graphs.
I mean, it depends on what kind of a motion you do
I front of the camera.
Thank you, thank you so much.
I hope you have enjoyed listening to this video.
Please be kind enough to like it and you can comment
any of your doubts and queries and we will reply them
at the earliest.
Do look out for more videos in our playlist
and subscribe to edureka! Channel to learn more.
Happy learning.