Now you may be thinking, "Oh my gosh, Michael
you have 12 CPUs, you can make your code
go 12 times faster."
Sorry to tell you that is almost never true.
Every now and then you'll run into algorithms
that are sometimes referred to
as embarrassingly parallelizable.
If you do, say, ray tracing, and every single pixel
is going to have it's own track of computation.
Yes, we could probably make that go nearly 12 times faster.
But most algorithms, and most code, doesn't work that way.
So if we look at maybe the execution
of a particular algorithm, we have these two sections
here, these two greens sections
that are potentially concurrent.
Right now they're not, but imagine when we said
"Oh that section and this other section.
We could do that concurrently."
And let's say those represent 15% and 5%
of the overall time.
If we were able to take this code
and run it on an algorithm that entirely broke
that up into parallelism, the green parts.
Let's say the orange part cannot be broken apart.
We'll talk about why that is in just a second.
If we can break up this green part
and let's imagine we had as many cores
as we want, a distributed system
on some cloud system.
We could add millions of cores if we want.
Then we could make those go to zero.
And if we could make the green parts go to zero
like an extreme, non-realistic experience
but think of it as a upper bound
then how much would be left?
80%, the overall performance boost
we could get would only be 20%.
So when you're thinking about concurrency
you need to think about, well how much
can be made concurrent
and is that worth the added complexity?
And the added challenges, as we'll see.
Maybe it is. It very well may be. But it might not be.
In this case, maybe a 20% gain but really added complexity.
Maybe it's not worth it. Remember that 20% is a gain
of if we could add infinite parallelism
basically, to make that go to zero
which won't really happen, right?
So you want to think about what is the upper bound
and why might there might be orange sections?
Why might there be sections
that we just can't execute in parallel?
Let's think about how you got in this course.
You probably went to the website, and you found the course and you clicked a button
and said, "I'd like to buy this course,"
put in you credit card, and the system
went through a series of steps.
It said, "Well, OK, this person wants to buy the course.
"Here's their credit card.
We're going charge their card
then we're going to record an entry in the database
that says they're in the course
and then we're going to send an email
that says, hey, thanks for buying the course
here's your receipt, go check it out."
That can't really be parallelized.
Maybe the last two.Maybe if you're willing to accept
that email going out potentially
if the database failed, it's unlikely
but, you know, possible.
But you certainly cannot take charging the credit card
and sending the welcome email
and make those run concurrently.
There's a decent chance that for some reason
a credit card got typed in wrong
it's flagged for fraud, possibly not appropriately
but, right, you got to see what the credit card system
and the company says.
There might not be funds for this for whatever reason.
So we just have to first charge the credit card
and then send the email.
There's no way to do that in parallel.
Maybe we can do it in a way that's more scalable
that lets other operations unrelated to this run.
That's a different consideration
but in terms of making this one request
this one series of operations faster
we can't make those parallel.
And that's the orange sections here.
Just, a lot of code has to happen in order
and that's just how it is.
Every now and then, though, we have these green sections
that we can parallelize, and we'll be focused
on that in this course.
So keep in mind there's always an upper bound
for improvement, even if you had infinite cores
and infinite parallelism, you can't always
just make it that many times faster, right?
There's these limitations, these orange sections
that have to happen in serial.