Let's talk about ways to minimize time spent on memory accesses.
The first strategy to think about is move frequently accessed data to fast memory.
We've talked about the memory spaces available to the GPU.
There's local memory, which represents a given thread's private variables, local variables,
parameters, things like that.
There's shared memory, shared by a thread block.
There's global memory shared by all the threads.
And more or less, it's true to say that local memory is faster than shared memory,
which in turn is faster than global memory.
In fact, shared memory and local memory are usually much faster than global memory.
I should mention that there are subtleties here.
For those of you who know something about computer organization,
the reason why I'm labeling local memory as so fast is that it tends to live
either in registers or in l1 cache, and those are both quite fast.
This isn't a hard and fast rule. There's some subtleties here.
But in general data that is kept local to a thread is going to be about as fast as possible.
Data that is shared in a thread's thread block shared memory is going to be very fast,
and data that is way out in global memory is going to be a lot slower,
although this is still much faster then CPU memory, also known as host memory.
Let's see an example of using local, shared, and global memory.
Here's a kernel.
I know that it's a kernel because it starts with either device or global.
It's called use local memory GPU, it has one parameter called in,
and it's got one local variable called f.
Because this is a local variable it's in local memory it's private to this thread,
every thread will have its own copy of a variable named f.
And parameters are also local memory.
Every thread will have it's own copy of a parameter called in.
You know real code would presumably do something with these variables.
But since this is just an example of how to use local memory,
I don't need to do that.
How would you call a kernel that shows using local memory?
Well, you would call a kernel,
you would have to launch it, meaning tell the GPU how many thread blocks to run with how many threads.
You pass in any parameters. So in this case, 2.0. Pretty simple.