Welcome back, so I just wanted to show you quickly here that few -shot prompting
has actually been empirically proven to increase the accuracy and effectiveness
of these models.
This isn't something that we've just made up.
This paper right here is a very famous scientific paper published by OpenAI,
titled Language Models are few -shot learners, and this is where few -shot
prompting comes from.
The terminology didn't exist before this
paper, and this figure right here is one of the key figures from that paper, and
it shows that these models actually do get more accurate as you provide more
shots or more examples, right?
So along the x -axis here, we have
parameters in billions, so the number of parameters that the different models
have, so here we have 400 million, 800 million, 1 .3 billion, and then here on
the y -axis, we have accuracy, so this is how accurate the responses were.
So let's take this model right here, the biggest one, 175 billion, and you can see
when it was given a zero -shot ask, it had about a 40 % accuracy rate.
When it was given one -shot, that increased the accuracy to about 50%, and
then when it was given few -shot, or multiple examples, it increased the
accuracy to about 60%, so that's really important to know for prompt engineering
, right?
The accuracy increases when you use one
-shot and even more for few -shot prompting, but here's the other thing I
want you to note as well, the accuracy, the jump in accuracy increases more for
the models that are bigger, right?
So this model here, 0 .4 billion, the
jump between zero -shot, one -shot, and few -shot is actually pretty small, but
as you go down the x -axis here and you get to bigger models like this 175
billion, the jump is actually quite substantial.
This is important to know because the
models are just going to get bigger and bigger and bigger over time, which means
using few -shot prompting is actually going to become more and more and more
important, assuming this trend continues, and this study was actually done in 2020
with GPT -3, so using few -shot prompting is actually even more important for these
subsequent GPT models.
So you can see how having the knowledge
of these prompt engineering techniques like few -shot prompting is already going
to put you miles ahead of anyone else who's trying to use these models for tasks.
One last note is that I highly recommend that you actually read this study.
It's a famous prompt engineering study, and it's quite long, but you don't have
to read it all, just read the first few pages even.
It's really going to give you a sense of how these models can be tested to
discover their abilities because these models are a bit of a black box, and
that's one of my favorite things about prompt engineering is that you can
actually scientifically find new techniques that allow you to use them
better, to make them more efficient, to make them more accurate.
Who knows, you could be the person who discovers the next prompt engineering technique.
So I'm going to include a link to this study in the handbook for this course.
Check it out, and once you have, I'll see you in the next lesson.