- So far, we've learned the basics of hallucinations.
In this chapter, we'll dive into what can cause LLMs
to give incorrect responses.
To start us off, let's talk about time-sensitive data.
LLMs are trained on data from a certain point in time
and usually share this in their model description.
For example, looking at the GPT-4o model description,
we can see that it was trained on data up to October, 2023,
signified by the knowledge cutoff.
If we head over to the Anthropic website,
we can see that Claude Sonnet 3.5
was trained on data up to April, 2024.
So if we try to ask either of these models questions
about something new, like the Euro 2024 football tournament,
the LLM will either make up an answer
or say that they can't respond.
Now let's go ahead and try this out on the ChatGPT website.
Let's type out,
"Who won the 2024 Euro Cup championship?"
Let's say, "Don't use internet knowledge," and hit Enter.
So there we go.
We got a refusal to respond on this tournament.
Now let's go ahead and say,
"You can use internet knowledge."
Okay, there we go.
We got the correct answer
based on using Wikipedia data.
Now, sometimes these models do hallucinate.
Let's go ahead and copy this prompt and refresh.
I'm going to paste this in, say, "Don't use internet knowledge
and only respond with the name of the country,"
and let's hit Enter.
In this case, the model still used internet data,
so it's hard to be reliable in these kinds of circumstances.
So in prompting large language models,
it's important to know what kind of information
your model has access to,
based on the knowledge cutoff date,
and any other context you provide.
In the next video, we'll cover how training models
with a poor mix of data can also cause hallucinations.