So guys, in this specific video, we are actually going to create an end to end project of chatting
with multiple PDF documents, uh, using Google Gemini.
Now, uh, one amazing thing about this entire project will be that we'll also be using Lang Chain,
and we'll try to understand how Lang Chain has bought some important ways of integrating Google Gemini
Pro, you know, and probably developing amazing applications altogether.
So everything we will try to develop it.
I know this will probably be uh, a lot many things.
There are two main important features that we are also going to use in this.
One is of uh, one of the vector embeddings technique.
You can use any different different vector embeddings.
It is up to you.
But today I will show you a vector embedding technique of which has been created by Facebook.
Okay.
We'll talk more about it as we go ahead.
Now first of all, let me just go ahead and show you how the demo will be.
Right.
The demo.
And then we will try to develop this entire application.
So over here, uh, when I say chat with PDF, I'm talking about chat with multiple PDF.
So what I will do I will go to browse.
Let's say I have two important PDFs over here.
And these are the research paper.
One is attention is all you need and YOLO right.
And I will just go ahead and open it.
Now once I open it what we will do we will go ahead and click submit.
Now what is going to happen is that this entire PDF files will be getting converted into vector embedding.
And it will be stored in local or in any database as such.
Right.
I've already showed you how you can use pinecone database.
You can use Cassandra DB, you know, to store all this kind of vectors.
Right now you can see over here I've got the status as done.
Now this is one important functionality here.
What we did, we uh, browse the file, we uploaded it, any number of PDFs, you can upload it.
The limit size will be 200 MB per file.
So any number of files, you can probably upload it.
And as soon as I probably clicked Submit and Process, it has actually got converted into vectors.
Now whatever question I ask about this specific PDF, I will be able to retrieve it.
Now as as said right, I've uploaded this two PDF.
One is attention is all you need, so you can go ahead and take up any question and probably ask to
it.
Okay, so let's say I will go ahead and ask what is scaled dot product attention okay.
So I will say what is scaled dot product attention.
So let's see whether I will be able to get the response or not.
So it is running over here.
It is already been converted into vectors.
So I hope I should be able to get some output over here.
So let's see.
Uh so here you can see scaled product attention is a mechanism used in the transformer model for calculating
the attention weights between different position in sequences.
So amazingly we are actually able to get the response.
Not only that uh, I will say please provide a summary about Multi-head attention.
Let's say, and you can write any prompt that you want.
Provide a summary about multiple head attention.
So I will just go ahead and, you know, submit it so you can probably see.
And many people have actually asked me, Krish, how do I probably interact with multiple PDFs.
Like they can be any number of PDFs over here.
I've even tried with 10 to 15 different kind of PDFs over here.
So here you can see the reply is Multi-head.
Attention is a technique used in transformer model to improve its ability to attend, uh, the different
parts of the input sequence.
Now this is with respect to attention pdf I also have yolo pdf.
So let's uh ask okay, provide a detailed summary.
I will just go ahead and write provide a detailed summary summary about VOC 2007 error analysis.
So I will go ahead and, and uh, press enter.
And I hope I should be able to get the response.
Now this is the entire demo.
But again, creating uh, the entire application will help you learn multiple things over here.
So we'll go ahead and talk about it as we, uh, go step by step.
Now here you can probably see you have got the answer also.
And you'll be able to see that you are getting the entire result.
Okay.
So let's go ahead and now develop this application.
I will completely start from scratch.
As I said that we are also going to use this Facebook AI similarity search for this.
Um, this is whenever you work with vector embeddings you can probably use this.
You can also use Chrome RDB if you want.
But uh, in this particular video I have already used in one of the Project or Chrome tab.
But let's go ahead and now use this for this.
So guys now let's go ahead and develop this project step by step.
Please follow all the steps that I'm specifically telling you.
Um, and uh, the best thing is that we'll try to keep it much more modular.
We'll try to use a front end application, a back end application.
We'll try to see that how when a vector embeddings is basically created, how you can store it in your
local or you can probably if you have an.
You can also go ahead and store it in any database as such.
Okay, so first of all, as usual, what I will do is that I will go ahead and open a terminal okay.
See guys, whenever you probably start it right now, you always need to create a new environment whenever
you want to probably start a project.
Okay, I've already created a environment called as v and v.
Okay.
But I here I will give you a command how you can actually create an environment.
Okay.
So for that you need to write conda create minus p v and v.
And then you have to probably use this is my environment name v and v.
And here Google Gemini Pro.
It works well with at least Python that is greater than 3.9.
Okay so here I will probably use 3.10.
So as soon as you write this and just press enter, what will happen is that environment will get created
on the name of V and V okay.
So please do this.
Steps I know.
See I'm not showing you completely from uh, executing the command because I've already done that in
my previous videos.
Right.
So you don't have to do again and again and wait till that end steps actually happen in this video.
Okay.
So once this V and V environment is created we need to go ahead and write conda activate this specific
environment that is v and v okay.
Because inside this environment only I will go ahead and probably install all the libraries.
So these are the first two steps that you really need to follow.
One is create the environment.
So for that you can write conda create minus p v env.
Um then you can give the python version python double equal to 3.10 okay.
As I said Google Gemini Pro works well with Python version greater than 3.9 plus.
Okay, now the second thing that we will go ahead and do is that create our environment variable.
Okay.
So here I have to use Google API key.
I have already shown you how you can use Google API key.
Just go to maker suit google.com/api key.
So this is the website that you really need to go maker suit.google.com/app/api key.
Just click on this create API key in a new project.
As soon as you click it you can copy the API key and you can paste it over here.
And that is the same thing that I've actually done over here.
So here is the same API key I have pasted it okay.
And the name that I'm actually giving is Google underscore API underscore key okay.
So this is the second step.
This is my environment variable.
Without this I will not be able to use Google Gemini okay Google Gemini Pro.
Now the next thing is that I will go ahead and install all the libraries that will be required for this
specific use case.
So I will go ahead and write step by step and what all libraries I will be requiring.
As I said, I will be requiring Streamlit.
I'll be requiring Google generative AI.
This is for Google Gemini Pro.
I will be requiring Python dot env.
Yeah to load all the environment variables.
I will be requiring lang chain.
Why?
Because we need to read the PDF.
Lang chain has lot of functionalities to read the pd pdf and for again reading the PDF.
Also we have something called as pi pdf two.
If you want you can use Chrome RDB, but right now in my use case I will go ahead and use fish CPU.
Okay, fish GPU will also come, but right now I'm going to use CPU because I have less number of files.
If you have more files you want to do parallel processing, you can use fish GPU.
Now this will be the another important library that you will probably be using.
That is nothing but lang chain Google underscore lang chain underscore Google underscore JNI.
So this basically means that with the help of lang chain, you will be able to access the APIs of Google
generative AI.
Okay.
So these are my important libraries that I will be requiring for this project.
Now what we will do we will go ahead and write pip install minus our requirement dot txt.
So once I do this you can probably see that I have already done the installation.
So it is showing requirement already satisfied.
The main reason is that you should not waste much time in this two steps.
So that is the reason I have already done this and I have.
I'm proceeding with it.
Okay, now this is done.
I'm good with the requirement dot txt my API key.
My uh API key is ready.
Everything is ready.
Now let's go ahead and do the coding.
And this time we will try to do it completely from scratch.
We'll try to understand what all things are there okay.
So first of all I as I said I will go ahead and import Streamlit as St.
So Streamlit is done.
Now.
Along with this I will also be using from Pi.
From Pi PDF to okay.
Import PDF reader.
PDF reader.
Okay, so this is the libraries that I will also be using because PDF reader, with the help of PDF
reader, we will try to read all the documents.
Okay.
And uh, apart from this, what I will do is that, uh, there are some important, uh, uh, libraries
that need to import with respect to long chain.
Okay.
So what all important libraries that we are going to import.
First of all I will go ahead and paste it over here okay.
So from long chain dot text splitter I'm going to use recursive character split text splitter.
Uh as you know that from uh what we need to do as soon as we read the PDF, as I said that we need to
convert those into vectors.
So, uh, Google generative AI, like basically the Gemini Pro also provides you an embedding technique.
Okay.
So for that I will write from long chain, long chain underscore uh, Google underscore gen I import
Google generative AI embeddings okay.
So we are going to specifically use this uh from Google long chain underscore Google Dot.
So don't worry about this.
Warnings.
It will go off as soon as we execute for the one time.
Okay.
Then along with that I will go ahead and import from Google generative AI as Jenny.
Okay.
So this is done because I'm going to use this generative AI functionalities only.
Now along with this uh, I will be importing four different libraries and I will explain you everything
step by step.
Okay.
So one is for this.
This is for the vector embeddings.
Uh, from long chain dot vector stores.
We will try to create that then long chain Google Jen I also has chat Google generative AI okay so this
I will try to make you understand why we are specifically using this.
Then from long chain dot question answer we are importing load queue chain.
So then this basically helps us to do the chat and any kind of prompts.
Also I want to define.
So for that also I am importing this long chain dot prompts import prompt template.
And finally I'm also going to import dot e and v load dot env.
So this is specifically to load all the environment variables.
So these are some of the imports.
Now I will go ahead and write load underscore dot e and v.
By this.
What will happen is that you will be able to see the environment variable okay.
And then finally I will write Jen I dot configure because I need to configure my API key from Google
Germany.
So I will write OS dot get os dot.
Get E and B, and here I will go ahead and write Google because that is the same thing.
API key.
So what we have done over here is that I'm configuring the API key with respect to whatever Google API
key we have loaded in this dot env file, or we have put them in this slot env file.
Now understand one thing.
I hope you have visualized the demo part right.
In that demo part you could see there was a left side where we are.
We got some place to upload the PDF and convert it to a vectors.
And in the right side there was a text right now in the left side, as soon as we upload a PDF, we
should be able to read the PDF and then whatever data is there inside the PDF we should be able to give
it right.
So for that what I will do I will create a definition.
I'll create a function and I'll write get underscore PDF underscore text okay.
Now in this case whatever will be my PDF underscore docs right.
Whatever document I'm uploading it now here I will go ahead and create a text variable.
And I will say for.
PDF in.
PDF underscore docs.
Right.
So what I'm saying I'm I'm just saying that okay, read all the PDF pages in this particular PDF underscore
docs, and I will go ahead and read this with the help of PDF reader.
Right.
So PDF reader I hope I have.
Okay.
I have imported it over here.
So this will basically be my PDF reader.
So PDF reader will be responsible in reading this specific PDFs.
Okay.
Now as soon as it probably reads this PDF in PDF, multiple pages will also be there.
Right?
So the main important internal component will be that soon as we read a PDF using PDF reader, it will
be able to get the details of all the pages.
So that basically means this PDF reader will be in the form of list.
So what I will do.
So I will go ahead and say for page in PDF reader okay.
So I'm going to probably read all the pages inside that.
So I will write dot pages.
And after this I will go ahead and say text plus is equal to page dot extract text.
So if you go ahead and write extract text, that basically means from this particular page we are going
to extract all the details, all the text in uh inside that okay.
So once we do this then what we'll do finally we'll go ahead and return the text.
So this is perfect right.
So this steps in short is saying that we read the PDF.
We go through each and every pages and we extract the text.
So this is the first information that we want okay.
Because later on we will try to get this text.
And now as usual what we'll do we will create another function.
As soon as we get this text I will divide this text into chunks, smaller chunks.
So I will write get underscore chunks okay.
Step by step.
We'll we will try to do this please.
You really need to follow all the steps because once you follow this.
So this will be generic for all the PDF files with respect to whatever application we develop in generative
AI.
Okay.
So here I'm going to give my text.
So this will be my another function.
And here I will go ahead and write text underscore splitter okay.
As you know we have used recursive uh character text splitter.
We have imported it over here.
Right.
So we are going to use this.
And I'm saying hey, make the chunk size of 1000 instead of 1000.
I'll say 10,000 and overlap since I have a very big PDF.
So here I'll say my chunk overlap will be 1000.
Okay.
So overlapping can basically happen.
So here what is basically happening I have the entire text.
And I want to divide those texts into smaller chunks of this particular size of 10,000 words I guess,
or 10,000 tokens.
And they can be an overlap of 1000.
Okay.
So that I get a detailed summary of any questions that I specifically ask.
Okay.
Now once this is done, then I will go ahead and write text splitter dot split bit underscore split
underscore text.
And I will give this specific text over here okay.
So I'm taking this particular text and splitting based on this information that I specifically have.
And then after this I will go ahead and return my chunks.
Perfect.
So this was my PDF.
I got the text.
I divided the text into chunks.
Now the main thing will be that I'll take this text chunks, whatever chunks I have and I will convert
that into vectors.
Right.
So here I will say definition get vector store.
And here I will go ahead and write text underscore chunks okay.
Whatever text chunks I'm specifically getting.
And here I'm going to upload my embeddings.
Embeddings which embeddings that I have used over here Google generative AI embedding.
So I'll copy this.
And in Google generative AI embeddings if you see the documentation, there is a model which is basically
called as inside the models folder, which is nothing but embedding 001.
Okay.
So I'm going to use this embedding technique okay.
To probably do the embedding again in long chain.
Also you have different embeddings in OpenAI.
Also you have different embeddings.
But right now I want to use an embedding which is completely free for everyone to use it from hugging
face.
Also, you can do it, but Google generative AI embeddings is already providing so many features.
So why to go with others and pay money?
So here I will go ahead and write.
Vector store is equal to phi s dot from underscore text.
Okay.
So I'm going to see when I probably create this embedding.
I have already imported files right.
So files is already here files from text.
And I will say take all this text chunks and embed according to this embeddings that I have initialized.
Okay that's it.
The embedding technique.
So this way it is basically creating the vector store.
Now this vector store can be saved in a database.
It can also be saved in a local environment.
Right.
So what I will do for right now I will go ahead and write vector store dot save and I will say local.
And here I will say fires underscore index okay.
Fires underscore index.
Now fires underscore index over here shows you that I'm going to save this entire information in local.
Right.
So this will be the folder that will be created.
And inside this folder I will be able to see my vectors right in some format which is not readable.
Okay.
So you'll also be able to see a pickle file so that whenever I ask a question to those vectors I should
be able to get the information.
So these are the three steps.
First we got the PDF.
Then we converted that into chunks.
And then we finally divided that into vectors.
Okay now I will go ahead and write definition get conversational chain okay.
So here I'm going to basically use this get conversational chain.
And here I'm going to define my prompt.
I'll define my prompt template and let me.
Provide some meaningful prompt.
Okay.
So I will say I'll give this question.
See.
So this is my question.
Answer the question as detailed as possible from the provided context.
And make sure to provide all the details.
If the answer is not in the provided context, just say answer is not available in the context.
Don't provide the wrong answer.
So this is the information that I've actually used in the prompt template.
I'm saying that you need to behave like this.
You need to, uh, act like a person who knows how to read PDF and all.
Okay, so this is my first step with respect to that.
And uh, after this, uh, we will see the next step, what it is going to happen.
Okay.
Now this is my prompt template that I've actually created.
Now I will go ahead and create this particular function that is chat Google generative AI.
And here I'm going to specifically use model which is my Gemini Pro.
I hope this is everyone is familiar with it.
I'm going to initialize my model Google chat, generative chat, Google generative AI, Gemini Pro,
and here I'm going to specifically use some temperature.
And let's say the temperature that I'm going to use is point three.
Okay.
Then obviously I've created prompt.
So I am going to use the prompt template which is available in Lang Chen.
And I will write template is equal to prompt template okay.
And you know what are the input variables.
Right.
So if you go ahead and define my input variables it will be in the form of list.
And there are only two.
One is context and the second one is specifically called as question.
Okay I think it should be question.
Yeah.
Question context and question okay.
So these are my information with respect to the prompt template.
And this will basically be my prompt.
I'll divide this into prompt.
So I have to probably create my prompt.
Now once we complete this once this is done I will go ahead and create my chain.
So it will be load queue chain.
And inside this I will use model whatever model that I have actually specifically defined.
And I'm going to use the chain type as stuff because I also need to do internal text summarization okay.
So this chain type stuff documentation chain will actually help us to do that.
And I will go ahead and give my prompt okay.
Done.
Perfect.
So this looks good.
And finally let's go ahead and return this chain.
So this is what is basically going to happen with respect to get conversational chain okay.
So this is basically all the functionality this is going to take.
We are going to load the Gemini Pro model.
We are going to create our template.
And then we are going to get the chain okay.
Now finally with respect to the user input, like as soon as I probably define or write in the text,
something should happen, right.
So here you go.
So here you can see I've created a user input.
So this will basically go with my user question.
Okay.
Uh if you probably go and see this is my embeddings that I'm loading, then I load the file index from
the local okay.
This is basically loading.
See, at the end of the day when I give any information in the text input, right, I will say, hey
tell me about the summary of this particular topic from that particular PDF.
Right.
Already that PDF is converted into vectors, so it is already stored in the files index.
Right.
So that is the reason we are loading files.
Right?
We are loading this file index from the local right.
And then we are trying to do the similarity search based on the user question.
Right.
And then we are calling as soon as we do the similarity search what we call we call this function right
after doing the similarity search.
So by this we will be getting the chain back.
And then we get the response.
And finally we display the response over here okay.
So these are the steps in user input.
So this is specifically related to our text box what is basically happening in the text box okay.
So again I'll tell you first of all we give the question.
Then what will happen as soon as we give the question.
It will do the similarity search on all the files index files, vectors that is basically created.
And then we go with the conversational chain.
I get the conversational chain.
And then we go to the response.
Here in the response we give the chain input documents as docs, which is basically based on the new
similarity search and question is basically my user question.
And finally we get the response.
Now quickly what we will do.
We will create our Streamlit app.
Okay, so let me go ahead and create my definition as main function because this will basically have
my Streamlit app.
So here let me go ahead and probably do this.
And again I've used ChatGPT okay to probably develop the front end.
So here chat with multiple PDF.
Let's go ahead and write multiple.
If okay then start header chat with multiple PDF again.
Using Gemini.
User question is stored text input.
Ask a question from the PDF file.
If user question, then user input.
See if I am giving this specific question, then this function should get executed automatically that
basically, as soon as I write the question over there in the prompt and I press enter, then it should
basically execute this.
Okay.
But there will be also a sidebar where I need to upload the PDF and convert it to vector.
So that is the reason this particular code is there right with sidebar then s t s t dot title menu,
then upload a PDF, an uploader button.
And I'm saying that as soon as we submit the button we call get PDF text.
Then we got get the text chunks and finally we get Get vector store.
So this function that you will be seeing this three function get PDF text get text chunk get vector
store.
It is only for making sure that your files index is created, right?
So that is the reason it is given in the sidebar.
And then we upload the file and we process it.
Okay.
That is what we are specifically going to do in that particular case.
So this is done very simple I think I hope everybody is familiar with this.
And finally I will go ahead and execute it.
So let's go ahead and done.
I will say hey if underscore underscore name is equal to underscore underscore main I'll call this main
function.
Now quickly let me go ahead and run this.
I think it should work absolutely fine.
So I will go ahead and write.
Streamlit.
Run.
App.py.
So let me just go ahead and call this.
Okay.
Now see this is the left side, right?
I gave a browse button.
As soon as I submit it then what will happen?
This entire code will be called right.
One by one function will be called.
Which function will be called first get PDF text will be called, then get text chunk will be called,
then vector store.
As soon as I do the vector store you will be able to see here.
Right now nothing is there.
No files?
Index?
Nothing is there?
Okay, now I will go ahead and browse it.
I will select both the PDF.
Now see as soon as I submit and process.
So it is running processing.
Now here you will be able to see one folder that is files index.
Now it has index dot file.
So obviously you cannot read it.
And there is one index dot pickle pickle.
Right.
So this files index will be my vector where the vector similarity will be done.
All this information can also be stored in the database.
Okay.
Uh as said right.
This can also be stored in the database.
And it can probably be called now this is done.
Now here I will go ahead and ask a question.
Now my question will be let's say uh provide a summary of provide a.
Detailed summary of.
And I'll paste it now.
See, as soon as I press enter now where it will go.
It was this function, right?
See over here.
As soon as I press enter, if user question I press enter, then it will call this function.
That is nothing but user input.
Then here embedding model will be called.
Then will load the files index.
Then it will do the similarity search and it will find out the chain.
And based on the chain it will provide me the response okay.
Response of output text.
So uh okay so let me just reload this.
I think I did not press enter or what I don't know.
Provide a summary of.
I will press enter.
Okay.
Price index is already there.
Let's see.
I think we should be able to see some chat.
Yes, here you are able to get it right.
So guys, I hope you like this particular video.
This was it, right?
Uh, an amazing end to end project.
Now you are able to see chat with multiple PDFs using Gemini.
And I've shown you how you can use with link chain, how you can use with files, index and all.
You can also do it with Chrome Web.