Hi.
In the last few lectures we split the data into train and test data sets.
Then we use the train data set to come up with a suitable dummy variables for our statistical model.
In fact this makes perfect sense because we aren't going to use the train dataset to build our model
and the test set to test it.
But in order to test the model in the test set we must have the same dummy variables as the ones in
the train set.
Fortunately we can easily use the code we've written for the train set.
As you can probably remember in the beginning before we started exploring and pre processing the data
we assigned the train data frame to a new working data frame.
That is we assign DLF inputs pre processing to be equal to loan data inputs train from then on we worked
with the DFA inputs pre processing to assess weight of evidence by variable and to create dummy variables
based on our assessment.
When we finish this task the DFT inputs pre processing data frame contained all the old data and the
newly created corresponding dummy variables.
So that means we can use the same code to create the same dummy variables for the test that offering
now the test data frame is used only to test and assess a model estimated on the training data.
Therefore when working with the test data frame we don't need to assess weight of evidence or come up
with categories we consider best to create dummy variables.
Rather we must create the exact same dummy variables we used to train the model OK.
We can effortlessly do that by assigning DFT inputs pre processing to be equal to loan data inputs test
instead of loan data inputs train and then rerun the whole code.
Please note that weight of evidence will be calculated and visualized.
However we don't care about them since calculating and visualizing weight of evidence does not make
any changes in the data at all.
We'll simply ignore the outputs of these steps.
There is one thing however we should do before starting as we mentioned after running the code with
the train data frame DFA inputs pre processing contains all the variables needed to train the model.
If we only assigned a new value to this data frame as we should in order to run all the same calculations
with the test data we'd overwrite the variables created for training the model.
That's why before we go back up there where we started our pre processing it is important to save the
train data set.
Therefore let loan data inputs train equal D inputs pre processing.
Now the data frame loan data inputs train and the data for and DFA inputs pre processing.
Both contain the pre process training set data OK so now we can go up here and make data frame.
Def inputs pre processing equal to loan data inputs test.
Of course we have to make data frame D F targets pre processing equal to loan data targets test as well.
Next we can simply run the code all the way up to where we made loan data inputs train equal def inputs
pre processing.
Thus def inputs pre processing contains the test data set with all the dummy variables needed for the
model.
Hence instead of loan data inputs train we make loan data inputs test equal to D F inputs pre processing
consequently data frames loan data inputs train and loan data inputs test contain the train and the
test data sets with all dummy variables needed for the model at the same time loan data targets train
and loan data targets test contain the corresponding outputs.
Great job.
That was much easier than expected wasn't it all the pre processing for our PD model is completed.
I suggest we export these data frames and store them in C S V files from the next section.
We will begin modeling using only the pre processed C S Vs.
Good job.
Thanks for watching.