Before we perform a transfer taking the face of one person and transferring it onto a video of some
other individual we want to make sure that we're doing the correct thing.
There are a lot of rounders and be tuned the model needs to be trained.
Color settings need to be specified and you have to make sure that you're getting the face is correctly
there aligned properly and so on.
For that reason you want to start with a dry run a quick test ideally with the most visibility you can
get before you start performing week long computations or more with expensive GP use.
For that reason I'd like to start with a gooey version of face while allowing me to see all the photos
or faces the results of the extractions the progress of the model as it's being trained and most importantly
it gives me the command line arguments that are later on be running on some GP you on the cloud that
won't have as convenient of an interface.
So for that reason we're going to perform our dry run now both for instructor purposes and for preparatory
work for a more involved face swap.
So first thing I'm gonna do is just install the gooey version of face while I'm going to do it on Windows
because that's the most convenient but there is an analogous path and you can do on Linux because the
computations are not supposed to be involved in this change.
You might want to just do this on a VM if you are on a Linux machine do it on a VM for Windows.
So I'm just going to download this file which is the current really save it and then you're just going
to install it so open it.
Click Next Next knickknacks etc or configure it the way you want it but there's nothing very complicated
there.
Installation does take a while and sometimes it looks like it's hanging but in all likelihood it's just
really slow it took something like an hour.
In my case so don't be alarmed if that happens we're going to skip the installation part just because
it's not instructive and it takes a long time and I'm going to assume that you've already finished installing
it when you're finished installing it.
You'll have this face web application ready to use and here's a shortcut that I created and when you
open it you'll see this problem.
It's normal and then it's going to open up a good which is what we want.
Going to start with a clean new session.
Now let's discuss our goal here.
So I've created a bunch of directory I've created an N and into out out to prediction training model.
So n contains a video of me.
So that's me giving some presentation.
And the point is it has my face and has my face in various angles and that's what I want and that's
what I want.
Then there is another video this time I've chosen a prominent political figure to the most recent thing
I could find on youtube where there's good visibility of this person's face and my goal is going to
be to take the face from N and overlay it on into in a convincing manner so obviously more generally
you can put in in the end folder the stuff you want to use as the overlay image and then add into the
target videos where you want to put the image onto there's a bunch more folders and we'll get to them
in a moment.
So now let's go back to face well when you open it you see the screen and we're gonna start with extract
so extract allows you to get the faces in your sources but more importantly allows you to fix problems.
So for instance if it gets the wrong face let's say there are several people in the video and you're
only interested in one of them then you'll be able to delete the faces that aren't needed if it incorrectly
gets a face say it sees some object that incorrectly mis classifies the face.
You can delete those and there is more sophisticated settings that you can play with different extractors
and normalization and so on.
But those aren't critical at the moment for first run getting started.
You just want to get a first basic prototype so input directory is gonna be our N and I put in a video
on there although you could also put a bunch of images or potentially frames on video.
So I'm going to point it to the video and that's that output.
That's where all the frames with faces will be abstracted.
So that's my first out and alignments for now.
We leave as is empty.
Then there's different choices for detector.
One of them might be better but it's slower and requires a Jeep you.
Right now I don't have a jeep.
Since this is just a dry run so I'll just pick this one which is faster.
The masking is currently not supported.
Leave it out and basically just leave all the default.
Basically just leave the defaults as they are and when I click generate it's gonna give me the command
line input.
To perform this operation and I want it I'm gonna make sure that I'm copying this to save it for later
when I'm working on a remote Linux machine where it's easier to just do the command line interface.
So let's record this and press extract and this is going to take a little bit of time in your case as
you run it the first time.
It'll also be downloading the model so it'll take a bit longer so you can see it's getting my face pretty
successfully but it's also getting some sort of play on the wall that apparently looks like a face and
that's actually exactly why we do this stuff.
So we can later on when it's finished adjust and make sure that it doesn't consider this to be a fake.
Otherwise we'll be superimposing this on our target which is not what we want.
So now we're finished with this process.
Let's take a look at what it extracted.
So I'm going to go out and you can see my face but there's also this play.
Now if I were to go one by one erasing these invalid images that would be kind of tedious.
So there is a faster way in this small example.
It doesn't give you much mileage but imagine if you had tons of footage then this could be very time
consuming.
So what we're gonna do is actually sort the images by faith so go to tool then sort.
And then we're going to put in this output folder and we'll put nothing here because we want to sort
and place and we're going to do sort by face.
So this is gonna download the v g g model and then use it for face similarity and then sort and leave
all the other stuff default.
I'm going to leave this even default because for some reason I see few versions in work or to generate
a command line.
Man save this.
Come back to this and undo it.
It's not doing all this sorting using the G model in your case it also download the V G.G. model.
But I've found this before.
So I've downloaded it.
Wait.
Alone.
OK now it's done.
Let's examine the results.
OK so what do we see all the plates are together and all my images are together.
That's good that it's not confusing my face with a plate.
Now we can easily erase all this keeping the face that we do want to keep success.
Now we have to fix the alignment because we've deleted a bunch of faces so we're gonna click remove
faces and select the alignment file that's gonna be in your end.
So it gets automatically created.
Here you put the output folder where we have adjusted the images so we'll want to make sure we generate
command line command copy save and then run.
And as a result you see there's a new alignment which has now take into account the images we delete.
Now we want to run what we just did with the first input on the second input.
So let's do that.
Now we're gonna select input.
Number two
so everything is exactly the same except we're now pointing to input number two rounded wouldn't see
it's getting the face of the target.
No plates this time
and looking into preview we might not even need to remove any faces because it's doing quite well.
Now let's examine the output.
So it looks pretty good and looks like there are no false positives here.
So we can skip.
Do you remove all faces.
Now what we want to do is train our models.
We finish all the extraction and fixing the extraneous spaces so to train you put in the initial phase
and then the final day so input A's and B's faces.
We want to remove in this case it's gonna be our out to folder and then there are the alignment which
are going to be an in to four input B is going to be the face we want to put on top to overlay.
So that's going to be out one.
And the alignments is going to be fixed alignments where I remove the plate
now the model that gets trained here will be saved somewhere and this is where we specify model directory.
So that's the training model folder.
Then you can pick different models for training.
For instance you might know that famous face swap with the Jennifer Lawrence and Steve Luscombe.
It uses the villain model.
So this is the villain model right here.
But as you can see it's very resource intensive.
There's the original DP.
There's a bunch of other ones.
Since we're going for a quick dry run I'm going to use lightweight which expects a small amount of computation.
We'll still give it less than it's collecting because we're not even using it for you here.
But that's OK because we are just doing a quick run to get the command line inputs and make sure that's
doing what it should.
So next we take our batch size by size has to be less than the number of images we have.
Otherwise you cannot satisfy.
So in particular I have here 115 images so bad science can be of 215.
Usually it's gonna be powers of two but this is fine now a number of iterations.
Again we're going for very few right now.
So I'll say a hundred skip all this stuff and then generate the command and then train and as the trains
will be able to see some progress we'll be able to see a graph some more information and preview if
this is your first time running this it might also download the appropriate model which might take a
little bit longer.
So here's a preview.
Let's try to understand what's happening.
So there's the original not being straight for those original error or original.
That means that the model found the original say then it has some conception of what the face looks
like.
And I put it in there.
Right now it's not very good as you can see because it's some sort of gray law and it has the result
of putting supposedly my feet onto the target as overlay against not very good because it hasn't yet
gotten a good understanding of what these faces should look like.
Basically it thinks that a face or analysis grave it buys it keeps training will come to be more and
more realistic.
So it's gray blobs will become faces.
So let's give it some more time.
The train
can also check out the graph so we can see here the losses.
Well there isn't too much data yet.
I can look at some metadata and otherwise just wait some longer.
Now it's finally finished the training.
Let's see what the results are.
So we look at our folders we looked at the training model folder and we see an encoder decoder.
There are some logs where we can see it's a relatively large model and we have completed the training.
Next step is to actually perform a prediction.
Now it's not going to be very good because we have spent very little time training and their data size
pretty small but that's OK.
All we want is a dry run.
So click on convert and other input directory.
We're going to put the target video and output that's going to be some other new folder called prediction
which is empty alignments.
So these are going to be found in into and in the model directory.
That's what we just trained.
Training model.
Yes.
Now there are some settings that you can play around with.
We can just skip those.
Now when it comes to provider actually the setting does matter.
So if you were to choose open CV for instance and the result is going to be a bunch of frames you would
have like a thousand images or whatever number it is.
And they're going to be the fake but there's still gonna be a bunch of frames and not a video.
So I would choose F F M P G and then the result will be an actual video the rest of the stuff we can
ignore.
Let's get the command line command
we'll add it to our list of commands and I'll press convert so now we'll have to wait in the meantime
we can take a look.
Prediction you can see it's already starting to create a video and here's a preview.
So you can see it's overlaying a face but it's not very good.
It didn't quite learn the concept of my face because I didn't have enough time and potentially enough
of a training set.
But our goal is to see that everything works and that we can scale it up
so now it's completed.
Let's see what monster we have created.
OK.
So we can see there it is.
There are good things about it.
There are some bad things.
Good thing is that it does seem to get the alignment of the target face to the original face.
Bad things.
Is that the face itself is very much blurry and ill defined.
No worries.
We now know what we're going to do.
So next stop we are going to do this.
But this time we're gonna do it for real.
We're going to use a Jeep you know scale it up.
See you then.