(upbeat music)
- [Instructor] Before we perform our deep fake transfer,
taking the face of one person
and transferring it onto a video of some other individual,
we want to make sure that we're doing the correct thing.
There are a lot of parameters that need to be tuned.
The model needs to be trained.
Color settings need to be specified,
and you have to make sure
that you're getting the faces correctly,
that they're aligned properly, and so on.
For that reason, you want to start with a dry run,
a quick test, ideally with the most visibility you can get
before you start performing week-long computations or more
with expensive GPUs,
so for that reason,
I like to start with a GUI version of Faceswap
that allows me to see all the photos, all the faces,
the results of the extractions,
the progress of the model as it's being trained,
and most importantly, it gives me the command-line arguments
that I'll later on be running on some GPU on the cloud
that won't have as convenient of an interface,
so for that reason, we're going to perform our dry run now,
both for instructive purposes
and for preparatory work for more involved face swap,
so first thing I'm going to do
is just install the GUI version of Faceswap.
I'm going to do it on Windows
'cause that's the most convenient,
but there is an analogous path that you can do on Linux.
Because the computations
are not supposed to be involved at this stage,
you might want to just do this on a VM.
If you are on a Linux machine, do it on a VM for Windows,
so I'm just going to download this file,
which is the current release, save it,
and then you're just going to install it,
so open it, click Next, Next, Next, Next, et cetera,
or configure it the way you want it,
but there's nothing very complicated there.
Installation does take a while,
and sometimes it looks like it's hanging,
but in all likelihood it's just really slow.
It took something like an hour in my case,
so don't be alarmed if that happens,
so we're going to skip the installation part
just because it's not instructive, and it takes a long time,
and I'm going to assume
that you've already finished installing it
When you're finished installing it,
you'll have this Faceswap application ready to use,
and here's a shortcut that I created,
and when you open it, you'll see this prompt.
It's normal,
and then it's going to open up a GUI, which is what we want,
so we're going to start with a clean new session,
and now let's discuss our goal here,
so I've created a bunch of directory.
I've created an in, an in2, out, out2,
prediction, training_model,
so in contains a video of me,
so that's me giving some presentation,
and the point is it has my face,
and it has my face in various angles,
and that's what I want, and that's what I want.
Then there's another video.
This time I've chosen a prominent political figure.
I took the most recent thing I could find on YouTube
where there's good visibility of this person's face,
and my goal is going to be to take the face from in
and overlay it on in2 in a convincing manner,
so obviously, more generally,
you can put in, in the in folder,
the stuff you want to use as the overlay image,
and then, in in2, the target videos
where you want to put the image onto.
There's a bunch more folders,
and we'll get to them in a moment,
so now let's go back to Faceswap.
When you open it, you see the screen,
and we're going to start with extract,
so extract allows you to get the faces in your sources,
but more importantly, it allows you to fix problems,
so, for instance, if it gets the wrong face,
let's say there are several people in the video,
and you're only interested in one of them,
then you'll be able to delete the faces that aren't needed.
If it incorrectly gets a face,
say it sees some object
that it incorrectly misclassifies as a face,
you can delete those,
and then there's more sophisticated settings
that you can play with, different extractors,
and normalization, and so on,
but those aren't critical at the moment.
For first run, getting started,
you just want to get a first basic prototype,
so input directory is going to be our in,
and I put in a video in there,
although you could also put a bunch of images
or potentially frames from a video,
so I'm going to point it to the video,
and that's that.
Output.
That's where all the frames with faces will be extracted to,
so that's my first out,
and alignments, for now, we leave as-is, empty.
Then there's different choices for detector.
One of them might be better,
but it's slower and requires a GPU.
Right now I don't have a GPU, since this is just a dry run,
so I'll just pick this one, which is faster.
The masking is currently not supported, so leave it out,
and basically, just leave all the defaults.
Basically, just leave the defaults as they are,
and when I click Generate,
it's going to give me the command-line input
to perform this operation,
and I want it,
so I'm going to make sure that I'm copying this
to save it for later
when I'm working on a remote Linux machine
where it's easier to just do the command-line interface,
so let's record this and press Extract,
and this is going to take a little bit of time.
In your case, as you run it the first time,
it'll also be downloading the model,
so it'll take a bit longer,
so you can see it's getting my face pretty successfully,
but it's also getting some sort of plate on the wall
that apparently looks like a face,
and that's actually exactly why we do this step,
so we can later on, once it's finished, adjust
and make sure that it doesn't consider this to be a face.
Otherwise, we'll be superimposing this on our target,
which is not what we want.
So now we're finished with this process.
Let's take a look at what it extracted,
so I'm going to out, and I can see my face,
but there's also this plate.
Now, if I were to go one by one
erasing these invalid images,
that would be kind of tedious,
so there is a faster way.
In this small example, it doesn't give you much mileage,
but imagine if you had tons of footage,
then this could be very time-consuming,
so what we're going to do is actually sort the images by face,
so go to tool, then sort,
and then we're going to put in this output folder,
and we'll put nothing here 'cause we want to sort in place,
and we're going to do sort by face,
so this is going to download the VGG model,
and then use it for face similarity and then sort
and then leave all the other stuff default.
I'm going to leave this even default
because for some reason the CPU version didn't work
or to generate a command-line (indistinct),
save this, come back to this, and then do it,
so now it's doing all this sorting using the VGG model.
In your case, it'll also download the VGG model,
but I've ran this before, so I've downloaded it.
Wait a little bit.
Okay, now it's done. Let's examine the result.
Okay, so what do we see?
All the plates are together,
and all of my images are together.
That's good that it's not confusing my face with a plate,
and now we can easily erase all this,
keeping the face that we do want to keep.
Success.
Now, we have to fix the alignment
because we've deleted a bunch of faces,
so we're going to click Remove Faces,
select the alignments file.
That's going to be in your in,
so it gets automatically created.
Here you put the output folder
where we have adjusted the images,
so we'll want to make sure we generate command-line command,
copy it, save it, and then run,
and as a result, you see there's a new alignment
which has now taken into account the images we've deleted.
Now we want to run what we just did
with the first input on the second input,
so let's do that.
Now we're going to select input number two,
so everything is exactly the same,
except we're now pointing to input number two.
Run this.
We can see it's getting the face of the target,
no plates this time,
and looking at the preview,
we might not even need to remove any faces
because it's doing quite well,
so now let's examine the output,
so it looks pretty good,
and looks like there are no false positives here,
so we can skip the removal of faces,
and now what we want to do is train our model,
so we finish all the extraction
and fixing the extraneous faces,
so to train, you put in the initial face
and then the final face,
so Input A is going to be faces we want to remove.
In this case, it's going to be our out2 folder,
and then there are the alignments,
which are going to be in in2.
For input B,
is going to be the face we want to put on top to overlay,
so that's going to be out1,
and the alignments is going to be the fixed alignments
where I removed the plate.
Now, the model that gets trained here
will be saved somewhere,
and this is where we specify model directory,
so that's the training_model folder.
Then you can pick different models for training.
For instance, you might know that famous face swap
with Jennifer Lawrence and Steve Buscemi.
It uses the Villain model,
so this is the Villain model right here,
but as you can see, it's very resource-intensive.
There's the original deep fake,
and there's a bunch of other ones.
Since we're going for a quick dry run,
I'm going to use a lightweight,
which expects a small amount of computation.
We'll still give it less than it's expecting
because we're not even using a GPU here,
but that's okay because we are just doing a quick run
to get the command-line inputs
and make sure that it's doing what it should,
so next we pick our batch size.
Batch size has to be less than the number of images we have,
otherwise you cannot satisfy it,
so in particular, I have here 115 images,
so batch size can be up to 115.
Usually, it's got to be powers of two, but this is fine.
Now, a number of iterations,
again, we're going for very few right now,
so we'll say 100, skip all this stuff,
and then generate the command
and then train,
and as it trains, we'll be able to see some progress.
We'll be able to see a graph,
some more information, and preview.
If this is your first time running this,
it might also download the appropriate model,
which might take a little bit longer.
So here is a preview.
Let's try to understand what's happening,
so there's the original.
(indistinct) straightforward.
There's original, arrow, original.
That means that the model found the original face.
Then it has some conception of what the face looks like,
and it put it in there.
Right now, it's not very good, as you can see,
because it's just some sort of gray lob,
and then it has the result of putting, supposedly,
my face onto the target as overlay.
Again, it's not very good
'cause it hasn't yet gotten a good understanding
of what the faces should look like.
Basically, it thinks that a face, right now,
is this gray thing,
but as it keeps training,
it'll come to be more and more realistic,
so its gray blobs will become faces,
so let's give it some more time to train.
Can also check out the graph,
so we can see here the losses,
but there isn't too much data yet.
We can look at some metadata
and otherwise just wait some longer,
so now it's finally finished the training,
so let's see what the results are,
so we look at our folders.
We look at the training_model folder,
and we see there's an encoder.
There's a decoder.
There's some logs,
and we can see it's a relatively large model,
and we have completed the training.
Next step is to actually perform a prediction.
Now, it's not going to be very good
because we have spent very little time training,
and their data size pretty small, but that's okay.
All we want is a dry run, so click and convert,
and under input directory,
we're going to put the target video and output.
That's going to be some other new folder called prediction,
which is empty.
Alignments, so these are going to be found in in2,
and then the model directory.
That's what we just trained,
so training_model, yeah.
Now, there are some settings that you can play around with.
We can just skip those.
Now, when it comes to the Writer,
actually, the setting does matter,
so if you were to choose open CV, for instance,
and the result is going to be a bunch of frames.
You have, like, 1,000 images or whatever number it is,
and they're going to be deep fake,
but there's still going to be a bunch of frames
and not a video,
so I would choose Ffmpg,
and then the result will be an actual video.
The rest of the stuff we can ignore.
Let's get the command-line command.
We'll add it to our list of commands,
and I'll press Convert,
so now we'll have to wait.
In the meantime, we can take a look.
Prediction.
You can see it's already starting to create a video,
and here is a preview,
so you can see it's overlaying a face,
but it's not very good.
It didn't quite learn the concept of my face
because it didn't have enough time
and potentially, enough of a training set,
but our goal is to see that everything works,
and then we can scale it up.
So now that it's completed,
let's see what monster we have created.
Okay, so we can see there it is.
There are good things about it. There are some bad things.
Good thing is that it does seem to get the alignment
of the target face to the original face.
Bad things is that the face itself
is very much blurry and ill-defined.
No worries.
We now know what we're going to do,
so next stop, we are going to do this,
but this time, we're going to do it for real.
We're going to use a GPU, and we're going to scale it up.
See you then.