Image captioning is another very popular application, and particularly this is this is a problem in
which ordinance really is succeeding very, very well as compared to the traditional timecards models.
The problem setup is, is the following.
You have some machine learning model that's a machine learning model here, some machine learning model
that you really want to build, which takes as input an image.
And IGB image or or some image, maybe a greyscale RGV image, and the output of this model is text
or maybe a paragraph or maybe a string, maybe a lot of sentences that actually explains this particular
image.
So take an example that I have taken from the CARUS GitHub.
Take an example, for example, if this is the input to the model, that image is input to the model.
The output here is, as you can see, that the text is basically the output of the model.
A young boy is playing basketball.
That is a description or the caption of this particular image that the model is is required to to generate.
See, for example, another image like this image.
And the caption the model has generated for it, as do dogs play in the grass.
So, I mean, the the authenticity of this caption, maybe you may add more details.
For example, a young boy is playing the basketball and they are and several other boys they are sitting
behind.
You may extend this caption, but see, for example, the caption that is available, although it's
a little short, but it it actually explains the image.
And this is very complicated problem.
And given the image, you're going to actually explain the image.
Why text?
Similarly here, you can you can have a different caption that may be valid.
For example, two dogs playing a grass and there is a there are certain trees behind on the area.
And I mean I mean, you can you can have a different explanation.
But this one is really a cool caption that the model actually came up with automatically.
Similarly for this particular image.
You can see the caption, a dog swims in the water.
I mean, you can see it.
It is it is not very easy to actually judge that this particular object as a dog.
And further it is actually swimming in the water.
I mean, getting that all these seen it belongs to water and the dog actually is swimming in there,
that's difficult.
At least before the birth of recurrent neural networks, it was close to impossible.
See, for example, this image, a little girl in a pink shirt, so see, for example, that that's
how detailed caption is there a little girl?
First of all, the girl is little a little girl in a pink shirt.
So it also detect that it's a pink shirt is swinging.
That's amazing.
I mean, the result of this particular for this particular image is really amazing.
You can see how detailed caption is that and how accurate is this?
A group of people walking down a street.
So that's the caption for this.
The caption for this is a group of women dressed in a formal attire.
You can see the caption for this.
The two children play in water.
I mean, yeah, it it actually came up with the description that all this scene actually corresponds
to water and these two objects.
First of all, these are two objects.
Second, they both are kids.
They are children.
And you can also see a dog jumps over a hurdle.
So that's so a model actually automatically detects that this is a hurdle.
That's a dog.
And actually the dog is jumping over hurdle.
So this image captioning actually is a really cool application where we really want to.
So, for example, think, for example, you have a database of images and you want their automatic
explanations or you want to generate another database of text explaining those images.
Yeah, I mean, several, several applications that came up with and and by the way, the best machine
learning model to date that is outperforming all the existing models is a recurrent neural network that
has convolutional neural networks as layers.
Let me let me show you a dataset corresponding to we're corresponding to these kind of applications.
For example, the Skoko dataset is there some sometimes called Kocho or Coocoo.
That's dataset that is available for object segmentation, recognition and context.
And particularly this dataset basically also belongs to five captions body image.
So it has a lot of images and each image actually contains a lot of captions.
So while training the model well, training the model, you need to have a lot of datasets to dataset
to actually train such a model that actually perform this kind of task.
So such a data sets, for example, several datasets are available.
One common and more popular dataset is concluded as it.
So let's discuss some more applications of recurrent neural networks.