Hello, and welcome back. I want to discuss now the Mumford-Shah Image Segmentation
technique, and I'm going to do that with one example. Let's just look at this
image. We talk that image segmentation means labeling the image. We want to label
every segment that has some coherence in it with a different name, with a different
number. One way of doing that is, of course, basically, painting that segment
with a uniform, or a very smooth grey value. And that's the type of stuff that
the Mumford-Shah algorithm and technique tries to do. Let's just discuss that.
Here, we have an original image. Don't pay attention to these two for the moment.
It's going to be very clear in just a few seconds. And this is the result of one
implementation of Mumford-Shah image segmentation. So, very rich gray values
become just a few smooth segments. Looks like almost like quantization when we talk
about image compression. So, the first part is that we want to approximate the
original image by just a few segments that are either constant or very smooth. Of
course, we want an approximation. So, one of the terms that this Mumford-Shah needs
to include is a penalty from deviating too much from the original image. Of course,
piece wise smooth, or piece wise constant, I could make this whole image wide. So,
that's one very smooth segment. But it's very far, if I do that, it would be very
far from the original image. So, one of the things is we have to somehow penalize
for the difference between these two images. For example, the mean square
error. We already talk about it very early on in this class. So, one penalty is the
mean square error. We want to get the piece wise smooth version of this image,
but not very far from it. Now, you might wonder, okay, if you're going to penalize
for the mean square error, why not to keep the image itself? That's very clever, but
we don't achieve any segmentation. So, how do I know I don't achieve any
segmentation? Because I'm going to add a term that penalizes for having too many
boundaries in the segment ed image. So, these are the regular edges of this image.
Very strong edges. These are boundaries of the segments that we have here. I don't
want to have too many. I don't want to have all these. I'm going to penalize for
the number of edges for how many, how much do I pay for having boundaries. So that
would be another term. I'm going to basically, on one hand, I want to penalize
for very large differences, and on the other hand, I'm going to penalize for
edges. So, if you have too much edges, you are going to pay a price. Bec, so, if I
keep the image as it is, I have no error here but I have lot of edges. Basically,
every pixel becomes a segment, so I'm paying a very large penalty for edges. If
I have a flat image, I don't pay any penalty for edges, but I pay a very high
penalty for error. And then, I have to do a compromise between these two and that's
what Mumford-Shah basic concept is, to write formulations that compromise between
a representation of the image that is too far from the original image. We want to
simplify representation not too far from the original image, and we also don't want
to pay a high price and get too many segments. Now, there are many ways of
doing this. There's some beautiful mathematical theory behind different
formulations that do this compromise. Some theory relates even to compression. You
have to compress this image and this edges, and then you try to optimize for
that, yours and my, need to compare for the error. And some very beautiful
techniques with, in the framework of what's called variation and formulations,
and, energy formulations really, a lot of very beautiful mathematical theory, which
actually relates to the mathematical theory that we're going to be discussing
next week when we talk about geometric differential equations and geometric
varation of problems. But, here's the concept. And very often in image
processing, you have a concept and the multiple ways of implementing that
concept. And I want to make sure that you basically learn during this class, the
concept behind Mumford-Shah. I should also mention to you that this concept of
approximation and penalty for too many edges applies also beyond image
segmentation. And people have extended the framework of Mumford-Shah type of
formulations to image registration and many, many other image and video
processing problems. Thank you very much. See you in the next video.