Lesson finished
Up next: Classical Segmentation: Thresholding, Region Growing and Watershed
Unlock all 26 lessons of Advanced Medical Image ProcessingFull course ₹40,000 · paid
Formal Definition of Segmentation and Image-Based versus Shape-Based Methods
A free lesson from Skill-Lync's Advanced Medical Image Processing course, on Grow by Skill-Lync · Last updated 16 Nov 2026 · Session: “Module 1 Session 2”
What will you learn in this free Advanced Medical Image Processing lesson?
The lesson recaps the clinical cases, then defines segmentation as mapping pixel intensities to class labels such as liver or spleen. It surveys method categories and compares the advantages and drawbacks of image-based, shape-modeling and hybrid techniques.
- Segmentation as a function from pixels to labels
- Common properties: intensity, texture and edges
- Categories of segmentation methods
- Pros and cons of image, shape and hybrid approaches
What comes after lesson 2?
The next free lesson is lesson 3, “Classical Segmentation: Thresholding, Region Growing and Watershed” (33 min). Later in Skill-Lync's Advanced Medical Image Processing course (paid): lesson 4, Module 1 Session 4; lesson 5, Module 1 Session 5; lesson 6, Module 2 Session 1. The full course has 26 lessons (8 h 43 min of video); see the course overview or its price and syllabus.
Lesson transcript
Machine transcript of the lesson audio (lightly cleaned). Technical terms may contain errors; the video is the reference.
Hello, everyone. Welcome back to the course on Advanced Medical Image Processing. We are going to continue our discussion on image segmentation. A quick recap from our previous discussion. So last time, we discussed about the course outline and also the learning objectives we are going to fulfill from this course. We also reviewed the motivation and applications for image segmentation, especially focusing on medical image segmentation. Then we also discussed a few clinical cases just to understand where segmentation is useful and how critical it is. For example, in the case of radiation therapy, we understand the need to not damage healthy tissue.
And in the case of 3D visualization, we noticed that how a 3D visualization of bone structures can help surgeons. Now, in this part, we are going to discuss further about image segmentation, starting out with different categories of image segmentation. So we defined previously that image segmentation is the process of identifying the boundaries of different objects in an image. So how do we formally define it? So I believe everyone is familiar that a digital image is a grid-like structure and it consists of multiple pixels. At different locations of the grid, there is an intensity value of each pixel.
So to formally define image segmentation, we formulate it as follows. So given a set L of labels or image classes, the process of image segmentation maps each pixel I to a specific class from L. So let's say the label of pixel I is L I. So formally, the function seg of I would map I, pixel intensity or pixel, to a label. Now this label can be anything depending on the particular context. In the set of natural images, it could be object categories like sky, grass, cow, animal, car, etc.
And in the case of medical images, it is primarily based on organ and background. So one category is definitely background, which is not very informative. Then the other categories can be different organs. For example, in this image, there are three different colors, which show three different categories. So it can be, let's say, liver, it can be spleen, it can be other situations, or it can be other organs that are present in the image. So from a high level, image segmentation is actually partitioning the image into different subsets or sub-regions of similar properties.
And some of the common properties that are considered are intensity, texture, and edges. So an important point that I would like to make here is that before the deep learning era, when image segmentation was focused on what we call the classical approaches, so we had to define specific properties. And some of the properties from the images that were extracted are intensity, texture, and edges. So these are common properties. Of course, there can be also other higher level statistics and all, but those are not frequently used.
So going to the discussion on segmentation approaches. So some of the approaches for image segmentation are, the first category can be extraction of the object information and representing it as a geometric structure. If you recall the example of the cranial vascular analysis, at the last stage, you are representing it as a set of objects or point cloud. So this is what is referred to as the geometric structure. Another category of segmentation approach is by recognition, where we determine the object's position in the image. And a third category is delineation, measuring the object's spatial extent, its composition, identifying the object boundaries, and where it is overlapping or coming closer to other objects.
So going to the discussion on segmentation by recognition methods. So they can be primarily interactive approaches. Interactive in the sense it requires some kind of human input, such as in identifying seed points or initializing a structure. So the interactive methods can be subcategorized into model-based approaches, where we have the active shape or appearance models. So this particular approach of active shape, they require us to have some kind of input on where the important landmark points are. Then in knowledge-based approaches, use statistical shape priors. What is required here is, from a training set of images and their corresponding segmentations, a statistical shape model can be constructed.
And then, of course, there are atlas-based methods, which construct an atlas or a reference shape from multiple training images. And as I mentioned, this is because there is some level of human intervention in these kinds of approaches. Now, in the case of delineation-based methods, it can be further categorized into image-based approaches, shape model-based approaches, and hybrid approaches that combine the above two. In the image-based approach, the methods, they rely on information available in the given image. So they don't use any additional prior knowledge. And so this involves, like, water thresholding, region growing, or water segmentation approaches.
Then in shape model-based approaches, shape model is constructed from a set of training images, which learns the different properties of the shape. The advantage of shape model approaches is that it learns a much richer information than pure image-based approaches. But a limitation is that it requires a set of training images and manual segmentations. Hybrid approaches try to combine the advantages of both in the sense that they use image-based approaches to generate segmentations, although they are not as accurate as manual segmentations. But it saves time. And they combine it with the rich feature extracting properties of shape model methods.
Now, this is a high-level categorization of different segmentation methods. Something I would like to point out is, in this description, we are focusing only on the classical image segmentation method. So that's why you can see there is no deep learning over here. Because deep learning, in itself, is a category. So some of the prominent methods under each category is highlighted with red. For example, threshold-based methods, OTSU's method is the most popular. Region-growing for region-based method. Then for clustering-based method, it's K-means. Whereas for edge-based methods, canny edge detector, gradient detector, those are equally effective.
Then for image-based segmentation method, it can be broadly categorized as boundary-based methods or region-based methods. In boundary-based methods, the sole objective is to identify the boundary of the structure of interest based on edge value. So for that purpose, some of the common approaches are snake-based, active contours, or level sets, which require an initialization of the initial contour. Then region-based methods, they identify the region based on continuity. So, for example, clustering, using KNNs or C-means, then there is graph cut, watershed segmentation, etc. So some of the advantages of image-based segmentation methods is they give very high accuracy for good quality images.
And they can also determine the degree of match of model to image. In the sense that it is easy to quantify the similarity between the image and the model. But some of the limitations are, in case of poor quality or low informative images, the accuracy suffers. Because it is quite intuitive because the whole approach is based on image properties and low quality images don't give a lot of distinctive information. And another disadvantage or limitation is that for these approaches is preceded by recognition and which can be a problem at times.
And another important limitation is that although it gives a segmented area, it does not give information about the object shape. For example, what are the other characteristics of the shape? Neither does it give information about the geographic location in the image, which means what is its position relative to other important organs or other important landmarks. So the next category is shape modeling. So what shape modeling does, it creates a statistical model of family of shapes. So given a set of training images, which have the corresponding organ of interest manually outlined.
So shape models create a statistical shape model by looking at the mean shape of these training images or training segmentations. And they calculate the mean and the different principal components to understand what is the mean representation, what is the variations across the training set. And then the shape can be represented as a set of points. Now for. To segment a new image, the model would match the new shape by moving the points. So what it does is first there is some kind of alignment with this shape model and then the shape model is superimposed on that image.
And then the individual points are moved to get the final point or location that best fits the particular shape model. So as you can see, this is an example where there is a set of points that represent the shape and they are superimposed on a particular image to get the segmentation. And some of the approaches are active shape or appearance models or atlas-based models and manual tracing. Some of the advantages of shape model and the hybrid methods are the object or the model is very good at learning the object shape and the geographic information.
The model is quite good in the case of bad images and it also helps in recognition. And the limitations are that it needs a best match information, which means that the points, the different individual points of the shape model can match to multiple locations in the image. So we need to calculate what is the best match. There has to be a quantitative measure. And the second limitation is that the accuracy is affected in good quality images. This sounds counterintuitive, but the real reason is that in a good quality image, there are many other structures of interest and the different shape points can actually match to different structures which are not part of the desired object or the desired shape.
So we come to the end of the discussion of this part. So to summarize, in this part, what we learned is that we formally defined image segmentation by representing as a function that maps the pixel eye to a set of labels. Then we also had an overview of segmentation method categories, and we discussed it in a bit high level. And we also discussed the pros and cons of different approaches for image segmentation, such as the image base, the shape base, and the hybrid approaches. So with this, I would like to conclude this part and we'll continue our discussion in the next part.
Thank you. Thank you.