← Previous lessonFree lesson 6 of 6 · lesson 25 of 48

Unlock all 48 lessons of Advanced Medical Image Processing in Clinical ApplicationsFull course ₹40,000 · paid

Price & syllabus
Free lesson · 33 min · no payment

Thresholding, Otsu's Method, Region Growing and Watershed Segmentation

A free lesson from Skill-Lync's Advanced Medical Image Processing in Clinical Applications course, on Grow by Skill-Lync · Last updated 16 Nov 2026 · Session: “Module 7 Session 3”

What will you learn in this free Advanced Medical Image Processing in Clinical Applications lesson?

The lesson covers classical segmentation methods, starting with thresholding and variants such as Otsu's method using the grey-level histogram. It then explains region growing with seed points, split and merge, and the watershed algorithm with its over-segmentation issue and related graph methods.

  • Thresholding to binarize images, and Otsu's method
  • Region growing from a seed with a similarity threshold
  • Split and merge to avoid over-segmentation
  • Marker-based watershed and priority flooding

What comes after lesson 25?

Later in Skill-Lync's Advanced Medical Image Processing in Clinical Applications course (paid): lesson 26, Module 7 Session 4; lesson 27, Module 7 Session 5; lesson 28, Module 8 Session 1. The full course has 48 lessons (21 h 26 min of video); see the course overview or its price and syllabus.

Lesson transcript

Machine transcript of the lesson audio (lightly cleaned). Technical terms may contain errors; the video is the reference.

Hello, everyone, and welcome back to the course on Advanced Medical Image Processing. So, in this section, we are discussing on image segmentation. In the last lecture, what we discussed in summary is that we formally defined what is image segmentation by defining a function that maps the image intensities to a set of image segmentation labels. Then, we also had an overview of different segmentation methods based on the different categories. And finally, we also discussed the pros and cons of different segmentation approaches, such as image-based, shape-modeling-based, and a hybrid approach.

Now, in this section, we are going to further continue on our discussion on image segmentation, and we will focus on thresholding. What exactly is thresholding? In thresholding, we define a function g such that it looks at every pixel intensity, and if that intensity value is greater than a threshold t, it will map that value to a constant value 1. If the intensity is below the threshold t, it is mapped to a value 0. So, in essence, g makes the image into a binarized image. As an example, let's look at the image at the leftmost.

So, in this particular image, so it's a color image, it shows different features like snow, tree, boundaries, walls, and then in the middle image actually shows the histogram plot of this particular image on the left side. As we can see, there are the prominent peak 1, and there are two minor peaks at 2 and 3. So, the threshold value t is taken to be very close to the two peaks 2 and 3, and the resulting binarized image is shown at the rightmost corner. So, as you can see, everything has been converted to either intensity value of 0 or 1.

There are multiple thresholding techniques that are used in the image segmentation community. For example, there is inter-mode, which involves choosing a value between the histogram mode maximums. Then there is maxentropy. It maximizes the information measure between object and the background. Then there is mean or average, which denotes the mean value of the region as the threshold. Then there is min-error, that is, we choose a threshold that reduces the impact of noise. Then finally, there is OTSU's method, which is a non-parametric method, which minimizes the overlap between the object and the background pixel.

By non-parametric method, what we mean is, we do not have to choose a value, and the algorithm itself determines the optimum value. So, we'll go deeper into OTSU's thresholding. Just to give you an idea, this is a set of images. The top row shows the original images. The second row shows the manual segmentation. And then finally, we have the segmentation result by three different thresholding approaches, OTSU's, Wong, and Ray. So, we are going to go a bit deeper into OTSU's thresholding. Basically, it uses the gray value histogram of a particular image, i, as input.

So, what do we mean by a gray value histogram? When an image is given, if it is a color image, then it is converted to a grayscale image. The difference between grayscale and color images are, for color image, every pixel has three values corresponding to R, G, and B, which is the red, green, and blue channel. And the grayscale image actually has only one value for every pixel. So, this conversion between RGB and grayscale is like a weighted mean of the three channels. So, now we have the grayscale image histogram.

And then the output of the thresholding approach is such that it aims to provide the best threshold, or such that the overlap between object and the background pixels is minimized. So, to give you a recap, object pixels are those which are assigned value 1, and background pixels are those which are assigned value 0. So, the threshold maximizes the between class variance, which is denoted as sigma b square, and it can be formulated as p1 mu 1 minus mu square plus p2 mu 2 minus mu whole square, where mu 1 and mu 2 are the means of the object and the background classes, and mu is actually the desired threshold, and p are the class probabilities.

Now, let's look at this a bit deeper. On the right side, we have two images, which are showing two different types of histogram distributions, and u is actually the threshold. p1, or the probability of the object category, can be determined from the formula summation of pi, which is the probability value of pixel intensity i, and p2 similarly for the other pixels. Then, mu 1 and mu 1 and mu 2 can be determined based on the formula. So, what is actually done is, a cumulative histogram is first calculated, which is denoted as c, and then p2 and p1 are the probabilities, and they can be approximated from the cumulative density function.

So, this is the summary of the algorithm for OTSU's thresholding. So, first we compute histogram for u equal to different values. Then, we take t0 as the increment for potential thresholds. Then, we denote u as t0, and t as u, and finally, s max as 0. So, as long as u is less than g max, g max is actually the maximum intensity value, what we do is, we compute the cumulative histogram c, and then the mu i's, or the means for the different classes, i, like 1, 2, object background.

If there are more than 2, it will have a different approach. Then, we compute the sigma b square, or the inter-class variance, according to this formula here. And finally, if sigma b square is greater than thresholds max, then we set s max as sigma b square, and t equal to u. And if this condition is satisfied, then end, if, then finally, we set u equal to u plus t0, go back to the main while loop. Now that we have discussed about OTSU's thresholding, I'm going to talk a bit about iterative thresholding methods.

This is very commonly used for segmentation of images, especially medical images. The whole reason being that, OTSU's thresholding acts very well when there are good peaks in the distribution, intensity distribution. However, especially in the case of medical images, we don't get very good multimodal intensity distributions. As a result, we have to do iterative thresholding where at every iteration, different thresholds are chosen. So as to get a good segmentation in a localized region. Since OTSU's thresholding focuses on a global threshold, this singular value of a global threshold may not give optimal segmentation results for noisy images.

Hence, as a result, the iterative thresholding method works best. The way it works is, imagine a term source to background ratio, which is basically the ratio of pixels in the background and for the foreground object. So iterative thresholding method, or ITM in short, is based on calibrating the threshold-to-volume curves at varying ratios when we acquire images using different settings, such as magnetic strength or different hardware settings. This is particularly useful when we are using phantom measurements using spears of known volumes. The measured ratios of the source to the background ratio of the lesion are then estimated from different images.

So to give you some context, this is especially relevant for PET images. PET, which stands for positon emission tomography images. They are especially useful to detect where there is abnormal blood flow or blood accumulation. So as a result, they can very easily detect tumors or lesions or other things, provided, of course, we get a very good separation of the intensities. And these iterative thresholding methods are particularly useful for PET images. So coming back to the ITM, the lesions are estimated from PET images, and the volumes are iteratively calculated using the calibrated signal-to-background ratio.

So once this step is done, the resulting PET volumes are compared with known SPEAR volumes and CT volumes. Generally, what we see there, CT volumes are co-scanned or co-acquired with PET volume when the problem in hand is about detection of tumors. And these CT volumes sometimes serve as gold standard. Now imagine when we have an initial PET image. Which looks like a heat map. In the first iteration, we get some kind of regions, separation. And then as we go ahead, as we move ahead with multiple iteration, it is able to identify all the different heat map regions, ones which are actually showing some kind of activity and separate them based on their intensity region.

So this is particularly useful when we are doing breast cancer detection or lesions in breast PET images. The way it is done is that when we have iterative thresholding method, when it is applied to a particular image, it gives an output which is quite different from the rest of the image, the rest of the background. After we get this processed output, by doing thresholding, we are able to get the desired lesion region. Also, iterative thresholding is very useful for segmenting bones in CT images. If you may recall, the CT images show the bones as a very prominently white region.

However, a simple thresholding will not work because even within the bone region, which shows up as a white region, there are some noisy pixels clustered around. And if we use a single threshold, it is bound to give us very spurious results. So as a result, we can do iterative thresholding in localized regions to give us the final output. Some common examples have been like head, neck, CT images, where the head and the neck are imaged together. The thresholding is done to model the skull. Using ITM, once we get the skull region, we can model the skull parameters, the skull topography and everything.

One can also use multiple thresholds, and that is especially relevant for MR images. In MR images, what happens is that there is a distribution of intensities. For example, the whole background is black, so that's easy to separate out. And even within the foreground regions, we have multiple peaks. And these peaks are not distinct in the way that we can't use multiple values to come up with a good thresholding, thresholded segmentation. So what happens is the thresholding is done interactively, and it separates the image into different regions.

So in the histograms, it is displayed, and the values in the histogram indicate potentially useful threshold value. So by doing this thresholding, actually we get two advantages. So we are able to do good contrast normalization of the images, in the sense that when pixel intensities below a certain threshold are set to 0 and above a certain threshold are set to 1, we get a much improved contrast where the high intensity regions are highlighted as white, and the other regions have a good intensity distribution, and that helps in automated image analysis.

Let's also discuss a bit about region growing. Region growing is also another important algorithm, which is very important even in the current stage of deep learning based approaches. So in region growing methods, first we select a set of seed points, then we define a growth function, and then we group all the voxels that are connected to the seed and satisfy the growth function. So the growth function can be based on four-neighbor connectivity or eight-neighbor connectivity. So in four-neighbor connectivity, the growth function is such that it determines the four closest neighbors in the two-dimensional space as the connected components, and in eight-neighbor connectivity, we consider all the neighbors for the pixel as its connected component.

Then we keep on grouping them and continue till no further voxels satisfy the growth criteria. And to do this, a depth-first search or a breadth-first search approach is used. So this is essential because many of the subsequent algorithms that were presented or that have been developed for image segmentation borrow a lot from region growing methodologies or the concepts of region growing. Even so as to say the current state of the art deep learning methods, they also incorporate a lot of fundamental approaches from region growing methods.

So a region is basically a group of connected pixels with similar properties and they have close boundaries. And the computation of regions is actually based on similarity of some kind of feature representation. The regions may correspond to objects in a scene or parts of objects, or in the context of medical images, they would correspond to different anatomical regions. And the spatial proximity of two regions indicates there is some kind of similarity. So one of the simplest approaches for region growing methods is called merging, which is something you have seen. So the idea is to select a seed point or points, we define a growth criteria, join voxels connected to the seed that satisfy the growth criteria, and then we stop when no adjacent voxel agrees with the growth criteria. So in this context we can have four connectivity or eight connectivity. This is particularly useful when we have to do fat segmentation in MR images.

So in the context of a T1-weighted MRI, we can actually able to, which is focused on liver segmentation, we can separate out the visceral fat, the liver, and the subcutaneous fat. This is possible because we are able to focus on points or group of points that have similar property. Furthermore, we can also do 3D segmentation of lung CT images, or we can also do lobe segmentation of images. And all this can be applied to multiple 3D slices. So to define a pseudocode for region segmentation segmentation would be like starting with a seed point in a gray level or a color image, having some brightness, let's call it B1. At the beginning, the current segment only contains the seed pixels. As part of an iterative process, we will merge any pixel, let's call it Q, to the current segment, that is 8 adjacent, or rather 8 connectivity, or switch satisfies 8 connectivity to a pixel, in the current segment. And this would also satisfy the condition that B of Q, or the intensity of Q, the difference between Q intensity and seed point intensity should be less than a particular threshold, let's call it tau, which has to be greater than 0. And we continue with this set of steps, still there is no further pixels which satisfy the above criteria. So once we choose the seed pixel, we check the neighboring pixels, add them to the region if they are similar to the seed. Then we repeat this till no more pixels can be added. So what happens is, when we grow the region, so starting from one single point, the region continues to expand. That means more and more pixels are being merged. And it could also be, so as with every iteration, we could keep the merging criteria as constant, or we could change it. So what happens is it, so there is something also called a split and merge algorithm. And unlike region growing, region splitting starts with the whole image as a single region, which is the opposite of a merge algorithm. And then it subdivides the whole image, or the region into subsidiary regions recursively, as long as the condition of homogeneity is not satisfied. And the region merging is actually the opposite of splitting. And it works as a way of avoiding over segmentation. So what does over segmentation mean is that, if the number of distinct objects is let's say n, and our algorithm comes up with some number of regions, let's say m, where m is greater than n, then that is called over segmentation. So a very interesting way to do split is to take the image, divide it into different blocks, take each block, subdivide it into further smaller blocks, and repeat that for each of the different blocks. So these continue, still we get some kind of uniformity of the different regions, or the different sub-blocks in the region. However, there is also some drawbacks associated with the split algorithm. The primary drawback being, the final image partition may contain adjacent regions with identical properties. Ideally, they should have been able to merge together. But since it does not happen so, it can give you distinct regions, which is not desirable. Well, intuitively, we can think that in such a situation, the solution is to do merging of the algorithms. Hence, we have this algorithm called split and merge. So what happens in merge is when a homogeneous region has been You can see that one of the perc of the world in the world in this world, the most part of this world, the movement of the world's, It's neighboring regions are checked. And if the neighboring region is similar to this particular homogeneous region in question, these two regions are merged and a newly created region is formed.

So basically we are looking at if the two regions to be merged, if they have similar or very identical properties, then merge them. And if the similarity criteria is met by more than one adjacent region, then the new region is merged with the most similar one. So basically this continues till we are able to get uniform region. And that is the advantage of using split and merge. So get an idea just to recap what we have discussed about split and merge is that if we over split or if we go to two deep splitting steps, then we are bound to get over segmentation where there are too many isolated regions.

So as a result, what we have to do when we reach a certain depth of splitting, we have to look at the different regions. And if there are homogeneous regions or regions with similar properties, they are then merged together. This happens iteratively till there are no more such regions to be merged. Now, where does all this fit into the picture? This is particularly important when we are doing virtual endoscopy or colonoscopy. So there is a CT image and then a 3D region growing is initiated from some seed point.

And then it is gradually grown till we get the desired organ of interest. As you can imagine, before the machine learning and deep learning based methods became popular, region growing was a very important method. And as a result, there have been lots and lots of works and papers that have been published for this particular technique. So, for example, you could do tumor segmentation from PET images and lung nodule segmentation, etc. One important concept of region growing is called adaptive parameterization. So what it means is that when we choose that threshold, like when a region is greater than a threshold, it could be mean of R, where R indicates the currently grown region.

So if we have an original image and we do region of interest selection, and then we do thresholding to get some kind of output. And finally, we are able to use this adaptive parameterization approach in order to get very good segmentation output. This can also be used, the region growing algorithm, for functional evaluation of left ventricle in magnetic resonance imaging. This is very important because left ventricle, the main objective of imaging left ventricle is to get a functional measure of the blood flow. So what happens is that contrast enhanced prospectively electrocardiogram, or ECG, gated multi-slice spiral CT images is used as an invasive or rather a non-invasive tool for detection of coronary artery stenosis.

And the assessment of the left ventricle at N-diastole and N-systole is used for quantifying the function of the left ventricle. So as a result, we need to segment the left ventricle and at both these phases. So in systole, the left ventricle is actually shrunk, whereas in the diastole stage, the left ventricle is expanded. So we need to get the volume of each stage, and the difference is actually the flow. So we could also do a region growing based segmentation for lung or soft tissue separation, and as well as for airway segmentation, where what we do is we start out with a seed point, let it grow, and then till we come to a stage where there are no more filamental structures for the algorithm to grow.

So an airway segmentation, as we can imagine, is actually consists of very fine structures, and it's not easy to get them in one single thresholding. So region growing is actually very much suited for this kind of problem. Then there is also something called bone vanishing disease, where the CT images are important, and also because of the volume of the bone is not continuous across slices, the region growing actually serves as a very important and essential tool technique in such a situation. And this can be applied for different kinds of bone vanishing situations, whether it is the chest or the brain or rather the skull or even the limbs.

There are also some limitations of region growing, such as the objects in medical images are actually most often they are non-homogeneous, so region growing can be a bit difficult, but still it is fairly good. And when there is poor contrast, it causes leakage of segmentation. Poor contrast is very common for medical images because they are not acquired and are very high resolution. Hence, we have to be very careful with how to select the threshold, and we have to monitor the segmented outputs. So as a result of this leakage, one can also have noisy segmentations where we have spurious regions or over-segmentations, leakage, and all.

So in summary, what we can say is that the region growing-based segmentation methods are still popular, and although they might be used in conjunction with deep learning-based methods, but still they are important and popular. And the similarity criteria for such approaches is homogeneous regions. Another important segmentation method that closely resembles region growing is called watershed segmentation. In the context of image processing, a watershed is a transformation defined on a grayscale image such that it converts the image into a kind of a geological watershed. In the study of geography or geological images or structures, a watershed is like a drainage divide which separates adjacent drainage basins.

And the watershed transformation. And the watershed transformation treats the image as if it's operating on a topographic map, and the brightness of each point kind of represents its height. And the watershed algorithm finds the lines that run along the tops of ridges. And as you can imagine, there are different technical definitions of a watershed. In graphs, watershed lines may be defined on the nodes, on the edges, or hybrid lines on nodes and edges. Waterset can also be defined in the continuous domain. And there are many different algorithms to compute watersheds.

The watershed algorithms are used mostly for object segmentation, and especially where we need to count the objects or for further analysis of the separated objects. So this particular approach of segmentation by watershed flooding, this basic idea consists of placing like a water source in each regional minimum. And then build barriers when different water sources meet. The resulting set of barriers constitute a watershed. And intuitively, what we can also imagine is a drop of water falling on a topographic relief flows towards the nearest minimum. The nearest minimum region is where lies the end of the path of steepest descent.

And in terms of topographical analogy, this occurs if the point lies in the catchment basin. And intuitively, we can understand that the watershed is a separation of the regional minima from which a drop of water can flow towards distinct minima. So how do we go about implementing this whole set of ideas in an algorithm with image pixels? So what happens is we can start out with this. We label each minimum with a distinct class label. Then initialize the set S with the label nodes. Then we extract from S a node of minimal altitude.

So let's call it as F. And then we attribute the label of X to each non-label node adjacent to this particular node in consideration. And repeat step 2 till there are no more nodes in the set S. So in this context of watershed-based segmentation, we can define a function W. And it is a watershed of a function if and only if W is less than F. And W preserves the contrast between the regional minima. And if we have to actually start implementing in the context of image segmentation, so what needs to be done is we have to first identify local minima of the gradient of the image.

And they can be used as markers. And in this case, over-segmentation is produced. And the second step involves merge different regions in order to get a watershed appropriate segmentation. So marker-based water transformation, what makes use of specific positions of the markers that have been either explicitly defined by the user or determined automatically with morphological operators. One of the most common watershed algorithms is the one introduced by Meijer in 1991. And it is also called priority flooding. And it has a lot of variance. And it is especially suitable for data sets consisting of huge number of pixels.

What happens is the algorithm works on a grayscale image. him. You. After successive flooding of the region, watersheds with adjacent catchment basins are constructed. Then this flooding process is performed on the gradient image, which means that the basins should emerge along the edges. Usually this leads to an over-segmentation, especially if the image is noisy, such as medical images, CT or MRI. So either the image must be pre-processed or the regions must be merged on the basis of similarity. criteria. So how this happens is, first we define the set of markers, pixels where the flooding will start.

And each is given a different label. Neighboring pixels of each marked region are inserted into a priority queue. Then the pixel with the highest priority level is extracted from this queue. And then if that neighbors of the extracted pixel have been labeled, then the pixel is labeled with the same. If different neighbors are not of the same label, then there is some kind of merging criteria which is used. And this step is repeated till the priority queue is empty. So a variance and improved versions of this particular flooding algorithm also been developed.

And then there is something called optimal spanning forest algorithms or also called watershed cuts. What they do is they establish consistency of different watersets. And in order to compute the dividing lines, minimum spanning forest concept is used, which means it introduces a linear time algorithm to compute this particular set of steps. The watershed algorithm or rather the principle of watershed algorithm is also used in many other techniques, such as in graph cuts, where it was established that the graph cuts are related to optimal spanning forest.

And when the weights of graph is above a certain number, the cut minimizing the graph energy is a cut by maximum spanning forest. Then there is also a shortest path forest where image foresting algorithms are used to compute shortest paths. Random walker is also a very important segmentation algorithm in the medical image analysis literature. And it is solves the combinatorial directed problem. And it also uses the concept of shortest path forest. So as a result, what happens is the watershed algorithm has been improved quite a lot.

And the current versions are very robust and reliable. However, with the advent of deep learning based methods, watershed algorithms by themselves are not used a lot. But with the output of deep learning methods, such as for example, unit can be used as input to the watershed algorithm. So thank you for your attention.

Want the rest of Advanced Medical Image Processing in Clinical Applications?

Get the full course syllabus and fees.

The full course is paid (₹40,000). We'll send the syllabus and call you back within one working day.

Talk to a career counsellor

Free 20-minute call. We'll map the right course to where you want to work.

By submitting you agree to our Terms and Privacy Policy.

Full course · paid

Unlock the full Advanced Medical Image Processing in Clinical Applications course

The remaining 42 lessons, mentor-reviewed projects and a certificate come with the paid course (₹40,000). Leave your details: we'll send the syllabus and fees and call you back.

Get the syllabus and fees

A counsellor calls you back within one working day.

By submitting you agree to our Terms and Privacy Policy.

Grow by Skill-Lync · free lesson

Start watching

Free, no payment. Leave your details once and every free lesson on Grow plays straight away. A Skill-Lync counsellor may call you about the full Advanced Medical Image Processing in Clinical Applications course.

By submitting you agree to our Terms and Privacy Policy.

Talk to a career counsellor

Talk to a career counsellor

Free 20-minute call. We'll map the right course to where you want to work.

By submitting you agree to our Terms and Privacy Policy.