I am currently writing a segmentation algorithm in Matlab based on the distance transform and watershed lines to analyse my data (Cell Microscopy, see picture 1 for the raw image). The part of separating cells works quite well, but I have troubles with the first part of the algorithm : the preprocessing and the edges detection parts.
Link towards the images : http://postimg.org/gallery/20pf5za9q/be769288/
Indeed, my segmentation is based on bright field images with natural contours quite well highlighted, but due to some noise illumination from nucleus, the contour of some cells if often truncated when I apply the edges detection method (Sobel Filter). For instance, the edges detected in image 2 lead to a good segmentation (image 3), but the next frame could have the same cell with an intern illumination leading to a bad edges detection (image 4) and thus a bad segmentation (image 5).
I would like to find either a edges detection method more sensitive, or a method allowing to diminish the importance of the cells intern illumination. As I am far to be an expert in Image processing algorithms, I do not know if what I want can be simple to get, or If I need to look at other strategies.
Thank you very much for your time and advices.
You could also "play" with the different settings and thresholds in the edge method in Matlab. I had the same problem a while ago and it was solved choosing the "log' method (Laplassian of Gaussian).
You could begin by sticking to the Sobel method and changing the threshold (see http://www.mathworks.com/help/images/ref/edge.html?refresh=true for details), then proceed to experimenting with the other methods to find if any of them suits you better than the sobel. I would also check matlab's boundary tracing function ( http://www.mathworks.com/help/images/ref/bwboundaries.html ) perhaps with noholes enabled (I think this would eliminate the nucleus area appearing as an extra object and messing your segmentation). Hope this helps a bit. :D