While smoothing images, which color-space version (grayscale, RGB, HSV, etc.) should I apply filters like Gaussian and Bilateral to get the best noise removal results? Is there a general trend, or does it change in different cases?
Moreover, what filter&color-space would you suggest for shadow removal in image processing?
You should always apply filters to the RGB color space. A few other color spaces would make sense also, such as CIE-XYZ (which is just a rotation of the RGB color space) and CIE-Lab (which is a non-linear transformation of XYZ, but where Euclidean distances still make sense).
Color spaces such as HSV and similar have a component (hue) that is an angle. Here Euclidean distances don't make sense: averaging together 10 degrees and 350 degrees should lead to 0 degrees, but yields 180 degrees instead: you'll get all sorts of nonsensical colors when filtering.
For linear filters (such as the Gaussian) you can filter each RGB channel separately and independently. As long as the filter kernel applied to each channel is identical, the result will be correct.
For non-linear filters, however, filtering each channel independently will lead to false colors. For example, the bilateral filter needs to construct a kernel at each pixel. This same kernel must be applied to each of the channels at that pixel, to prevent false colors at edges of objects.