Ok I've read lots of articles over the net speaking about the finalize() method and the fact that a we cannot really rely on it...
Right, it's clear (I guess)
But when memory leaks occur, and before going deeper and deeper inside VisualVM, can I just use the finalize method as a primary debugging tool ?
(A tiny Log message like "object released" in finalize and a "object created" in the constructor)
You want to trace object being retained, not those being released (which is what finalize shows you). More importantly, you want to know why objects are being retained, i.e. the path of references which are holding onto them. I suggest using VisualVM as a start, and possibly a commercial profiler if the heap is large or the problem is obscure.
I have use finalize() for logging when a resource has not been cleaned up deterministically. i.e. it should have been close()d by was not.