c# has FileStream class and wrap OS file handle. FileStream has destructor, consider we get instance from FileStream and forget it, if there aren't any refrences to it, GC must call it's destructor and release OS file handle so why it will not happen untile myself call it's dispose or application terminated?
As noted in the comments, whenever GC begins to collect it will release the unreachable objects. However to avoid performance issue the GC does not start until one of these conditions happens. It's better not to wait for garbage collector and ourself quickly dispose our unmanaged resources.