Both statements are true.
The Finalize() method is essentially a destructor and like a destructor, is automatically invoked when the object is garbage-collected. With GC we are never sure when the object will be removed or when the exception will be thrown (if an exception existed in Finalize()).
The Dispose() method must be implemented by the coder via the System.IDisposable Interface. This method can be explicitly called by the clientâ€™s code when the object is no longer needed.
Both these methods are the last methods called before the object is removed or marked for GC. If an exception were to occur in either these methods then the exception would be discarded.
And yes; it is prefectly legitimate to throw exception from catch and finally blocks, that's what they're intended for.