As per MSDN help file this is the difference:
Some objects support a method named Dispose whose purpose is to release system resources more expeditiously. Classes that support the Dispose method should implement the IDisposable interface. The Dispose method needs to be explicitly called when you want to release object resources. For example:
Another method supported by some classes, Finalize, runs automatically when an object is released and can be used to perform other cleanup tasks. The Finalize method is similar to the Class_Terminate() method used in previous versions of Microsoft Visual Basic. Unlike the Dispose method, the CLR automatically calls the Finalize method sometime after an object is no longer needed.