Yes, this is an example of a memory leak.
Unfortunately, the exact behavior depends on the kind of resource that is leaking. Some objects (I think including some graphics objects) are limited on a system-wide basis, so if the program uses them all up, other programs will also suffer. That may cause weird behavior for this program and others.
I think that these sorts of leaks usually recover after the leaking program ends, but there are cases where the system could be left in a strange state. The most common example is when a program dies but leaves a file locked.
Some leaks can also mess up the system badly enough that it's hard to kill the leaking program.
The moral is, if you're using unmanaged resources, you should test with a lot of allocations and deallocations to see if there is a leak before you release the program into the wild.