According to MSDN (regarding the GetEnumerator method)
"Items can be added to or removed from the cache while this method is enumerating through the items."
If so, seems to me this would work:
protected static void PurgeCacheItems(string prefix)
IDictionaryEnumerator enumerator = Cache.GetEnumerator();
However, like you said, it was always my understanding that deleting elements from a collection invalidates the enumerator, in which case an InvalidOperationException would be thrown. Hmm... have to try it now...
EDIT: Just tried it. seems to work fine.