Well, this is a minor issue as the references are kept in WeakHashMaps. Indeed, the HibernateAnnotationScanner contains static weak maps of the Package/Classes of the current application. They are indexed by PersistenceUnitMetadata.
When the application gets undeployed the PersistenceUnitMetadata is no longer referenced, and should be garbage collected. However the value of the WeahHashMap won't be removed until the next expunging is done. As the application has been undeployed, it does not really make sense to keep strong references to the classes or packages. So, using Set<WeakReference<Package>> and Set<WeakReference<Class<?>>> would improve garbage collecting of the application's class loader and make more PermGen easily available.