While I don't see a case where this would happen in practice we should guard against it.

The issue is that we create a new temp group for the with clause, but then reuse the old metadata ids where referenced. The projection logic works off of equality, so it needs the references to be updated or the original to stay in use.