When you rename a runtime but the server doesn't know about the change yet and you right-click on the server in server view, you will get a NPE.
Perhaps the affected servers can be notified when the runtime is renamed and their runtime settings can be modified automatically? Or at least the error should be comprehensive and say what's wrong.