Injection into finally block wrapping a synchronized block may cause verify error

Injecting into a finally block which wraps a synchronized block which includes a return can lead to a verify error. This happens because the exception flow analysis does not correctly identify shadowing relationships between nested catch-all (null type) exception handlers.

Exceptions occuring within a synchronized block that are not caught by catch handlers defined by the user are directed to a catch-all handler which executes a monitorexit and then rethrows the exception (the javac compiler inserts this handler automatically). If the synchronized block is wrapped in a try finally the then injected handler is protected with two nested try regions. The inner try region ends at the monitorexit and redirects control back to the start of the handler in order to retry the failed monitorexit. The outer one ends at the rethrow and redirects control to the finally code. Both these try-catch regions specify catch-all (null type) handlers.

This implies that the outer handler is shadowed by the inner one i.e. the outer handler cannot be reached from code protected by the inner handler. It can only be reached from code which succeeds the monitorenter. hat also means that the outer try-catch region can only pass control to the finally block after the monitorenter has executed. So, the outer handler cannot be entered with a monitorexit still pending.

Byteman checks overlapping try-catch regions for shadowing relationships precisely so it can identify whether a given exception flow may leave a monitorexit still pending. This is necessary because catch handlers for code injected by Byteman will close any open monitors before implementing a RETURN or THROW requested by rule code.

The bug arises because Byteman is only checking try-catch regions starting at some point in the bytecode to see if they overlapped existing try-catch regions with a start offset preceded the current bytecode position. It omits to compare new try-catch regions against other try-catch regions starting at the same offset. For the case described above this means it propagates an open monitorenter to the finally handler code and then tries to correct by inserting a spurious monitorexit. The verifier recognises that this spurious monitorexit is trying to unlock an object which has already been unlocked.