Uploaded image for project: 'JBoss Remoting'
  1. JBoss Remoting
  2. JBREM-1297

Deadlock between MicroRemoteClientInvoker.establishLease and Client.notifyListeners

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Critical
    • Resolution: Done
    • Affects Version/s: 2.2.4, 2.5.4.SP3
    • Fix Version/s: 2.2.4.SP1, 2.5.4.SP4
    • Component/s: None
    • Labels:
      None

      Description

      Deadlock is probably caused by changes introduced in this fix: JBREM-1242.

      Thread A:
      1. MicroRemoteClientInvoker.terminateLease, locks clientLeaseLock
      2. Client.notifyListeners, locks connectionValidatorLock

      Thread B:
      1. Client.setupClientLease, locks connectionValidatorLock
      2. MicroRemoteClientInvoker.establishLease, locks connectionValidatorLock

      Please check stacktraces when this deadlock occurred:

      "Timer-16" - Thread t@486453
         java.lang.Thread.State: BLOCKED
      	at org.jboss.remoting.Client.notifyListeners(Client.java:1873)
      	- waiting to lock <747c39d3> (a java.lang.Object) owned by "WorkManager(2)-241" t@6121
      	at org.jboss.remoting.LeasePinger.stopPing(LeasePinger.java:134)
      	at org.jboss.remoting.MicroRemoteClientInvoker.terminateLease(MicroRemoteClientInvoker.java:434)
      	- locked <751714e> (a java.lang.Object)
      	at org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask.run(ConnectionValidator.java:1081)
      	at java.util.TimerThread.mainLoop(Timer.java:512)
      	at java.util.TimerThread.run(Timer.java:462)
      

      and

      "WorkManager(2)-241" - Thread t@6121
         java.lang.Thread.State: BLOCKED
      	at org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:507)
      	- waiting to lock <751714e> (a java.lang.Object) owned by "Timer-16" t@486453
      	at org.jboss.remoting.Client.setupClientLease(Client.java:2056)
      	- locked <747c39d3> (a java.lang.Object)
      	at org.jboss.remoting.Client.connect(Client.java:1918)
      	at org.jboss.remoting.Client.connect(Client.java:737)
      

      One way to fix this would be to add lock on connectionValidatorLock in MicroRemoteClientInvoker.terminateLease. I'm trying to analyse code to check if this fix will not introduce another deadlock problem.

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  ron_sigal Ronald Sigal
                  Reporter:
                  artur.karpinski Artur Karpinski
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  3 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: