Uploaded image for project: 'JBoss Enterprise Application Platform'
  1. JBoss Enterprise Application Platform
  2. JBEAP-5295

(7.1.0) Endpoint can be closed before doConnect tasks finished causing AbstractHandleableCloseable.close to wait forever

    XMLWordPrintable

Details

    Description

      Endpoint can be closed before doConnect tasks finished causing AbstractHandleableCloseable.close to wait forever.

      Seen when application is making remote naming calls to a remote server.

      Such as this:

              Properties props = new Properties();
              props.put("java.naming.factory.initial", "org.jboss.naming.remote.client.InitialContextFactory");
              ...
              Context remoteContext = null; 
              Object object = null;
              try {
                 remoteContext = new InitialContext(props);
                 object = remoteContext.lookup(lookup);
              } finally {
                 remoteContext.close();
              }
              ...
      

      With cpu starvation under load, the timing of the connection start up tasks can be delayed. so JBoss received a connection before the endpoint close, but the full connection start (handleDone) didn't occur until after the endpoint close was initiated. This gets us into a state where the connection is able to come in, endpoint closes, then the connection finishes adding itself to the closed endpoint's connections map and will never be closed.

      EndpointImpl.doConnect

      • connHandlerFuture.getIoFuture().addNotifier(...
      • public void handleDone

      EndpointImpl.closeAction()

      https://github.com/jboss-remoting/jboss-remoting/blob/3.3.6.Final/src/main/java/org/jboss/remoting3/EndpointImpl.java#L277

      "Remoting "master:client-one:MANAGEMENT" task-3" prio=10 tid=0x00007f4a100fc800 nid=0x44b3 in Object.wait() [0x00007f4a68999000]
         java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method) 
        - waiting on <0x00000000fe87aee0> (a java.lang.Object)
        at java.lang.Object.wait(Object.java:485)
        at org.jboss.remoting3.spi.AbstractHandleableCloseable.close(AbstractHandleableCloseable.java:190)
        - locked <0x00000000fe87aee0> (a java.lang.Object)
        at org.xnio.IoUtils.safeClose(IoUtils.java:137)
        at org.jboss.remoting3.EndpointImpl$5.handleDone(EndpointImpl.java:317)
        at org.jboss.remoting3.EndpointImpl$5.handleDone(EndpointImpl.java:293)
        at org.xnio.IoFuture$HandlingNotifier.notify(IoFuture.java:212)
        at org.xnio.AbstractIoFuture$NotifierRunnable.run(AbstractIoFuture.java:729)
        at org.xnio.IoUtils$2.execute(IoUtils.java:72)
        at org.xnio.AbstractIoFuture.runNotifier(AbstractIoFuture.java:702)
        at org.xnio.AbstractIoFuture$NotifierState.doNotify(AbstractIoFuture.java:275)
        at org.xnio.AbstractIoFuture$NotifierState.notifyDone(AbstractIoFuture.java:256)
        at org.xnio.AbstractIoFuture.setResult(AbstractIoFuture.java:628)
        at org.xnio.FutureResult.setResult(FutureResult.java:83)
        at org.xnio.IoUtils$ResultNotifier.handleDone(IoUtils.java:684)
        at org.xnio.IoUtils$ResultNotifier.handleDone(IoUtils.java:673)
        at org.xnio.IoFuture$HandlingNotifier.notify(IoFuture.java:212)
        at org.xnio.AbstractIoFuture$NotifierRunnable.run(AbstractIoFuture.java:729)
        at org.xnio.IoUtils$2.execute(IoUtils.java:72)
        at org.xnio.AbstractIoFuture.runNotifier(AbstractIoFuture.java:702)
        at org.xnio.AbstractIoFuture$NotifierState.doNotify(AbstractIoFuture.java:275)
        at org.xnio.AbstractIoFuture$NotifierState.notifyDone(AbstractIoFuture.java:256)
        at org.xnio.AbstractIoFuture$NotifierState.notifyDone(AbstractIoFuture.java:257)
        at org.xnio.AbstractIoFuture$CancellableState.notifyDone(AbstractIoFuture.java:334)
        at org.xnio.AbstractIoFuture.setResult(AbstractIoFuture.java:628)
        at org.xnio.FutureResult.setResult(FutureResult.java:83)
        at org.jboss.remoting3.remote.ClientConnectionOpenListener$Authentication$2.run(ClientConnectionOpenListener.java:763)
        at org.jboss.remoting3.EndpointImpl$TrackingExecutor$1.run(EndpointImpl.java:731)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
        at java.lang.Thread.run(Thread.java:662)
      

      Attachments

        Issue Links

          Activity

            People

              elguardian@gmail.com Enrique González Martínez (Inactive)
              rhn-cservice-bbaranow Bartosz Baranowski
              Richard Janik Richard Janik
              Richard Janik Richard Janik
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: