HornetQ
  1. HornetQ
  2. HORNETQ-252

Connections not closed if client/server version mismatch

    Details

    • Type: Bug Bug
    • Status: Closed Closed (View Workflow)
    • Priority: Major Major
    • Resolution: Done
    • Affects Version/s: 2.0.0.CR2
    • Fix Version/s: 2.0.0.CR3
    • Component/s: None
    • Labels:
      None
    • Similar Issues:
      Show 10 results 

      Description

      If the client and server versions don't match (e.g. during an upgrade where the server is upgraded first) the server logs an error but then leaves the connection open instead of closing it. In our case we have clients which try to reconnect every 10 seconds which means every 10 seconds another connection is left open. We have about 30 clients so this means that within minutes we hit the "too many open files" limit on linux. Here is output from grepping number of connections to one of our clients over time:

      jms1:~# netstat | grep streamer12 | wc -l
      8
      jms1:~# netstat | grep streamer12 | wc -l
      10
      jms1:~# netstat | grep streamer12 | wc -l
      12
      jms1:~# netstat | grep streamer12 | wc -l
      14

      So you can see the number of connections just keeps going up and up.

      This is the strack trace we eventually see in the server:

      [New I/O server boss #1 (channelId: 782092838, /10.101.1.73:5445)] 10:00:39,072 WARNING [org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink] Failed to accept a connection.
      java.io.IOException: Too many open files
      at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
      at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:145)
      at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink$Boss.run(NioServerSocketPipelineSink.java:227)
      at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46)
      at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
      at java.lang.Thread.run(Thread.java:619)

      After which nothing can connect to the server anymore.

        Activity

        Hide
        Adrian Woodhead
        added a comment -

        Here is what we see in the client logs at the same time:

        [2009-12-16 10:29:22] ERROR (fm.last.jms.JMSProducer:48) - Error connecting PRODUCER for /topic/connectionControl connected
        javax.jms.JMSException: Server and client versions incompatible
        at org.hornetq.core.remoting.impl.ChannelImpl.sendBlocking(ChannelImpl.java:268)
        at org.hornetq.core.client.impl.FailoverManagerImpl.createSession(FailoverManagerImpl.java:387)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.createSessionInternal(ClientSessionFactoryImpl.java:1123)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.createSession(ClientSessionFactoryImpl.java:846)
        at org.hornetq.jms.client.HornetQConnection.authorize(HornetQConnection.java:544)
        at org.hornetq.jms.client.HornetQConnectionFactory.createConnectionInternal(HornetQConnectionFactory.java:600)
        at org.hornetq.jms.client.HornetQConnectionFactory.createConnection(HornetQConnectionFactory.java:114)
        at org.hornetq.jms.client.HornetQConnectionFactory.createConnection(HornetQConnectionFactory.java:109)
        at fm.last.jms.BaseJMSObject.connect(BaseJMSObject.java:89)
        at fm.last.jms.JMSProducer.connect(JMSProducer.java:41)
        at fm.last.jms.QueueingProducer.connect(QueueingProducer.java:79)
        at fm.last.jms.ConnectionMonitor.run(ConnectionMonitor.java:48)
        Caused by: HornetQException[errorCode=108 message=Server and client versions incompatible]
        ... 12 more
        [2009-12-16 10:29:22] WARN (fm.last.jms.ConnectionMonitor:53) - Connection down, trying again in 10000ms...
        [2009-12-16 10:29:23] INFO (fm.last.jms.JMSReceiver:26) - JMS RECEIVER for /topic/connectionControl connecting...
        [2009-12-16 10:29:23] ERROR (fm.last.jms.JMSReceiver:34) - Error connecting RECEIVER for /topic/connectionControl connected
        javax.jms.JMSException: Server and client versions incompatible
        at org.hornetq.core.remoting.impl.ChannelImpl.sendBlocking(ChannelImpl.java:268)
        at org.hornetq.core.client.impl.FailoverManagerImpl.createSession(FailoverManagerImpl.java:387)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.createSessionInternal(ClientSessionFactoryImpl.java:1123)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.createSession(ClientSessionFactoryImpl.java:846)
        at org.hornetq.jms.client.HornetQConnection.authorize(HornetQConnection.java:544)
        at org.hornetq.jms.client.HornetQConnectionFactory.createConnectionInternal(HornetQConnectionFactory.java:600)
        at org.hornetq.jms.client.HornetQConnectionFactory.createConnection(HornetQConnectionFactory.java:114)
        at org.hornetq.jms.client.HornetQConnectionFactory.createConnection(HornetQConnectionFactory.java:109)
        at fm.last.jms.BaseJMSObject.connect(BaseJMSObject.java:89)
        at fm.last.jms.JMSReceiver.connect(JMSReceiver.java:27)
        at fm.last.jms.ConnectionMonitor.run(ConnectionMonitor.java:48)
        Caused by: HornetQException[errorCode=108 message=Server and client versions incompatible]
        ... 11 more

        Show
        Adrian Woodhead
        added a comment - Here is what we see in the client logs at the same time: [2009-12-16 10:29:22] ERROR (fm.last.jms.JMSProducer:48) - Error connecting PRODUCER for /topic/connectionControl connected javax.jms.JMSException: Server and client versions incompatible at org.hornetq.core.remoting.impl.ChannelImpl.sendBlocking(ChannelImpl.java:268) at org.hornetq.core.client.impl.FailoverManagerImpl.createSession(FailoverManagerImpl.java:387) at org.hornetq.core.client.impl.ClientSessionFactoryImpl.createSessionInternal(ClientSessionFactoryImpl.java:1123) at org.hornetq.core.client.impl.ClientSessionFactoryImpl.createSession(ClientSessionFactoryImpl.java:846) at org.hornetq.jms.client.HornetQConnection.authorize(HornetQConnection.java:544) at org.hornetq.jms.client.HornetQConnectionFactory.createConnectionInternal(HornetQConnectionFactory.java:600) at org.hornetq.jms.client.HornetQConnectionFactory.createConnection(HornetQConnectionFactory.java:114) at org.hornetq.jms.client.HornetQConnectionFactory.createConnection(HornetQConnectionFactory.java:109) at fm.last.jms.BaseJMSObject.connect(BaseJMSObject.java:89) at fm.last.jms.JMSProducer.connect(JMSProducer.java:41) at fm.last.jms.QueueingProducer.connect(QueueingProducer.java:79) at fm.last.jms.ConnectionMonitor.run(ConnectionMonitor.java:48) Caused by: HornetQException [errorCode=108 message=Server and client versions incompatible] ... 12 more [2009-12-16 10:29:22] WARN (fm.last.jms.ConnectionMonitor:53) - Connection down, trying again in 10000ms... [2009-12-16 10:29:23] INFO (fm.last.jms.JMSReceiver:26) - JMS RECEIVER for /topic/connectionControl connecting... [2009-12-16 10:29:23] ERROR (fm.last.jms.JMSReceiver:34) - Error connecting RECEIVER for /topic/connectionControl connected javax.jms.JMSException: Server and client versions incompatible at org.hornetq.core.remoting.impl.ChannelImpl.sendBlocking(ChannelImpl.java:268) at org.hornetq.core.client.impl.FailoverManagerImpl.createSession(FailoverManagerImpl.java:387) at org.hornetq.core.client.impl.ClientSessionFactoryImpl.createSessionInternal(ClientSessionFactoryImpl.java:1123) at org.hornetq.core.client.impl.ClientSessionFactoryImpl.createSession(ClientSessionFactoryImpl.java:846) at org.hornetq.jms.client.HornetQConnection.authorize(HornetQConnection.java:544) at org.hornetq.jms.client.HornetQConnectionFactory.createConnectionInternal(HornetQConnectionFactory.java:600) at org.hornetq.jms.client.HornetQConnectionFactory.createConnection(HornetQConnectionFactory.java:114) at org.hornetq.jms.client.HornetQConnectionFactory.createConnection(HornetQConnectionFactory.java:109) at fm.last.jms.BaseJMSObject.connect(BaseJMSObject.java:89) at fm.last.jms.JMSReceiver.connect(JMSReceiver.java:27) at fm.last.jms.ConnectionMonitor.run(ConnectionMonitor.java:48) Caused by: HornetQException [errorCode=108 message=Server and client versions incompatible] ... 11 more
        Hide
        Jeff Mesnil
        added a comment -

        fixed in the trunk (see changelog)

        Show
        Jeff Mesnil
        added a comment - fixed in the trunk (see changelog)

          People

          • Assignee:
            Jeff Mesnil
            Reporter:
            Adrian Woodhead
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: