JGroups
  1. JGroups
  2. JGRP-1594

UNICAST3: introduce connection establishment / teardown

    Details

    • Type: Feature Request Feature Request
    • Status: Resolved Resolved (View Workflow)
    • Priority: Major Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 3.3
    • Labels:
      None
    • Similar Issues:
      Show 10 results 

      Description

      [Use case: UNICAST_ConnectionTests.testBClosingUnilaterally()]

      There can be issues when simply sending messages with a conn-id, and the peer closing the connection, e.g.:

      • A and B have connections (send,receive) to each other
      • A sends 10 messages to B on conn-id=1
      • B accepts the 10 messages and passes them up
      • B closes its connection (but hasn't sent ack(10) yet)
        • Closing the connection, B removes the receive table for A
      • A hasn't received an ACK so it resends the 10 messages with conn-id=1
      • B creates a new connection for A and a new (empty) receive table
        • B now accepts the 10 retransmitted messages from A and delivers them

      B therefore delivers the 10 messages twice !

      SOLUTION #1:

      • When closing a connection, don't remove it immediately, but only mark is as closed and keep it around for a few minutes
      • A reaper then removes connections that have been marked as closed and have been idle for a few minutes
        • If messages are received for a closed connection on the same conn-id, remove the closed mark and treat the connection as open again
          • If A resent the 10 messages, B would still have the receive window, and know that it already received the 10 messages from A and discard them, preventing duplicate delivery

      SOLUTION #2:

      • Introduce explicit connection establishment and teardown, similar to TCP
      • When creating a connection to B, run a simplified SYN-ACK protocol, whereby both parties establish their sending and receiving conn-ids (perhaps also seqnos)
      • When closing a connection, run a simplified FIN-ACK protocol
        • This would prevent A from resendin the 10 messages, as the send table would have been removed
      • Connection creation and teardown should be time bounded, ie. if a connection cann be created to a peer after some attempts and timeout, throw an exception, which propagates back to the caller, If a connection cannot be closed (e.g. no ACK is received for the FIN), simply close it unilaterally

        Issue Links

          Activity

          Hide
          Bela Ban
          added a comment -

          This should include flushing pending (non-acked) messages when closing a connection

          Show
          Bela Ban
          added a comment - This should include flushing pending (non-acked) messages when closing a connection
          Hide
          Bela Ban
          added a comment -

          I picked solution #1:

          • When a connection is closed, it won't get removed immediately, but marked as CLOSING (from OPEN)
          • After conn_close_timeout ms of no access, the connection will be marked as CLOSED and removed
          • Message batching was also changed:
            • When a batch is received, we check if it contains messages with first=true
            • If true, we grab a new ReceiverEntry
            • Else we grab the existing ReceiverEntry
            • Then all messages of the batch are added to the receiver window in one operation (locks are acquired only once)
          Show
          Bela Ban
          added a comment - I picked solution #1: When a connection is closed, it won't get removed immediately, but marked as CLOSING (from OPEN) After conn_close_timeout ms of no access, the connection will be marked as CLOSED and removed Message batching was also changed: When a batch is received, we check if it contains messages with first=true If true, we grab a new ReceiverEntry Else we grab the existing ReceiverEntry Then all messages of the batch are added to the receiver window in one operation (locks are acquired only once)

            People

            • Assignee:
              Bela Ban
              Reporter:
              Bela Ban
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: