ConnectionTable: spurious Connection instances on concurrent connects

If we have members A and B, and they don't yet have connections to each other, and start sending messages to each other at the exact same time, then we run into the following scenario:

• A attempts to connect to B (new Socket())
• B attempts to connect to A
• A accepts B's connection and adds an entry for B into its table
• B adds an entry for A into its table
• B gets the accept() from A, but doesn't add the new Connection into its table because there is already a Connection. This leads to a spurious Connection object, which will only be closed and garbage collected once A or B leaves the cluster
• We used to close Connections which were already in the table, but this lead to ping-pong effects where concurrent initial senders always closed each others connections
• Even worse: a new Socket() always creates a new Connection object and places it into the table, regardless of whether a Connection for a given destination already exists or not !

GOAL:
#1 Have only 1 TCP connection between any 2 members
#2 Establish a handshaking mechanism, to avoid spurious connections

SOLUTION:

• When A establishes a connection to B, A needs to get agreement from B first (and vice versa)