Uploaded image for project: 'JGroups'
  1. JGroups
  2. JGRP-1999

TCP_NIO2: single selector slows down writes and reads

    Details

    • Type: Enhancement
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 3.6.7
    • Labels:
      None

      Description

      TCP_NIO2 has a single selector which looks like this:

      x=select()
      if(read) -> read data from connection x, de-serialize data, pass to thread pool
      if(write) -> write data pending on connection x
      

      This means that any operation (read,write) delays other operations. It seems that especially the de-serialization done in reads delays other reads and writes.
      A quick test showed that having a reader thread per connection (so that reads don't delay other reads or writes) improved perf from 15'000 reqs/sec to 22'000.

      The idea is to have a reader thread in NioConnection which reads and de-serializes as many messages as possible. When no more messages are ready to be read, it blocks for a max wait time and then terminates unless more messages are ready. This means that idle connections will have no threads allocated.
      Investigate: we might possibly also null the pre-allocated buffer when a thread terminates, reducing memory usage even more.

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                belaban Bela Ban
                Reporter:
                belaban Bela Ban
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: