Uploaded image for project: 'JGroups'
  1. JGroups
  2. JGRP-2453

Change default TOS for UDP

XMLWordPrintable

    • Icon: Enhancement Enhancement
    • Resolution: Done
    • Icon: Major Major
    • 5.0.0.Alpha3, 4.2.0
    • 4.1.9, 4.0.22
    • None

      The Linux kernel has some fairly sophisticated queueing disciplines like fq_codel, but the default one is pfifo_fast.

      pfifo_fast is like three tc-pfifo(8) queues side by side,
      where packets can be enqueued in any of the three bands based on
      their Type of Service bits or assigned priority.

      Not all three bands are dequeued simultaneously - as long as lower
      bands have traffic, higher bands are never dequeued. This can be used
      to prioritize interactive traffic or penalize 'lowest cost' traffic.
      http://man7.org/linux/man-pages/man8/tc-pfifo_fast.8.html#ALGORITHM

      These are some examples of band mappings:

      TOS     Bits  Means                    Linux Priority    Band
      ------------------------------------------------------------
      0x0     0     Normal Service           0 Best Effort     1
      0x8     4     Maximize Throughput      2 Bulk            2
      0x10    8     Minimize Delay           6 Interactive     0
      0x18    12    mt+md                    4 Int. Bulk       1
      

      http://man7.org/linux/man-pages/man8/tc-prio.8.html#QDISC_PARAMETERS

      By default UDP.tos="8", which makes all UDP traffic go into band 2, lowest priority (bulk).
      FD_ALL and FD_ALL2 heartbeats are UDP traffic, and in some of our benchmarks there is enough
      client-server traffic in band 1 to delay the FD_ALL heartbeats for more than 10 seconds.

      We could either set the default TOS to 0 (best effort), or 0x18 (maximize throughput + minimize delay), the result is the same: band 1.

            rhn-engineering-bban Bela Ban
            dberinde@redhat.com Dan Berindei (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: