Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-9397

Once hitting the max-connections limit, new connections are not accepted any more and CLOSE_WAIT connections increase gradually

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • 12.0.0.Final
    • 11.0.0.CR1
    • Web (Undertow)
    • None
    • Hide
      1. Set max-connections attribute in listener and Start WildFly: 
        <http-listener name="default" socket-binding="http" redirect-socket="https" enable-http2="true" max-connections="10"/>
      2. Sent concurrent requests which can overflow the max-connections limit: 
        ab -c 100 -n 100 http://localhost:8080/

      You will see most requests will be failed with "The timeout specified has expired". Once the above command hit the max-connections limit, WildFly does not accept new connection any more even after all request processing are completed.

      • Try curl command to confirm you can not connect: 
        curl -v http://localhost:8080/
      • Check an output of netstat. CLOSE_WAIT will be increased one by one when you execute curl command and cancel it: 
        netstat -tunapl | grep 8080
      Show
      Set max-connections attribute in listener and Start WildFly: <http-listener name= " default " socket-binding= "http" redirect-socket= "https" enable-http2= " true " max-connections= "10" /> Sent concurrent requests which can overflow the max-connections limit: ab -c 100 -n 100 http: //localhost:8080/ You will see most requests will be failed with "The timeout specified has expired". Once the above command hit the max-connections limit, WildFly does not accept new connection any more even after all request processing are completed. Try curl command to confirm you can not connect: curl -v http: //localhost:8080/ Check an output of netstat . CLOSE_WAIT will be increased one by one when you execute curl command and cancel it: netstat -tunapl | grep 8080

    Description

      Once hitting the max-connections limit of listener, WildFly does not accept new connections any more. Even after all request processing are completed, WildFly does not accept. So, any client can not connect to the server.

      In addition, when a client send a new request and close the connection from client-side due to timeout or cancellation in such situation, CLOSE_WAIT connections remain until WildFly Java process is stopped.

      I think this issue is related to XNIO-276 and XNIO-279, which both fixes are merged in XNIO 3.4 branch but it's not merged (or partially merged) in XNIO 3.5 branch:

      • XNIO-276 - If QueuedNioTcpServer hits the max connection limit it gets stuck in an infinite loop
        This issue is fixed by the two commits 6a5d04e and 8c0af87 which were merged in 3.4 branch. However, 8c0af87 is not included in 3.5 branch.
      • XNIO-279 - QueuedNioTcpServer can go into an infinite loop if accept fails
        This issue is fixed by the commit 86d0f6c in 3.4 branch. However, this is not included in 3.5 branch.

      Attachments

        Issue Links

          clones

          Bug - A problem that impairs or prevents the functions of the product. JBEAP-13272 [GSS](7.1.0) Once hitting the max-connections limit, new connections are not accepted any more and CLOSE_WAIT connections increase gradually

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Verified
          is related to

          Bug - A problem that impairs or prevents the functions of the product. XNIO-276 If QueuedNioTcpServer hits the max connection limit it gets stuck in an infinite loop

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Resolved

          Bug - A problem that impairs or prevents the functions of the product. XNIO-279 QueuedNioTcpServer can go into an infinite loop if accept fails

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Resolved

          Activity

            People

              dlloyd@redhat.com David Lloyd
              rhn-support-mmiura Masafumi Miura
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: