Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-9913

Balancer fails to balance requests according to load provided by custom load metric

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • 12.0.0.Beta1
    • mod_cluster, Web (Undertow)
    • None

      When balancer uses load provided by custom load metric, it intermittently fails to redirect requests to expected workers. We use following scenario in our test with EAP 7.1.1 nodes:

      1. Prepare one balancer and three workers with custom load metric which has these attribute values:
        history: 0, decay: 0, capacity: 1000 and uses implementation from [1].
      2. The implementation takes dummy load values from files in local filesystem. So the values in files are set then.
      3. The verification, that expected load factor is loaded is made by running "/subsystem=undertow/configuration=filter/mod-cluster=modcluster:read-resource(include-runtime=true, recursive=true, recursive-depth=100).
      4. 1000+ requests is made on balancer and for each request, the JVM route of handling node is noted.
      5. It is verified, that expected amount of request went to each node according to load factor.
      6. Load values in files are changes and verification, that they were loaded is made again.
      7. 1000+ requests is made on balancer and for each request, the JVM route of handling node is noted.
      8. After the requests were made, the verification, that all went according to expectations is made. In this step, the verification sometimes fails, because there were unexpected count of requests on each node.

      We suspect, that this is an error in mod_cluster implementation. The problem is, that sometimes more requests than expected ends up on the worker with lowest load:
      java.lang.AssertionError: Assert #5, Request distribution according to the load was supposed to be [jboss-eap-7.2-1:106, jboss-eap-7.2-2:106, jboss-eap-7.2-3:789] with tolerance +/-10, but was: [jboss-eap-7.2-1:36, jboss-eap-7.2-2:35, jboss-eap-7.2-3:930]

      [1]: https://github.com/Karm/mod_cluster-custom-load-metric

            rhn-engineering-rhusar Radoslav Husar
            jkasik@redhat.com Jan Kašík
            Jan Kašík Jan Kašík
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: