Details
-
Bug
-
Resolution: Unresolved
-
Critical
-
None
-
8.0.0.GA-CR3
-
None
-
False
-
None
-
False
-
+
-
Known Issue
Description
Scenario: 4 nodes cluster where we deploy a JSF application; the cluster is hammered by 3000 clients (4 seconds between subsequent requests on each client) for approximately 11 hours;
The cluster is configured with a replicated-cache as in the following:
embed-server --server-config=standalone-ha.xml /subsystem=jgroups/channel=ee:write-attribute(name=stack,value=tcp) # replicates the cache configuration used in https://issues.redhat.com/browse/JBEAP-17682 /subsystem=infinispan/cache-container=web/replicated-cache=rrr:add() /subsystem=infinispan/cache-container=web:write-attribute(name=default-cache, value=rrr) /subsystem=infinispan/cache-container=web/replicated-cache=rrr/store=file:add()
At the end of the test, there is always one node (it happened on every other node of the cluster other than the coordinator) which cannot be shut-down;
When we try and shut it down manually we get a connection refused error:
[hudson@tborgato-clustering-s-2 jboss-eap-4]$ jps 50916 Jps 36886 jboss-modules.jar [hudson@tborgato-clustering-s-2 jboss-eap-4]$ ps -ef | grep 36886 hudson 36886 36714 20 Jan25 ? 03:22:37 /qa/tools/opt/x86_64/openjdk17_last//bin/java -D[Standalone] -Xlog:gc*:file=/tmp/tests-clustering/jboss-eap-4/standalone/log/gc.log:time,uptimemillis:filecount=5,filesize=3M -Djdk.serialFilter=maxbytes=10485760;maxdepth=128;maxarray=100000;maxrefs=300000 -Xms1303m -Xmx1303m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true --add-exports=java.desktop/sun.awt=ALL-UNNAMED --add-exports=java.naming/com.sun.jndi.ldap=ALL-UNNAMED --add-exports=java.naming/com.sun.jndi.url.ldap=ALL-UNNAMED --add-exports=java.naming/com.sun.jndi.url.ldaps=ALL-UNNAMED --add-exports=jdk.naming.dns/com.sun.jndi.dns=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.security=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.management/javax.management=ALL-UNNAMED --add-opens=java.naming/javax.naming=ALL-UNNAMED -Djava.security.manager=allow -Dorg.jboss.boot.log.file=/tmp/tests-clustering/jboss-eap-4/standalone/log/server.log -Dlogging.configuration=file:/tmp/tests-clustering/jboss-eap-4/standalone/configuration/logging.properties -jar /tmp/tests-clustering/jboss-eap-4/jboss-modules.jar -mp /tmp/tests-clustering/jboss-eap-4/modules org.jboss.as.standalone -Djboss.home.dir=/tmp/tests-clustering/jboss-eap-4 -Djboss.server.base.dir=/tmp/tests-clustering/jboss-eap-4/standalone -b=10.0.144.44 -bmanagement=10.0.144.44 -bprivate=10.0.144.44 --server-config=standalone-ha.xml -Djboss.default.multicast.address=230.0.0.138 -Dprogram.name=wildfly4 -Djboss.node.name=wildfly4 hudson 50944 50696 0 13:20 pts/0 00:00:00 grep --color=auto 36886 [hudson@tborgato-clustering-s-2 jboss-eap-4]$ ./bin/jboss-cli.sh You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands. [disconnected /] connect The controller is not available at localhost:9990: java.net.ConnectException: WFLYPRT0053: Could not connect to remote+http://localhost:9990. The connection failed: WFLYPR T0053: Could not connect to remote+http://localhost:9990. The connection failed: Connection refused [disconnected /] connect 10.0.144.44:9990 The controller is not available at 10.0.144.44:9990: java.net.ConnectException: WFLYPRT0053: Could not connect to remote+http://10.0.144.44:9990. The connection failed: WF LYPRT0053: Could not connect to remote+http://10.0.144.44:9990. The connection failed: Connection refused
Find attached the stack trace of the process in file jstack-36886.txt
;