Details
-
Bug
-
Resolution: Cannot Reproduce
-
Critical
-
2.0.8.Final
-
None
-
-
User Experience
-
Medium
Description
When executing management commands using jboss-cli.sh against the domain controller of a cluster repeatedly the host controller uses up more and more memory in oldgen. After several thousands of runs of jboss-cli the host controller eventually becomes unresponsive (see attached picture for memory consumption, dc became entirely unresponsive at roughly 6:30am):
[root@dc broken]# /opt/wildfly-10.0.0.Final-DC/bin/./jboss-cli.sh --connect --user="username" --password="password" --command=":read-children-names(child-type=host)"
Failed to connect to the controller: The controller is not available at xx.xx.xx.xx:9993: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://xx.xx.xx.xx:9993. The connection timed out: WFLYPRT0023: Could not connect to https-remoting://xx.xx.xx.xx:9993. The connection timed out
I discovered the issue when testing whether https://issues.jboss.org/browse/WFCORE-974 was actually resolved in wildfly-10.0.0.Final as advertised. I can confirm that the issue is different, since no OOM-Exceptions are thrown. However the DC still becomes useless, since it won't accept any connections anymore. I will check whether the work-around from However the work-around from WFCORE-974 applies to this issue as well.WFCORE-974 doesn't fix this issue.
Please note that the attached logs are UTC, while the monitoring is UTC+2. Also the collection values are misleading since I haven't adapted my monitoring to the new output of jstat in JDK8. PU and PC are thus MU and MC.