Infinispan local cache error: Unable to invoke method public void org.infinispan.globalstate.impl.GlobalConfigurationManagerImpl.start()

Summary:
I have upgraded a Spring boot service to Infinispan 9.4.16.Final from 5.2.20.Final. The service had two XML files. Both have local-cache entries and no other types of caches. I used the conversion script to convert them.

One was left with an empty transport element by the conversion tool. That transport element seems to have been the cause of errors preventing the service from running. I believe the defect is that the config-converter tool added empty `transport` element.

Details:

When we deploy and run the service, we see this warning at startup:

org.infinispan.manager.EmbeddedCacheManagerStartupException: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.globalstate.impl.GlobalConfigurationManagerImpl.start() on object of type GlobalConfigurationManagerImpl

The above is the first warning/error we see. There is no stack trace. It shouldn't be calling GlobalConfigurationManagerImpl when we're only using local cache.

A few lines later in the log, then I see many The cache has been stopped and invocations are not allowed! errors. The last error we see is as follows. The service fails to start up successfully.

Caused by: org.infinispan.commons.CacheException: Initial state transfer timed out for cache org.infinispan.CONFIG on <server_name>

Interestingly, we started seeing log messages containing "JGroup" after the upgrade. Since we're using local cache, we don't expect to see that. (Example: "Unable to use any JGroups configuration mechanisms provided in properties {}. Using default JGroups configuration!")

The culprit was I believe the empty <transport /> element as shown below. This is one of our two XML config files:

<infinispan
   xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation = "urn:infinispan:config:9.4 http://www.infinispan.org/schemas/infinispan-config-9.4.xsd"
   xmlns = "urn:infinispan:config:9.4">

    <threads/>
    <cache-container name = "TestCenterServiceCache">
<!-- The conversion tool added this empty "transport" element. It was not present in our old config file -->
        <transport/>
        <jmx domain = "org.infinispan.TestCenterServiceCache"/>
        <local-cache name = "authorizedLocations">
            <expiration lifespan = "3600000"/>
        </local-cache>

        <!--caching for 24 hours: 3,600,000 milliseconds/hr x 24 hours -->
        <local-cache name = "proximitySearchConfiguration">
            <expiration lifespan = "86400000"/>
        </local-cache>
    </cache-container>
</infinispan>

Once I removed the empty <transport />, then the warnings, errors, and references to JGroup went away.

The original XML was:

<?xml version="1.0" encoding="UTF-8"?>
<infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="urn:infinispan:config:5.1 http://www.infinispan.org/schemas/infinispan-config-5.1.xsd"
	xmlns="urn:infinispan:config:5.1">

	<global>
		<globalJmxStatistics
			cacheManagerName="TestCenterServiceCache"
			jmxDomain="org.infinispan.TestCenterServiceCache"/>
	</global>

	<!--caching for 1 hour: 3600000 milli secs-->
	<namedCache name="authorizedLocations">
		<expiration lifespan="3600000"/>
	</namedCache>

	<!--caching for 24 hours: 3,600,000 milliseconds/hr x 24 hours -->
	<namedCache name="proximitySearchConfiguration">
		<expiration lifespan="86400000"/>
	</namedCache>

</infinispan>

In order to convert it, I updated the schema to:

<infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:infinispan:config:6.0 http://www.infinispan.org/schemas/infinispan-config-6.0.xsd"
xmlns="urn:infinispan:config:6.0">

before converting.

Note: When I removed the `global` element from the above before converting, then the empty `<transport />` element was not added.

Finally, as proof that the transport element was the culprit, the MBeans showed that under org.infinispan.TestCenterServiceCache -> Cache there was:

"org.infinispan.CONFIG(repl_sync)"

Without `<transport />`, there was:

"org.infinispan.CONFIG(local)"