Status: Resolved (View Workflow)
Affects Version/s: 3.0.2.GA
Security Level: Public (Everyone can see)
Similar Issues:Show 10 results
JBCACHE-1149 FIFOPolicy is adding a visited node to the evictionqueue JBCACHE-96 Memory leak in FileCacheLoader JBCACHE-1274 Ability to configure zero nodes in memory for a region JBCACHE-1170 Leakage of marshalling region stored in ThreadLocal causes partial state transfer to fail when multiple regions present JBCACHE-1580 Type safety problem with dead member buddy backup region JBCACHE-1246 TransactionTable leaks memory when used with FAIL_SILENTLY option JBCACHE-1626 <N,09/11/12 08:09:32 UTC,RegionImpl,[Thread-165]> putNodeEvent(): eviction node event queue size is at 98% threshold value of capacity: 200000 Region: / You will need to reduce the wakeUpIntervalSeconds parameter. JBCACHE-1059 Memory leak in undo list JBCACHE-1454 Memory leak in CacheStoreInterceptor JBCACHE-1500 RegionManager.createRegion allows duplicate region names
There is a serious bug in EvictCommand.perform() which leads to cache regions declining in effective capacity over time. The syndrome is:
- Cache fails to fill - node count initially reaches around 90% of configured maxNodes capacity
- Cache capacity declines over time - in our case down to 70% of capacity after 36 hours.
In order to manifest this issue, you need to be removing as well as adding nodes to the cache. The bug is exposed by a race condition between the Client Code thread removing a node and the Eviction Thread processing the EvictionEventQueue, but the race condition is not the problem (the race condition just explains why you don't see this issue all the time), the problem is with the semantics of the boolean return value of EvictCommand.perform().
Failure Case: EvictCommand.perform() tries to evict a node X from the cache, when X is no longer in the cache:
- the EvictCommand.perform() method returns the boolean 'false', attempting to indicate: "the EvictCommand has done no work"
- the EvictionInterceptor interprets the return value of 'false' to mean: "only data has been removed from the node X, and the node still exists in the cache".
As a consequence the EvictionInterceptor creates a new ADD_NODE_EVENT for X, to ensure that X will be revisited by the eviction thread in future, and adds the event to theEvictionQueue.
The EvictionQueue now contains a NodeEntry for a Node which is not in the cache. The EvictionAlgorithm will evict nodes based on how many nodes it thinks are in the cache (the length of its EvictionQueue), rather than how many nodes are /actually/ in the cache, so the cache can not grow to it's full capacity. These dead NodeEntries build up over time and can take up a substantial portion of your cache if you're frequently performing node removals.
The fix for this issue is thankfully simple: As only EvictionInterceptor reads the boolean value returned by EvictCommand.perform(), the semantics of the return value should be aligned with those of the EvictionInterceptor, and should simply return whether the node STILL exists in the cache after the evict has been performed, as that is what the EvictionInterceptor is really interested in.