Refactor CacheLoader API to make it thread safe

Introduction

Currently CacheLoader thread safety can be easily compromised by broken implementations. We should provide a better implementation to make it more convenient and less error prone.

Problem statement

CacheLoader API looks like this:

public interface AdvancedCacheLoader<K, V> extends CacheLoader<K, V> {
...
   void process(KeyFilter<? super K> filter, CacheLoaderTask<K, V> task, Executor executor, boolean fetchValue, boolean fetchMetadata);
...
}

The CacheLoader implementation controls CacheLoaderTask as well as the number of threads used for processing (specified by Executor). If one wants to collect all keys (similarly to AdvancedCacheLoader) it is very easy to hit concurrently issues while adding them to the set (See previous JIRAs: ISPN-3938 and ISPN-6276).

Solution

The solution should take away currently control (Executor) and keyset (the simplest idea is to add it to CacheLoaderTask) from from the implementation. One of the ideas is to create a Lambda expression for pulling records from Cache Store. Note that similar changes will need to be performed to CacheWriter (to make it similar).