+++ This bug was initially created as a clone of Bug #1323310 +++

Description of problem:

This issue relates to BRMS Integration with Fabric and more specifically to how Drools is loading KJARs from Maven.

In this case, drools does not use the fabric maven proxy or any of the configured maven repo's in fabric to load a KJAR. It uses 3 potential methods;

1) a drools CUSTOM_SETTINGS_PROPERTY pointing to a settings.xml
2) the 'userHome + "/.m2/settings.xml"'
3) the artifact is in 'userHome + "/.m2/repository"'

This is because the KJAR is loaded by KIE specific logic and not as bundle loaded through a feature or profile. Because fabric customers manage resources and configurations through the fabric, this represents additional configuration that is hard to track and maintain.

A KJAR is current loaded as follows;

1) InternalKieModule kieModule = (InternalKieModule) getRepository().getKieModule(releaseId);
https://github.com/droolsjbpm/drools/blob/6.3.x/drools-compiler/src/main/java/org/drools/compiler/kie/builder/impl/KieServicesImpl.java#L114

2) return getKieModule(releaseId, null);
https://github.com/droolsjbpm/drools/blob/6.3.x/drools-compiler/src/main/java/org/drools/compiler/kie/builder/impl/KieRepositoryImpl.java#L92

3) kieModule = loadKieModuleFromMavenRepo(releaseId, pomModel);
https://github.com/droolsjbpm/drools/blob/6.3.x/drools-compiler/src/main/java/org/drools/compiler/kie/builder/impl/KieRepositoryImpl.java#L105

4) return getInternalKieScanner().loadArtifact(releaseId, pomModel);
https://github.com/droolsjbpm/drools/blob/6.3.x/drools-compiler/src/main/java/org/drools/compiler/kie/builder/impl/KieRepositoryImpl.java#L129

5) return internalKieScanner; -> InternalKieScanner
https://github.com/droolsjbpm/drools/blob/6.3.x/drools-compiler/src/main/java/org/drools/compiler/kie/builder/impl/KieRepositoryImpl.java#L133

6) KieRepositoryScannerImpl implements InternalKieScanner
https://github.com/droolsjbpm/drools/blob/6.3.x/kie-ci/src/main/java/org/kie/scanner/KieRepositoryScannerImpl.java

7) getArtifactResolver();
https://github.com/droolsjbpm/drools/blob/6.3.x/kie-ci/src/main/java/org/kie/scanner/KieRepositoryScannerImpl.java#L118

8) mavenRepository = MavenRepository.getMavenRepository();
https://github.com/droolsjbpm/drools/blob/6.3.x/kie-ci/src/main/java/org/kie/scanner/ArtifactResolver.java#L52

9) defaultMavenRepository = new MavenRepository( defaultAether );
https://github.com/droolsjbpm/drools/blob/6.3.x/kie-ci/src/main/java/org/kie/scanner/MavenRepository.java#L78

10) remoteRepositoriesForRequest = initRemoteRepositoriesForRequest();
https://github.com/droolsjbpm/drools/blob/6.3.x/kie-ci/src/main/java/org/kie/scanner/MavenRepository.java#L67

11) final MavenRepositoryConfiguration repositoryUtils = getMavenRepositoryConfiguration();
https://github.com/droolsjbpm/drools/blob/6.3.x/kie-ci/src/main/java/org/kie/scanner/MavenRepository.java#L87

12) return MavenSettings.getMavenRepositoryConfiguration();
https://github.com/droolsjbpm/drools/blob/6.3.x/kie-ci/src/main/java/org/kie/scanner/MavenRepository.java#L70

13) return SettingsHolder.mavenConf;
https://github.com/droolsjbpm/drools/blob/6.3.x/kie-ci/src/main/java/org/kie/scanner/embedder/MavenSettings.java#L49

14) private static final MavenRepositoryConfiguration mavenConf = new MavenRepositoryConfiguration(settings);
https://github.com/droolsjbpm/drools/blob/6.3.x/kie-ci/src/main/java/org/kie/scanner/embedder/MavenSettings.java#L35

In terms of artifact resolution, everything happens based on the configured maven settings above in step 14.

For USER settings, it's looking for: private static File initUserSettingsFile()

1) CUSTOM_SETTINGS_PROPERTY
or
2) File userSettingsFile = new File( userHome + "/.m2/settings.xml" );

Otherwise it returns null.

Then it initializes settings: private static Settings initSettings(File userSettingsFile)

1) Is user settings wasn't null;
request.setUserSettingsFile( userSettingsFile );

2) Look for M2_HOME - set global settings
File globalSettingsFile = new File( mavenHome + "/conf/settings.xml" );

3) If user.home is set, sets the local repository:
settings.setLocalRepository( userHome + "/.m2/repository" );

Finlly;

15) All of the repos from the found settings are initialized:

this.extraRepositories = initExtraRepositories();
this.remoteRepositoriesForRequest = initRemoteRepositoriesForRequest();
this.artifactRepositoriesForRequest = initArtifactRepositories();

https://github.com/droolsjbpm/drools/blob/6.3.x/kie-ci/src/main/java/org/kie/scanner/MavenRepositoryConfiguration.java#L43

Version-Release number of selected component (if applicable):

BRMS 6.2.x
BRMS 6.3

How reproducible:

100%

Steps to Reproduce:
1. Deploy an application which loads a KJAR and only configure the maven repository or settings.xml in fabric.

Actual results:

KJAR artifact is unable to be found.

Expected results:

Should resolve and download the correct artifact.

Additional info:

In the customers eyes, in a fabric environment, drools should plug into fabric and leverage the already configured remote / local repositories configured within the environment.

— Additional comment from JBoss Product and Program Management on 2016-04-01 16:40:07 EDT —

Since this issue was entered in Red Hat Bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.

— Additional comment from Edson Tirelli on 2016-04-05 19:48:47 EDT —

Alessandro, we don't have official support for Fabric, at least from an engineering point of view. Not sure what we are supposed to do here?

Mario, meanwhile, if you can take a look please and identify if this is something that can be fixed in kie-ci?

— Additional comment from Mario Fusco on 2016-04-06 03:29:29 EDT —

Edson, I gave a look at Fabric documentation and found this http://fabric8.io/gitbook/mavenProxy.html so I guess I should figure out how to access and use this Maven proxy provided by Fabric from kie-ci, but I haven't been able to find anything about this so far.

Matthew, your description of how kie-ci currently works is correct, but I haven't understood how to integrate what we have at the moment with the Fabric Maven proxy. Do you have any hint about this?

Also consider that I don't want kie-ci to depend from any specific Fabric library/API, so if this will be necessary I'll have to develop this integration in a separated module. The users will have to also add this new kie-ci-fabric dependency when trying to use kie-ci from inside Fabric.

— Additional comment from Matthew Robson on 2016-04-06 09:27:39 EDT —

Edson, I don't think you can make that distinction because fabric8 is just the integration platform that ships with Fuse. This issue (and the other BZ 1323167) are coming from Cigna where there is a massive investment in Fuse infrastructure with fabric and there are many groups developing and migrating BxMS applications with the plan of moving into this infrastructure.

Mario,

The most complete answer on the best way to do this would come from Fuse Engineering, but a few of the ideas are:

1) Use a MVN URL Handler http://fabric8.io/gitbook/urlHandlers.html#profile which should delegate to the Fabric Maven Proxy to load the artifact.

2) Read the default, local and remote repository URL list from fabric when attempting to load a KJAR.

3) Attach the settings.xml to a profile and reference it via a ZooKeeper URL

• This also does not work once again because java.io.File presumably does not understand how to handle the zk: URI scheme.

4) Attach the settings.xml as a resource to the profile and refer to it with a relative file path

• This was at the request of their team because they claimed to have been able to do it with something else, and mentioned another support ticket they filed that outlined this. I could not find that ticket. According the docs I dug up here (https://access.redhat.com/documentation/en-US/Red_Hat_JBoss_Fuse/6.0/html/Getting_Started/files/ProfileEdit-OtherResources.html) the file is actually stored in ZooKeeper, so no local copy of it would be made on either the root or the target container

5) Distribute the settings.xml as a bundle and instruct the profile to load it using a bundleresource: URI scheme

• This does not work because in MavenSettings.java in BRMS you can see that it interprets the value of kie.maven.settings.custom as a regular java File path, which does not know how to handle bundleresource: and thus reverts back to using the default

— Additional comment from Mario Fusco on 2016-04-06 11:55:59 EDT —

Matthew,

thanks for your clarifications.

If I'm understanding correctly the only limitation of kie-ci preventing it to be seamless integrated with Fabric is in the fact that it only accepts a File and not a generic URL as user's custom settings. If so I'm giving a look at maven/aether API and I think that this problem could be easily fixed.

I'll send a pull request to allow this asap. I believe this would be a useful improvement also regardless of Fabric integration. Please let me know if you think that this would be enough or I'll need to perform any further action regarding this issue.

Thanks again,
Mario

— Additional comment from Matthew Robson on 2016-04-06 13:37:39 EDT —

Yes, that is possibly the only limitation, but I think we should double check with Fuse engineering as well.

Paolo, could you provide from input from the fabric8 point of view?

Matt

— Additional comment from Mario Fusco on 2016-04-06 13:53:56 EDT —

Ciao Paolo,

in case you'll have time to give this a try I already sent a pull request with the change I suggested in my former comment https://github.com/droolsjbpm/drools/pull/719

You could probably apply this PR to drools' master, recompile it and check if it is enough to cover this use case.

Let me know if I can help you with this or you need any further clarifications.

Thanks,
Mario

— Additional comment from Mario Fusco on 2016-04-07 03:04:56 EDT —

Paolo,

I talked about the fix I proposed with my pull request with Edson and we decided that it will be a good improvement also regardless of the issue reported here. For this reason I just merged that PR into master and so you could give this a try by just recompiling our master branch.

At the moment this is also the only limitation of which I'm aware about Fabric and kie-ci integration, so I'm also flagging this ticket as MODIFIED. In case you'll find that something else is missing feel free to set this back to ASSIGNED and clarify which further action is required on my side.

— Additional comment from Matthew Robson on 2016-04-07 12:49:50 EDT —

Mario,

I cherry-picked the commit into 6.3.x and rebuilt. I swapped out the kie-ci-osgi artifact from BRMS 6.2 for the newly built one.

On my Fuse server, I set -Dkie.maven.settings.custom=profile:settings.xml

This should use the fabric URL Handler to load the file, settings.xml, from the current profile or from any of the profiles in it's hierarchy.

First deploy with no settings.xml created, you see an IllegalArgumentException which is good and descriptive.

Caused by: java.lang.IllegalArgumentException: Resource settings.xml does not exist in the profile overlay.

I created a settings.xml on my drools profile with a basic local repo:

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
<localRepository>/mnt/fabric</localRepository>
</settings>

That resulted in:

2016-04-07 12:22:50,439 | INFO | agent-1-thread-1 | Aether | org.kie.scanner.Aether 124 | 644 - org.kie.scanner.osgi - 6.3.0.201602012307 | The local repository directory /mnt/fabric doesn't exist. Creating it.

That created an empty directory structure:

/mnt/fabric/com/cigna/jurac/new-jurac-rules/0.0.1-SNAPSHOT/

I installed the KJAR to the local repo and the KJAR was loaded (albeit with some issues). It started showing a WARN about 'org.kie:kie-ci:6.3.0.Final-redhat-7' followed by a NPE 'org.kie.scanner.KieRepositoryScannerImpl.indexAtifacts'. It could be simply due to the fact that I swapped out the kie-ci-osgi JAR only which is now 6.3.1...

2016-04-07 11:52:24,524 | WARN | agent-1-thread-1 | MavenRepository | org.kie.scanner.MavenRepository 161 | 617 - org.kie.scanner.osgi - 6.3.1.201604071253 | Unable to resolve artifact: org.kie:kie-ci:6.3.0.Final-redhat-7
2016-04-07 11:52:24,589 | WARN | agent-1-thread-1 | MavenRepository | org.kie.scanner.MavenRepository 161 | 617 - org.kie.scanner.osgi - 6.3.1.201604071253 | Unable to resolve artifact: org.kie:kie-ci:pom:6.3.0.Final-redhat-7
2016-04-07 11:52:24,663 | WARN | agent-1-thread-1 | MavenRepository | org.kie.scanner.MavenRepository 161 | 617 - org.kie.scanner.osgi - 6.3.1.201604071253 | Unable to resolve artifact: org.kie:kie-ci:6.3.0.Final-redhat-7
2016-04-07 11:52:25,919 | ERROR | agent-1-thread-1 | AbstractKieModule | e.builder.impl.AbstractKieModule 247 | 613 - org.drools.compiler - 6.3.0.201602012307 | Unable to build KieBaseModel:jurac-account

2016-04-07 11:52:25,929 | INFO | agent-1-thread-1 | MavenEmbedderUtils | nner.embedder.MavenEmbedderUtils 52 | 617 - org.kie.scanner.osgi - 6.3.1.201604071253 | In OSGi: using programmatically wired maven parser
2016-04-07 11:52:27,777 | WARN | agent-1-thread-1 | MavenRepository | org.kie.scanner.MavenRepository 161 | 617 - org.kie.scanner.osgi - 6.3.1.201604071253 | Unable to resolve artifact: org.kie:kie-ci:6.3.0.Final-redhat-7
2016-04-07 11:52:27,782 | WARN | agent-1-thread-1 | BeanRecipe | s.blueprint.container.BeanRecipe 851 | 8 - org.apache.aries.blueprint.core - 1.0.1.redhat-611423 | Object to be destroyed is not an instance of UnwrapperedBeanHolder, type: null
2016-04-07 11:52:27,785 | ERROR | agent-1-thread-1 | BlueprintContainerImpl | container.BlueprintContainerImpl 398 | 8 - org.apache.aries.blueprint.core - 1.0.1.redhat-611423 | Unable to start blueprint container for bundle new-jurac-brms-service
org.osgi.service.blueprint.container.ComponentDefinitionException: Error when instantiating bean kieSessionService of class class com.redhat.drools.camel.beans.KieScannerKieSessionService
at org.apache.aries.blueprint.container.BeanRecipe.getInstance(BeanRecipe.java:333)[8:org.apache.aries.blueprint.core:1.0.1.redhat-611423]
at org.apache.aries.blueprint.container.BeanRecipe.internalCreate2(BeanRecipe.java:806)[8:org.apache.aries.blueprint.core:1.0.1.redhat-611423]
at org.apache.aries.blueprint.container.BeanRecipe.internalCreate(BeanRecipe.java:787)[8:org.apache.aries.blueprint.core:1.0.1.redhat-611423]
at org.apache.aries.blueprint.di.AbstractRecipe$1.call(AbstractRecipe.java:79)[8:org.apache.aries.blueprint.core:1.0.1.redhat-611423]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)[:1.7.0_60]
at org.apache.aries.blueprint.di.AbstractRecipe.create(AbstractRecipe.java:88)[8:org.apache.aries.blueprint.core:1.0.1.redhat-611423]
at org.apache.aries.blueprint.container.BlueprintRepository.createInstances(BlueprintRepository.java:245)[8:org.apache.aries.blueprint.core:1.0.1.redhat-611423]
at org.apache.aries.blueprint.container.BlueprintRepository.createAll(BlueprintRepository.java:183)[8:org.apache.aries.blueprint.core:1.0.1.redhat-611423]
at org.apache.aries.blueprint.container.BlueprintContainerImpl.instantiateEagerComponents(BlueprintContainerImpl.java:676)[8:org.apache.aries.blueprint.core:1.0.1.redhat-611423]
at org.apache.aries.blueprint.container.BlueprintContainerImpl.doRun(BlueprintContainerImpl.java:374)[8:org.apache.aries.blueprint.core:1.0.1.redhat-611423]
at org.apache.aries.blueprint.container.BlueprintContainerImpl.run(BlueprintContainerImpl.java:261)[8:org.apache.aries.blueprint.core:1.0.1.redhat-611423]
at org.apache.aries.blueprint.container.BlueprintExtender.createContainer(BlueprintExtender.java:270)[8:org.apache.aries.blueprint.core:1.0.1.redhat-611423]
at org.apache.aries.blueprint.container.BlueprintExtender.modifiedBundle(BlueprintExtender.java:233)[8:org.apache.aries.blueprint.core:1.0.1.redhat-611423]
at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$Tracked.customizerModified(BundleHookBundleTracker.java:500)[10:org.apache.aries.util:1.0.1.redhat-611423]
at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$Tracked.customizerModified(BundleHookBundleTracker.java:433)[10:org.apache.aries.util:1.0.1.redhat-611423]
at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$AbstractTracked.track(BundleHookBundleTracker.java:725)[10:org.apache.aries.util:1.0.1.redhat-611423]
at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$Tracked.bundleChanged(BundleHookBundleTracker.java:463)[10:org.apache.aries.util:1.0.1.redhat-611423]
at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$BundleEventHook.event(BundleHookBundleTracker.java:422)[10:org.apache.aries.util:1.0.1.redhat-611423]
at org.apache.felix.framework.util.SecureAction.invokeBundleEventHook(SecureAction.java:1103)[org.apache.felix.framework-4.0.3.redhat-610379.jar:]
at org.apache.felix.framework.util.EventDispatcher.createWhitelistFromHooks(EventDispatcher.java:696)[org.apache.felix.framework-4.0.3.redhat-610379.jar:]
at org.apache.felix.framework.util.EventDispatcher.fireBundleEvent(EventDispatcher.java:484)[org.apache.felix.framework-4.0.3.redhat-610379.jar:]
at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:4650)[org.apache.felix.framework-4.0.3.redhat-610379.jar:]
at org.apache.felix.framework.Felix$4.run(Felix.java:2123)[org.apache.felix.framework-4.0.3.redhat-610379.jar:]
at org.apache.felix.framework.Felix.runInContext(Felix.java:2147)[org.apache.felix.framework-4.0.3.redhat-610379.jar:]
at org.apache.felix.framework.Felix.startBundle(Felix.java:2121)[org.apache.felix.framework-4.0.3.redhat-610379.jar:]
at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:955)[org.apache.felix.framework-4.0.3.redhat-610379.jar:]
at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:942)[org.apache.felix.framework-4.0.3.redhat-610379.jar:]
at io.fabric8.agent.DeploymentAgent.install(DeploymentAgent.java:955)[58:io.fabric8.fabric-agent:1.0.0.redhat-424]
at io.fabric8.agent.DeploymentAgent.doUpdate(DeploymentAgent.java:572)[58:io.fabric8.fabric-agent:1.0.0.redhat-424]
at io.fabric8.agent.DeploymentAgent$2.run(DeploymentAgent.java:293)[58:io.fabric8.fabric-agent:1.0.0.redhat-424]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)[:1.7.0_60]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)[:1.7.0_60]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)[:1.7.0_60]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)[:1.7.0_60]
at java.lang.Thread.run(Thread.java:745)[:1.7.0_60]
Caused by: java.lang.NullPointerException
at org.kie.scanner.KieRepositoryScannerImpl.indexAtifacts(KieRepositoryScannerImpl.java:358)[617:org.kie.scanner.osgi:6.3.1.201604071253]
at org.kie.scanner.KieRepositoryScannerImpl.setKieContainer(KieRepositoryScannerImpl.java:90)[617:org.kie.scanner.osgi:6.3.1.201604071253]
at org.drools.compiler.kie.builder.impl.KieServicesImpl.newKieScanner(KieServicesImpl.java:139)
at com.redhat.drools.camel.beans.KieScannerKieSessionService.<init>(KieScannerKieSessionService.java:24)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)[:1.7.0_60]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)[:1.7.0_60]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)[:1.7.0_60]
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)[:1.7.0_60]
at org.apache.aries.blueprint.utils.ReflectionUtils.newInstance(ReflectionUtils.java:329)[8:org.apache.aries.blueprint.core:1.0.1.redhat-611423]
at org.apache.aries.blueprint.container.BeanRecipe.newInstance(BeanRecipe.java:962)[8:org.apache.aries.blueprint.core:1.0.1.redhat-611423]
at org.apache.aries.blueprint.container.BeanRecipe.getInstance(BeanRecipe.java:331)[8:org.apache.aries.blueprint.core:1.0.1.redhat-611423]
... 34 more