The CDI project is part ofJakarta and uses GitHub issues as it's issue tracking system.
Therefore, all issues in CDI JIRA project are being bulk-closed as described in this GitHub issue.

If you feel like this particular issue deserves ongoing discussion, investigation or fixes in CDI/CDI TCK, please create a new issue under GitHub repository and include a link to this JIRA.

For specification related question/issues, please use - https://github.com/eclipse-ee4j/cdi/issues
For CDI TCK related questions/issues, please use - https://github.com/eclipse-ee4j/cdi-tck/issues

let me put this issue in a different context:

(1) let us say an application is deployed with three CORE modules that are essentials and cannot be disabled or removed, each having their own named CDI beans, which may depend on other such beans from the other two core modules.
this setup works perfectly as all the modules (and CDI beans, interceptors, qualifiers, decorators, etc) are always loaded , and at deployment

(2) a new module is installed into this system at runtime - with the following CDI usage:
inject existing CDI beans, supplied by one or more of the always present core modules, into its own injectable classes like Servlets, JSF backing beans, transaction Services, etc. i suppose this should not be an issue at all, since the new module is only trying to use existing CDI components. if for example for Events that need to be fired, the newly registered CDI components from the installed module (plugin, for clarity) can be added as EventListeners or interceptors during bean registration. the core modules shall never depend on the additional modules. it is only the CDI components from additional modules that may listen to or intercept the events and method invocations of CDI components from core modules. ---- what is the problem in supporting this feature?

(3) let us say module-x and module-y are deployed as additional modules, and module-y specified dependency on module-x because one or more of the CDI component in module-y is using or listening or intercepting one or more of the CDI components from module-x. in this case - the module framework will not let the installation of module-y if module-x is not installed; similarly, the framework will not let the uninstallation of module-x without the uninstallation of module-y. so, we can always be sure that the CDI components required by module-y are always present in the application because when module-y is there, the module-x shall always be there. at no point in time, a CDI component in module-x can refer to (depend on) a CDI component of module-y. that dependency management is the responsibility of the module framework, if it fails there, it is not the problem of CDI. why can't this be supported?

(4) i guess you could be thinking about a scenario where one CDI component from module-x depends on other CDI compoent of module-y while another CDI component of module-y depends on yet another CDI component of module-x.... i.e. cyclical dependency of modules... i agree that makes it much more complex, and require checking boundaries too often-- but why can't a well-defined dependency graph be supported?

all that is needed is registering the CDI components (by component, i mean bean/interceptor/qualifier/producer/etc) at runtime, and broadcasting the events (including method invocation - for interceptors) to runtime registered CDI components. what is so dangerous in this?

as a realistic use case :
suppose a web app is deployed with some core set of JPA Entities, JSF backing beans, JSF pages, CDI Transaction beans that may offer REST services too (because EJB container do not allow dynamic adding of beans, we rely solely on CDI) . now into this system, a new module gets installed that brings in new JPA entities, JSF backing beans, JSF pages, its own CDI transactional components, etc. now, any of the CDI components (including backing beans) may like to listen for or depend or intercept any of CDI components deployed at startup (or installed before). why can’t this be achieved since the previously installed CDI components can never depend on the newly installed CDI components? what problems do you see in this? in fact, we can easily uninstall this newly installed module and remove all of its CDI components with out any impact to the core web app system that was deployed initially. — all that is needed is to track what came in new, and which new ones are listening to or intercepting around which old ones. refusing to implement this feature because improper usage may cause unpredictable errors is like saying synchronized key word cannot be supported because its improper use leads to deadlocks in multi-threaded systems.

Hmm, does it means xwiki is leaking cause previously injected values can be unregistered ("remove extension")?

Actually I don't know how we would behave if you remove at runtime a component that's been previously injected in another component. In theory yes there should be a problem.In practice in 12 years of XWiki I've never seen this happen.

Or did you mean you never inject the component by type but always Provider<X> everywhere?

No we don't do that systematically, although we recommend it. It's done mostly at places where it's important to be able to use the latest registered component for a given role.

If so you also do lazy validation which means you accept to start a not functional app cause of a missing component - CDI prevents it. Is that right?

Yes we do allow this.

Thanks

Hi ljnelson+github@gmail.com. Thanks for mentioning HK2, I didn't know about it. I'll check it out. At first glance, it look very similar to what we have implemented in XWiki. I'll read more about it.

However I'm not sure it would justify the effort to switch to it. We don't have any problem with our custom Component Manager and it's low cost maintenance now. What would improve the XWiki technology stack would be to move to a "standard" DI framework. CDI would probably have been the most standard option if it had supported runtime addition of beans. Guice would be another possibility but I'd need to check how dynamicity could be implemented (I remember reading that it was possible).

Thanks

(vmassol_jira, you may wish to use HK2 instead of CDI in these cases as it is designed for this sort of dynamic situation. I personally prefer the static nature of CDI and am very glad that the relationships between beans are set up in advance, and consider this static bias part of the nature of CDI. HK2 is a much more freewheeling dependency injection framework with some similar features; it may more closely match your requirements.)

Hmm, does it means xwiki is leaking cause previously injected values can be unregistered ("remove extension")? Or did you mean you never inject the component by type but always Provider<X> everywhere? If so you also do lazy validation which means you accept to start a not functional app cause of a missing component - CDI prevents it. Is that right? If so I fear you need something to "mock" until you can inject an actual component. Means you can refactor your code but not change it drastically cause of these rules.

since xwiki is dynamic it needs to control what was injected before to be able to update which is not in Spring/Guice/CDI as basic features (always done through proxies and as an internal bean impl).

In XWiki we do lazy instantiation of components, i.e. we instantiate them when they are required (ie. when the ComponentManager.getInstance() is called or when a component needs to be injected in another component). However we do not control what was injected before. FTR XWiki only supports field injection. Thus if you register a new implementation of a Component role at runtime, all existing components that have been instantiated already are not modified. However when a component needs to have an aways up to date @Inject(ed) field, we use a Provider<> and the code using it calls get() on it to return the latest registered component instance from the Component Manager.

BTW thanks for your replies!

locally yes but in term of lifecycle not sure since xwiki is dynamic it needs to control what was injected before to be able to update which is not in Spring/Guice/CDI as basic features (always done through proxies and as an internal bean impl). So concretely if you do the hierarchic solution with a custom BeanManagerAggregator then you are fine and replaced the Map from your manager but you still need a dynamic runtime design "by design" which intends to breaks the startup validation and runtime safety. (that's what i meant)

rmannibucau@gmail.com You lost me with your last reply! AFAIK the XWiki Component Manager does exactly what Guice or CDI do (except that it handles runtime registration of beans).

We have components both in the core and in extensions.

producers are Bean so you can reuse CDI semantic (qualifiers etc) but not interceptors (until you use the factory cdi 2.0 introduces which is another part of the DIY) and other parts like post constuct etc as well (Unmanaged or equivalent). So whatever you do producers are for 3rd party integration and you keep all the work in this 3rd party. Until you add scanning and use default managed Beans you will not benefit of all CDI but just a subset like contexts, light lifecycle (produces/disposes), integration with other beans etc...

Also it is unclear to me how CDI would fit your need since you have components but generally in extensions only and a set of selectable component depending a key where CDI will not help much (you still need a custom lookup aligned on your design so CDI or not is equivalent there which the Bean is managed by CDI which is done with an inherited BeanManager solution). Also note adding at runtime Beans still requires a classloader (CDI is not doing anything there) so this solution is actually low cost for you anyway.

rmannibucau@gmail.com If we need to keep our XWiki Component Manager then there's little interest in moving to CDI since (if I understand correctly) we would be able to use the CDI features only for core components that are registered statically at application boot time (and thus have two distinct Component models depending on whether you're coding a core extension or a non-core one). We're also reducing our number of core components to the maximum as time progresses to have the smallest possible core and all the rest are extracted into extensions. We currently have hundreds of extensions and this will grow even more in a very near future. So having 2 models is not a good idea IMO

but producers generally means 'do it yourself'

I don't understand this part. AFAIK you can use CDI constructs in producers, using injections, qualifiers, etc. Aren't there plenty of CDI apps that use producer constructs too without requiring to implement a Component Manager that would handle instantiation/injection? If the object created in the producers requires to be injected other beans, CDI doesn't provide any helper for that?

Thanks

If you know the type you can surely sort it out with producers but producers generally means 'do it yourself' which would lead to keeping the component manager more or less. However an extension can produce any bean based on a qualifier (see https://gist.github.com/antoinesd/3097661ca99fa61900fb but instead of Object you can use the xwiki-api/interfaces). Issue allowing runtime registration is you would need to reboot the container more or less bypassing the scanning only to validate injection points and to reinject all created beans to ensure the consistency. It is a lot of work and overhead at runtime for not that much gain since a manager is easy and fast for this kind of dynamic registration - even with subclassloaders. Hope this makes some sense

rmannibucau@gmail.com I think xwiki component == cdi bean. Basically an XWiki component has a lifecycle (instantiation, injection, disposal) managed by the XWiki Component Manager. The goal of of using CDI for us would be to drop our Component Manager in favor of CDI (I guess it's the Bean Manager in CDI parlance but I'm a newbie so I could use the wrong terminology).

Regarding the scripted macros, it's a special case. The way we handle this now is that the XWiki Component Manager has two ways to register a new component: by passing a Class/Type and it would instantiate it itself or by passing an existing instance (in which case, it'll not perform the instantiation but it'll still perform the populate/injections). As you mentioned, I guess that the equivalent in CDI is the Producer strategy.

vmassol_jira don't you mix xwiki component and cdi bean? a macro is not a Bean generally until you generate a custom class at runtime which in any case doesn't need CDI no? The producer option would still work but is still overkill to handle wiki macros since there is a manager in xwiki, no?

So you have multiple 'apps' already running as each of your Wiki-namespace is essentially an own 'app' from a CDI pov, right?

Not sure. Basically we have a Root Component Manager containing components seen by all namespaces (ie all wikis) and then each subwiki have their own Wiki Component Manager that contain components only seen by those subwikis (so in addition to the components located in the Root Component Manager).

Would it be possible to bootstrap a new namespace instance and then just swapping out the original one with the freshly booted one? Looks much cleaner to me.

So you're essentially saying the same thing as what was said earlier but with even most cost in term of performance, no? I.e recreate the classloaders (the root one and the wiki ones affected by the new component/bean being added) + reinitialize CDI (ie scanning and bean resolving), every time a new component is to be added at runtime.

I don't know the performance of such an option but as I mentioned, we do have thousands and thousands of components/beans loaded, with some have initialization steps. IMO that would really be overkill in term of performance. Also in XWiki it's not that unfrequent to add new components at runtime. For example users can creates new macros as script in wiki pages and underneath they get registered as proper components/beans.

Thanks

I do that in an app (except I have a single plugin but design is reusable for that case) starting N children CDI context inheriting from the parent one (which would be xwiki webapp). Child having access to the parent they can register services etc and still use CDI. Limitation is children are isolated but it can be a nice thing too for plugins. Would it be an option? CDI 2 with its se API should enable that since the managed instance (EE/Servlet) will start several SE contexts.

I see.

So you have multiple 'apps' already running as each of your Wiki-namespace is essentially an own 'app' from a CDI pov, right?

Would it be possible to bootstrap a new namespace instance and then just swapping out the original one with the freshly booted one? Looks much cleaner to me.

So you allow that your users upload some jar which you then dynamically add to you Classpath?
Or how does the XWiki container become aware of that new functionality?
Does XWiki use OSGi (which would be suited for such a dynamic reload)?
Or do you roll your own ClassLoader system?

Yes admin users (or any user having the right permission) can install JAR extensions.

XWiki doesn't use OSGI. We asked ourselves the question initially but it was way too compex and heavy so we decided that we wouldn't be able to have several versions of the same class with different versions in XWiki. We have a simple custom Component system to manage this (the main class is here if you're curious: https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki-commons-component/xwiki-commons-component-default/src/main/java/org/xwiki/component/embed/EmbeddableComponentManager.java).

We do have our own classloader (we have one per namespace - XWiki supports multiple wikis and a namespace is for example one wiki, to provide isolation per wiki). When a JAR extension is uninstalled, we recreate the classloader, removing the JAR URL. FYI: https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwiki-platform-classloader/xwiki-platform-classloader-xwiki/src/main/java/org/xwiki/classloader/xwiki/internal/ContextNamespaceURLClassLoader.java and and https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki-commons-classloader/xwiki-commons-classloader-api/src/main/java/org/xwiki/classloader/URIClassLoader.java (this later one is just there to overcome JDK bugs/limitations).

atijms That is totally up to the container. It's simply not defined, so you cannot be sure.

vmassol_jira I see. So you allow that your users upload some jar which you then dynamically add to you Classpath?
Or how does the XWiki container become aware of that new functionality?
Does XWiki use OSGi (which would be suited for such a dynamic reload)?
Or do you roll your own ClassLoader system?

This could also help with the longstanding limitation that web.xml content can not be used for registering beans, since at the time the CDI extension is executed the ServletContext (and with it, WEB-INF/*) is not yet available.

Hi struberg. My exact use case is the following: allowing users of XWiki the ability to install extensions at runtime and without restarting XWiki, and these extensions contain Beans (Components in XWiki terminology) that need to be instantiated/populated and injected in other beans/code requiring them. So this is indeed not for development time but at runtime, when using XWiki in production.

Thanks

What is your exact use case? Usually the bean graph and Classpath per application is pretty much static once booted.
Except during development.
If it's just for development then you might look at https://github.com/HotswapProjects

Hi guys. I was wondering if there was now a recommended way of supporting the main use case I listed above back in 2011, namely:

If you have a CDI application, how can you add support for installing extensions at runtime for this application (for example by adding JARs containing beans to the classloader), and tell the Bean Manager to take into account those news beans?

I suspect this is a common need for apps as they grow (i.e. the ability to dynamically add extensions). As I mentioned back then, we've been supporting this in XWiki for about 10 years now with our custom handmade solution but I hate using custom solutions when there exists standard solutions so I'm always on the lookout for such solutions. I've been wanting to try to use CDI for years now but have dropped it in 2011 because of the lack of a solution.

The only option I've heard so far would be to tell the Bean Manager to "restart" and thus re-run the process of creating/resolving the Bean graph. Is that still the only solution? However, I'm worried that this would take too long for our needs since XWiki has thousands (possibly tens of thousands) of Beans running, coming from hundreds of JAR files. Is this "restart"/"reload" already implemented and optimized so that there's no rescanning of JAR files for example? Does anyone have any experience with this?

Thanks a lot!

I use the JRebel tool with Weld and on-the-fly restart of beans works like a charm. Of course, Session scoped beans only works when you logout/login. Application scoped beans requires application restart. Request scoped beans get reloaded as soon as you place a new request.

I'm very reluctant to allow dynamically defined beans. If we did, it would really be CDI2 not CDI 2.0 (completely different non-backwards compat design).

If the CDI container can restart fast enough, there is no real difference between dynamic beans and restarting the container. Of course, you don't want to restart the whole app as you loose session state. I'm talking about just restarting the CDI container.

Plus security aspects.

Hi Vince!

Will need to sleep over it, but currently this looks more like a 'hot redeploy' than a real 'dynamic Bean' change. The problem with allowing to add dynamic Beans is that they e.g. can also 'disable' other Beans, change Interceptors, Decorators, etc. So by adding one Bean, you might in the worst case change the behaviour of all the application.

Another area which we need to clarify: what happens if a definition error occurs as result of adding such a new bean? It would also be very hard (or close to almost impossible) to guarantee the behaviour of previously serialized beans.

I'm not saying that it's impossible, but we must really think about the consequences and how to cope with them.

Maybe you could explain what kind of 'dynamics' you have in XWiki?

Here are some use cases we have:

• We have an Extension Manager which allows us to install various kind of extensions (http://extensions.xwiki.org) at runtime in your wiki. One type of extensions is a JAR which contains Java classes and components. All components found in that JAR are discovered and registered in the Component Manager. For example there could be a Java Macro in the JAR and in the WYSIWYG editor we list all available macros so that new macro will appear automatically in the list. A component can also override an existing component, thus replacing it altogether.

• Another use case is that users can define new wiki macros in wiki pages (there's a form with various fields like macro id, macro description, macro parameters and macro content which is a script to execute. When the page is saved, an event is fired and we have a WikiMacro Manager component listening which then registers that wiki macro as a Macro Component against the Component Manager.

• A last use case is that in wiki pages we can put scripts (velocity, groovy, python, ruby to name a few) and we can define a Java Class in script and register through scripting against the Component Manager making the new component either replace an existing one or register it as a new component.

More information on our Component model is available here too in case it's useful: http://extensions.xwiki.org/xwiki/bin/view/Extension/Component+Module

Thanks Mark!

Hi Vincent!

Maybe we need to clarify further what we both understand under the term 'dynamic Bean'

What I meant is: dynamically adding Bean<T> metainformation to the container at runtime. E.g. disabling a class or adding a Bean<T> under load which would completely change the behaviour of the container.
I think this is very dangerous.

What will perfectly work already is the following: Have a Bean<T> which gets parsed and scanned on startup and which behaves differently in Contextual<T>#create() depending on the situation and it might even change it's behaviour at runtime (to some degree).

Currently it will be a bad idea to e.g. change the EL name of a bean at runtime. This will produce crap if the EL-Resolver caches the name-> bean mapping for example. We could introduce a Bean<T>#isDynamic() to prevent the container from applying caching in CDI-1.1.
But a fundamental design decision as of today is that the rules which lead to finding that Bean<T> do not change.

Maybe you could explain what kind of 'dynamics' you have in XWiki?

Actually not supporting beans which can dynamically change at runtime is a BIG benefit.

What this means is that CDI will not have the future I imagined (i.e. be the standard Component framework that becomes the standard reference). Because by this statement you're telling everyone who has a need for dynamic registration to go away. What you're doing is actually moving the people to OSGi or other such frameworks (I'd be curious to know how you position CDI vs OSGi/Jigsaw for the future - they seem to fill the same overall need for me and I'm not yet sure we'll need to have several).

Of course it's easy to be fast when you don't do much but if your use case isn't covered that doesn't help you much...

FTR here's an implementation we have done for XWiki:
https://github.com/xwiki/xwiki-commons/tree/master/xwiki-commons-core/xwiki-commons-component

More specifically the default Component Manager implementation we have is here:
https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki-commons-component/xwiki-commons-component-default/src/main/java/org/xwiki/component/embed/EmbeddableComponentManager.java

Of course our implementation has some consequences on performance since we instantiate components lazily and and since we have a singleton lifecycle we need to use ConcurrentHashMap and have some synchronized blocks. This is also needed for dynamic component registration.

I have no figure to show about OSGi perf but it probably is less performant because of its dynamic nature. However it doesn't seem to me that it's preventing people from using it. I think that vast majority of applications (let's say 99% for the sake of arguing ) would be just fine with the small overhead that synchronization brings. Now I also agree that 99% of apps may not need dynamicity ATM (although I think more and more will want it as time passes and apps become more complex). It's just a pity that CDI isn't a more generic Component framework.

Anyway for the XWiki project we have our own implementation for now for lack of a standard solution. I would have liked to see CDI being that solution but if it doesn't support dynamically adding beans it won't work for us.

Thanks

Actually not supporting beans which can dynamically change at runtime is a BIG benefit. We are now >30 times faster than Spring3 core, mostly because of the ability to aggressively cache intermediate results. If we would support dynamic beans, then we would need to re-evaluate all bits from the very bottom, basically starting with picking up all annotations and meta-info for each and every method invocation to a bean (as spring does today) I already know big customers which moved over to CDI because of that.

Of course we might introduce some support for hot-deployment which can be triggered (and would then stale the whole app for that timeframe).

Yes, but as a scripting language where the class can be updated without redeployment/restart. Guys at JBoss have been doing some JRuby/Weld integration (http://bit.ly/fz4Bgq) Yet, it does not allow a JRuby class act as a managed bean among other features. Their use case was fast front-end development cycle.

Adding a bit of dynamic behavior through a SPI wouldn't hurt the spirit of CDI as much as the inclusion of the Reflection API didn't make Java developers overuse unsafe invocations.

Anyway, it sounds hard to implement and maybe should be first implemented as an extension to Weld before becoming part of the spec.

Do you mean generally adding support for dynamic languages such as Groovy?

I remember the thread and the general case is supporting scripted beans (injecting into them, injecting them into others).

Most large enterprise apps one day or another can really benefits from scripting. It might be out of scope of CDI 1.x, but script support is something to be taken into account in the future.

We can discuss this in the EG however this would be a radical change from the 1.0 design both in API and in spirit. I for one would be very against this. Perhaps explain your problem on IRC and I'll try to help you with a way around it.