JBoss ClassLoader
  1. JBoss ClassLoader
  2. JBCL-117

Performance issue: VFSClassLoaderPolicy uses URL.hashCode(), causing a DNS lookup

    Details

    • Similar Issues:
      Show 10 results 

      Description

      There is a performance issue in JBoss AS 5.1.0.GA caused by the use of a ConcurrentHashMap with URL objects as the hash key. The offending usage is in VFSClassLoaderPolicy, in a member field named manifestCache. The field manifestCache is initialized as ConcurrentHashMap<URL, Manifest>();

      The underlying issue is that URL.hashCode() is a non-trivial operation, often requiring a DNS lookup. This DNS lookup can take a long time to timeout, adding up to 1-minute to the startup time of the application server .The issue is also encountered during runtime if certain class loader features are used (ie. getResource), causing up to a 1-minute delay when such features are used.

      The DNS lookups will never resolve in most cases, as the URL objects in question are usually in the form of:

      vfsmemory:aa15-xok7cr-fznfpulr-1-fznfpxto-7...

      A simple fix is to use the String returned form URL.toString() as the map key.

      Backtrace showing URL.hashCode causing a DNS lookup when used in ConcurrentHashMap from VFSClassLoaderPolicy:
      Inet6AddressImpl.lookupAllHostAddr(String) line: not available [native method]
      InetAddress$1.lookupAllHostAddr(String) line: 849
      InetAddress.getAddressFromNameService(String, InetAddress) line: 1200
      InetAddress.getAllByName0(String, InetAddress, boolean) line: 1153
      InetAddress.getAllByName(String, InetAddress) line: 1083
      InetAddress.getAllByName(String) line: 1019
      InetAddress.getByName(String) line: 969
      Handler(URLStreamHandler).getHostAddress(URL) line: 420
      Handler(URLStreamHandler).hashCode(URL) line: 337
      URL.hashCode() line: 857 [local variables unavailable]
      ConcurrentHashMap<K,V>.get(Object) line: 768
      VFSClassLoaderPolicy.getClassPackageInformation(String, String) line: 606
      BaseClassLoader.definePackage(String, URL) line: 722
      BaseClassLoader$2.run() line: 567
      BaseClassLoader$2.run() line: 532
      AccessController.doPrivileged(PrivilegedAction<T>, AccessControlContext) line: not available [native method]
      BaseClassLoader.loadClassLocally(String, boolean) line: 530
      BaseClassLoader.loadClassLocally(String) line: 507
      DelegateLoader(BaseDelegateLoader).loadClass(String) line: 134
      ClassLoadingTask$ThreadTask.run() line: 452
      ClassLoaderManager.nextTask(Thread, ClassLoadingTask) line: 251
      ClassLoaderManager.process(Thread, ClassLoadingTask) line: 150
      ClassLoaderDomain(BaseClassLoaderDomain).loadClass(BaseClassLoader, String, boolean) line: 265
      ClassLoaderDomain(BaseClassLoaderDomain).loadClass(BaseClassLoader, String) line: 1119
      BaseClassLoader.loadClassFromDomain(String, boolean) line: 798
      BaseClassLoader.loadClass(String, boolean) line: 441
      BaseClassLoader(ClassLoader).loadClass(String) line: 252
      ToClassInvoker.toClass(ToClassInvokerPoolReference, CtClass, String, ClassLoader, ProtectionDomain) line: 88
      JBossClDelegatingClassPool.toClass(CtClass, ClassLoader, ProtectionDomain) line: 81
      CtNewClass(CtClass).toClass(ClassLoader, ProtectionDomain) line: 1094
      TransformerCommon$ToClassAction$2.toClass(CtClass, ClassLoader, ProtectionDomain) line: 331
      TransformerCommon.toClass(CtClass, ClassLoader, ProtectionDomain) line: 139
      ProxyFactory.createProxyClass(ClassLoader, ProxyMixin[], Class<?>[]) line: 126
      ProxyFactory.createInterfaceProxy(GUID, ClassLoader, Class<?>[], ProxyMixin[], InstanceAdvisor) line: 96
      ProxyFactory.createInterfaceProxy(GUID, ClassLoader, Class<?>[]) line: 86
      Remoting.createRemoteProxy(Object, ClassLoader, Class[], InvokerLocator, List<Interceptor>, String) line: 102
      ManagedOperationProxyFactory.createDispatcherProxy() line: 203
      ManagedOperationProxyFactory.start() line: 109
      NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
      NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39
      DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25
      Method.invoke(Object, Object...) line: 597
      ReflectionUtils.invoke(Method, Object, Object[]) line: 59
      ReflectMethodInfoImpl.invoke(Object, Object[]) line: 150
      BasicMethodJoinPoint.dispatch() line: 66
      KernelControllerContextAction$JoinpointDispatchWrapper.execute() line: 241
      KernelControllerContextAction$JoinpointDispatchWrapper(ExecutionWrapper).execute(AccessControlContext) line: 47
      KernelControllerContextAction.dispatchExecutionWrapper(KernelControllerContext, ExecutionWrapper) line: 109
      KernelControllerContextAction.dispatchJoinPoint(KernelControllerContext, Joinpoint) line: 70
      StartStopLifecycleAction(LifecycleAction).installActionInternal(KernelControllerContext) line: 221
      StartStopLifecycleAction(InstallsAwareAction).installAction(KernelControllerContext) line: 54
      StartStopLifecycleAction(InstallsAwareAction).installAction(ControllerContext) line: 42
      StartStopLifecycleAction(SimpleControllerContextAction<T>).simpleInstallAction(T) line: 62
      StartStopLifecycleAction(AccessControllerContextAction<S,T>).install(ControllerContext) line: 71
      KernelControllerContextActions(AbstractControllerContextActions).install(ControllerContext, ControllerState, ControllerState) line: 51
      AbstractKernelControllerContext(AbstractControllerContext).install(ControllerState, ControllerState) line: 348
      AbstractKernelController(AbstractController).install(ControllerContext, ControllerState, ControllerState) line: 1631
      AbstractKernelController(AbstractController).incrementState(ControllerContext, boolean) line: 934
      AbstractKernelController(AbstractController).resolveContexts(ControllerState, ControllerState, boolean) line: 1082
      AbstractKernelController(AbstractController).resolveContexts(boolean) line: 984
      AbstractKernelController(AbstractController).install(ControllerContext, boolean) line: 774
      AbstractKernelController(AbstractController).install(ControllerContext) line: 540
      ServiceController.doInstall(KernelController, ServiceControllerContext) line: 670
      ServiceController.register(ObjectName, Collection<ObjectName>, boolean, Object) line: 373
      ServiceControllerRegistrationLifecycleCallback.install(ControllerContext) line: 104
      NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
      NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39
      DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25
      Method.invoke(Object, Object...) line: 597
      ReflectionUtils.invoke(Method, Object, Object[]) line: 59
      ReflectMethodInfoImpl.invoke(Object, Object[]) line: 150
      BasicMethodJoinPoint.dispatch() line: 66
      AbstractBeanInfo.invoke(Object, String, String[], Object[]) line: 300
      AbstractKernelControllerContext.invoke(String, Object[], String[]) line: 286
      AbstractLifecycleCallbackItem.install(ControllerContext) line: 87
      AbstractKernelController(AbstractController).handleLifecycleCallbacks(ControllerContext, ControllerState, boolean) line: 1568
      AbstractKernelController(AbstractController).handleInstallLifecycleCallbacks(ControllerContext, ControllerState) line: 1533
      AbstractKernelController(AbstractController).incrementState(ControllerContext, boolean) line: 943
      AbstractKernelController(AbstractController).resolveContexts(ControllerState, ControllerState, boolean) line: 1082
      AbstractKernelController(AbstractController).resolveContexts(boolean) line: 984
      AbstractKernelController(AbstractController).change(ControllerContext, ControllerState, boolean) line: 822
      AbstractKernelController(AbstractController).change(ControllerContext, ControllerState) line: 553
      ServiceController.doChange(KernelController, ServiceControllerContext, ControllerState, String) line: 688
      ServiceController.start(ObjectName) line: 460
      ServiceControllerStartStopLifecycleCallback.install(ControllerContext) line: 44
      NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
      NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39
      DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25
      Method.invoke(Object, Object...) line: 597
      ReflectionUtils.invoke(Method, Object, Object[]) line: 59
      ReflectMethodInfoImpl.invoke(Object, Object[]) line: 150
      BasicMethodJoinPoint.dispatch() line: 66
      AbstractBeanInfo.invoke(Object, String, String[], Object[]) line: 300
      AbstractKernelControllerContext.invoke(String, Object[], String[]) line: 286
      AbstractLifecycleCallbackItem.install(ControllerContext) line: 87
      AbstractKernelController(AbstractController).handleLifecycleCallbacks(ControllerContext, ControllerState, boolean) line: 1568
      AbstractKernelController(AbstractController).handleInstallLifecycleCallbacks(ControllerContext, ControllerState) line: 1533
      AbstractKernelController(AbstractController).incrementState(ControllerContext, boolean) line: 943
      AbstractKernelController(AbstractController).resolveContexts(ControllerState, ControllerState, boolean) line: 1082
      AbstractKernelController(AbstractController).resolveContexts(boolean) line: 984
      AbstractKernelController(AbstractController).install(ControllerContext, boolean) line: 774
      AbstractKernelController(AbstractController).install(ControllerContext) line: 540
      BeanMetaDataDeployer.deploy(DeploymentUnit, BeanMetaData) line: 121
      BeanMetaDataDeployer.deploy(DeploymentUnit, Object) line: 51
      BeanMetaDataDeployer(AbstractSimpleRealDeployer<T>).internalDeploy(DeploymentUnit) line: 62
      BeanMetaDataDeployer(AbstractRealDeployer).deploy(DeploymentUnit) line: 50
      DeployerWrapper.deploy(DeploymentUnit) line: 171
      DeployersImpl.doDeploy(Deployer, DeploymentUnit) line: 1439
      DeployersImpl.doInstallParentFirst(Deployer, DeploymentContext) line: 1157
      DeployersImpl.doInstallParentFirst(Deployer, DeploymentContext) line: 1178
      DeployersImpl.install(ControllerContext, ControllerState, ControllerState) line: 1098
      DeploymentControllerContext(AbstractControllerContext).install(ControllerState, ControllerState) line: 348
      AbstractKernelController(AbstractController).install(ControllerContext, ControllerState, ControllerState) line: 1631
      AbstractKernelController(AbstractController).incrementState(ControllerContext, boolean) line: 934
      AbstractKernelController(AbstractController).resolveContexts(ControllerState, ControllerState, boolean) line: 1082
      AbstractKernelController(AbstractController).resolveContexts(boolean) line: 984
      AbstractKernelController(AbstractController).change(ControllerContext, ControllerState, boolean) line: 822
      AbstractKernelController(AbstractController).change(ControllerContext, ControllerState) line: 553
      DeployersImpl.process(List<DeploymentContext>, List<DeploymentContext>) line: 781
      MainDeployerImpl.process() line: 702
      MainDeployerAdapter.process() line: 117
      ProfileDeployAction.install(Profile) line: 70
      ProfileDeployAction(AbstractProfileAction).install(ProfileContext) line: 53
      AbstractProfileService.install(ControllerContext, ControllerState, ControllerState) line: 361
      ProfileContext(AbstractControllerContext).install(ControllerState, ControllerState) line: 348
      ScopedProfileServiceController(AbstractController).install(ControllerContext, ControllerState, ControllerState) line: 1631
      ScopedProfileServiceController(AbstractController).incrementState(ControllerContext, boolean) line: 934
      ScopedProfileServiceController(AbstractController).resolveContexts(ControllerState, ControllerState, boolean) line: 1082
      ScopedProfileServiceController(AbstractController).resolveContexts(boolean) line: 984
      ScopedProfileServiceController(AbstractController).change(ControllerContext, ControllerState, boolean) line: 822
      ScopedProfileServiceController(AbstractController).change(ControllerContext, ControllerState) line: 553
      AbstractProfileService.activateProfile(ProfileKey) line: 306
      ProfileServiceBootstrap.start(Server) line: 271
      ServerImpl(AbstractServerImpl).start() line: 461
      Main.boot(String[]) line: 221
      Main$1.run() line: 556
      Thread.run() line: 619

        Issue Links

          Activity

          Hide
          Tolga Tarhan
          added a comment -

          The issue also appears on line 526 of VFSClassLoaderPolicy when we call urls.add(child.toURL()) where urls is an object of type Set<URL> passed in to getResources.

          Causes the exact same symptoms, at slightly different times. This is a little trickier than the previous case since the Set is passed-in from the caller.

          Show
          Tolga Tarhan
          added a comment - The issue also appears on line 526 of VFSClassLoaderPolicy when we call urls.add(child.toURL()) where urls is an object of type Set<URL> passed in to getResources. Causes the exact same symptoms, at slightly different times. This is a little trickier than the previous case since the Set is passed-in from the caller.
          Hide
          Adrian Brock
          added a comment -

          Forums are down at the moment.

          Isn't the real issue here that
          org.jboss.virtual.protocol.vfsmemory.Handler
          doesn't override

          URLStreamHandler.hashCode(url)

          It is using the "host" part of the url for something that is not a host.
          This is leading to spurious DNS lookups that cannot be cached because they always fail.

          You could argue (as I have in the past) that it shouldn't use the "host" in this way for this very reason.

          Show
          Adrian Brock
          added a comment - Forums are down at the moment. Isn't the real issue here that org.jboss.virtual.protocol.vfsmemory.Handler doesn't override URLStreamHandler.hashCode(url) It is using the "host" part of the url for something that is not a host. This is leading to spurious DNS lookups that cannot be cached because they always fail. You could argue (as I have in the past) that it shouldn't use the "host" in this way for this very reason.
          Hide
          Jason Greene
          added a comment -

          Yes, you are right, there are really two separate issues here.

          1. vfsmemory - the handler needs to be fixed regardless.

          2. Usage of URLs as keys - not a problem if classloading never uses host-based urls.

          Show
          Jason Greene
          added a comment - Yes, you are right, there are really two separate issues here. 1. vfsmemory - the handler needs to be fixed regardless. 2. Usage of URLs as keys - not a problem if classloading never uses host-based urls.
          Hide
          Tolga Tarhan
          added a comment -

          I agree that fixing org.jboss.virtual.protocol.vfsmemory.Handler is better.

          The attached patch does this by overriding getHostAddress(URL) inside org.jboss.virtual.protocol.vfsmemory.Handler to always return null. This is also semantically correct, since vfsmemory "hosts" are not real hosts at all.

          There's no reason to override hashCode() or equals(), as they both use getHostAddress().

          I've also attached this patch to JBCL-118; I wasn't sure which was more appropriate in this case.

          Show
          Tolga Tarhan
          added a comment - I agree that fixing org.jboss.virtual.protocol.vfsmemory.Handler is better. The attached patch does this by overriding getHostAddress(URL) inside org.jboss.virtual.protocol.vfsmemory.Handler to always return null. This is also semantically correct, since vfsmemory "hosts" are not real hosts at all. There's no reason to override hashCode() or equals(), as they both use getHostAddress(). I've also attached this patch to JBCL-118 ; I wasn't sure which was more appropriate in this case.
          Hide
          Ales Justin
          added a comment -

          I've applied the patch to VFS (JBVFS-120).

          Show
          Ales Justin
          added a comment - I've applied the patch to VFS ( JBVFS-120 ).

            People

            • Assignee:
              Jason Greene
              Reporter:
              Tolga Tarhan
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: