Uploaded image for project: 'WINDUP - Red Hat Application Migration Toolkit'
  1. WINDUP - Red Hat Application Migration Toolkit
  2. WINDUP-1078

"Too many open files" error in large runs (again)

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • None
    • 2.5.0.Final
    • MTA CLI
    • None

    Description

      I tried to run against all modules of our biggest app, and got into troubles with files limits.

      Not sure if we can do something about it besides increase the limits on the system were Windup runs.

      We have seen this error before.

      Windup Execution failed due to: Error encountered while evaluating rule: .addRule().perform(ParseJavaSource).withId("AnalyzeJavaFilesRuleProvider_1")
        From: org.jboss.windup.rules.apps.java.scan.provider.AnalyzeJavaFilesRuleProvider loaded from ModuleClassLoader for Module "org.jboss.windup.rules.apps.windup-rules-java:2.6.0-SNAPSHOT_1ad8b599-46fd-4c82-9cd6-c026fe4e07bf" from AddonModuleLoader
        Defined in: org.jboss.windup.rules.apps.java.scan.provider.AnalyzeJavaFilesRuleProvider.getConfiguration(AnalyzeJavaFilesRuleProvider.java:101)
      org.jboss.windup.util.exception.WindupException: Error encountered while evaluating rule: .addRule().perform(ParseJavaSource).withId("AnalyzeJavaFilesRuleProvider_1")
        From: org.jboss.windup.rules.apps.java.scan.provider.AnalyzeJavaFilesRuleProvider loaded from ModuleClassLoader for Module "org.jboss.windup.rules.apps.windup-rules-java:2.6.0-SNAPSHOT_1ad8b599-46fd-4c82-9cd6-c026fe4e07bf" from AddonModuleLoader
        Defined in: org.jboss.windup.rules.apps.java.scan.provider.AnalyzeJavaFilesRuleProvider.getConfiguration(AnalyzeJavaFilesRuleProvider.java:101)
              at org.jboss.windup.config.RuleSubset.perform(RuleSubset.java:344)
              at org.jboss.windup.exec.WindupProcessorImpl.execute(WindupProcessorImpl.java:152)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.jboss.forge.furnace.proxy.ClassLoaderInterceptor$1.call(ClassLoaderInterceptor.java:87)
              at org.jboss.forge.furnace.util.ClassLoaders.executeIn(ClassLoaders.java:42)
              at org.jboss.forge.furnace.proxy.ClassLoaderInterceptor.invoke(ClassLoaderInterceptor.java:103)
              at org.jboss.windup.exec.WindupProcessorImpl_$$_javassist_15853e78-ca2a-404c-ba60-11ece51ddb6b.execute(WindupProcessorImpl_$$_javassist_15853e78-ca2a-404c-ba60-11ece51ddb6b.java)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.jboss.forge.furnace.proxy.ClassLoaderAdapterCallback$2.call(ClassLoaderAdapterCallback.java:123)
              at org.jboss.forge.furnace.util.ClassLoaders.executeIn(ClassLoaders.java:42)
              at org.jboss.forge.furnace.proxy.ClassLoaderAdapterCallback.invoke(ClassLoaderAdapterCallback.java:96)
              at org.jboss.windup.exec.WindupProcessor_$$_javassist_37f87ae3-a50e-4cfa-a0d7-594a903aa15c.execute(WindupProcessor_$$_javassist_37f87ae3-a50e-4cfa-a0d7-594a903aa15c.java)
              at org.jboss.windup.bootstrap.commands.windup.RunWindupCommand.runWindup(RunWindupCommand.java:264)
              at org.jboss.windup.bootstrap.commands.windup.RunWindupCommand.execute(RunWindupCommand.java:75)
              at org.jboss.windup.bootstrap.Bootstrap.executePhase(Bootstrap.java:308)
              at org.jboss.windup.bootstrap.Bootstrap.run(Bootstrap.java:171)
              at org.jboss.windup.bootstrap.Bootstrap.main(Bootstrap.java:97)
      Caused by: com.thinkaurelius.titan.core.TitanException: Could not commit transaction due to exception during persistence
              at com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx.commit(StandardTitanTx.java:1348)
              at com.thinkaurelius.titan.graphdb.blueprints.TitanBlueprintsGraph.commit(TitanBlueprintsGraph.java:60)
              at org.jboss.windup.config.RuleSubset.perform(RuleSubset.java:306)
              ... 22 more
      Caused by: com.thinkaurelius.titan.core.TitanException: Could not execute operation due to backend exception
              at com.thinkaurelius.titan.diskstorage.util.BackendOperation.execute(BackendOperation.java:44)
              at com.thinkaurelius.titan.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:86)
              at com.thinkaurelius.titan.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:140)
              at com.thinkaurelius.titan.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:197)
              at com.thinkaurelius.titan.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:117)
              at com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.commit(StandardTitanGraph.java:670)
              at com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx.commit(StandardTitanTx.java:1337)
              ... 24 more
      Caused by: com.thinkaurelius.titan.diskstorage.PermanentBackendException: Permanent failure in storage backend
              at com.thinkaurelius.titan.diskstorage.berkeleyje.BerkeleyJEKeyValueStore.insert(BerkeleyJEKeyValueStore.java:206)
              at com.thinkaurelius.titan.diskstorage.berkeleyje.BerkeleyJEKeyValueStore.insert(BerkeleyJEKeyValueStore.java:183)
              at com.thinkaurelius.titan.diskstorage.berkeleyje.BerkeleyJEStoreManager.mutateMany(BerkeleyJEStoreManager.java:193)
              at com.thinkaurelius.titan.diskstorage.keycolumnvalue.keyvalue.OrderedKeyValueStoreManagerAdapter.mutateMany(OrderedKeyValueStoreManagerAdapter.java:100)
              at com.thinkaurelius.titan.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:89)
              at com.thinkaurelius.titan.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:86)
              at com.thinkaurelius.titan.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:56)
              at com.thinkaurelius.titan.diskstorage.util.BackendOperation.execute(BackendOperation.java:42)
              ... 30 more
      Caused by: com.sleepycat.je.EnvironmentFailureException: (JE 5.0.73) Environment must be closed, caused by: com.sleepycat.je.EnvironmentFailureException: Environment invalid because of previous exception: (JE 5.0.73) /tmp/Windup/resClfy/graph/titangraph java.io.FileNotFoundException: /tmp/Windup/resClfy/graph/titangraph/00000019.jdb (Too many open files) LOG_FILE_NOT_FOUND: Log file missing, log is likely invalid. Environment is invalid and must be closed. fetchTarget of 0x26b/0x35bb56 parent IN=90744 IN class=com.sleepycat.je.tree.IN lastFullVersion=0x285/0x1f8c6d lastLoggedVersion=0x285/0x1f8c6d parent.getDirty()=false state=0
              at com.sleepycat.je.EnvironmentFailureException.wrapSelf(EnvironmentFailureException.java:196)
              at com.sleepycat.je.dbi.EnvironmentImpl.checkIfInvalid(EnvironmentImpl.java:1512)
              at com.sleepycat.je.Database.checkEnv(Database.java:1884)
              at com.sleepycat.je.Database.put(Database.java:1164)
              at com.thinkaurelius.titan.diskstorage.berkeleyje.BerkeleyJEKeyValueStore.insert(BerkeleyJEKeyValueStore.java:194)
              ... 37 more
      Caused by: com.sleepycat.je.EnvironmentFailureException: Environment invalid because of previous exception: (JE 5.0.73) /tmp/Windup/resClfy/graph/titangraph java.io.FileNotFoundException: /tmp/Windup/resClfy/graph/titangraph/00000019.jdb (Too many open files) LOG_FILE_NOT_FOUND: Log file missing, log is likely invalid. Environment is invalid and must be closed. fetchTarget of 0x26b/0x35bb56 parent IN=90744 IN class=com.sleepycat.je.tree.IN lastFullVersion=0x285/0x1f8c6d lastLoggedVersion=0x285/0x1f8c6d parent.getDirty()=false state=0
              at com.sleepycat.je.log.LogManager.getLogEntryHandleFileNotFound(LogManager.java:862)
              at com.sleepycat.je.log.LogManager.getEntryHandleFileNotFound(LogManager.java:1121)
              at com.sleepycat.je.tree.BINDelta.reconstituteBIN(BINDelta.java:141)
              at com.sleepycat.je.log.entry.BINDeltaLogEntry.getIN(BINDeltaLogEntry.java:53)
              at com.sleepycat.je.log.entry.BINDeltaLogEntry.getResolvedItem(BINDeltaLogEntry.java:43)
              at com.sleepycat.je.tree.IN.fetchTarget(IN.java:1422)
              at com.sleepycat.je.tree.Tree.searchSubTreeInternal(Tree.java:1544)
              at com.sleepycat.je.tree.Tree.searchSubTree(Tree.java:1381)
              at com.sleepycat.je.tree.Tree.search(Tree.java:1240)
              at com.sleepycat.je.tree.Tree.getParentBINForChildLN(Tree.java:873)
              at com.sleepycat.je.cleaner.FileProcessor.processLN(FileProcessor.java:830)
              at com.sleepycat.je.cleaner.FileProcessor.processFile(FileProcessor.java:674)
              at com.sleepycat.je.cleaner.FileProcessor.doClean(FileProcessor.java:289)
              at com.sleepycat.je.cleaner.FileProcessor.onWakeup(FileProcessor.java:148)
              at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:163)
              at java.lang.Thread.run(Thread.java:745)
      Caused by: java.io.FileNotFoundException: /tmp/Windup/resClfy/graph/titangraph/00000019.jdb (Too many open files)
              at java.io.RandomAccessFile.open0(Native Method)
              at java.io.RandomAccessFile.open(RandomAccessFile.java:316)
              at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243)
              at java.io.RandomAccessFile.<init>(RandomAccessFile.java:124)
              at com.sleepycat.je.log.FileManager$DefaultRandomAccessFile.<init>(FileManager.java:3209)
              at com.sleepycat.je.log.FileManager$6.createFile(FileManager.java:3237)
              at com.sleepycat.je.log.FileManager.openFileHandle(FileManager.java:1280)
              at com.sleepycat.je.log.FileManager.getFileHandle(FileManager.java:1152)
              at com.sleepycat.je.log.LogManager.getLogSource(LogManager.java:1141)
              at com.sleepycat.je.log.LogManager.getLogEntry(LogManager.java:845)
              at com.sleepycat.je.log.LogManager.getLogEntry(LogManager.java:799)
              at com.sleepycat.je.log.LogManager.getLogEntryHandleFileNotFound(LogManager.java:860)
              ... 15 more
      

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              ozizka_jira Ondrej Zizka (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: