Uploaded image for project: 'JBoss A-MQ'
  1. JBoss A-MQ
  2. ENTMQ-980

java.lang.NegativeArraySizeException when failover using new preallocation strategies

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Blocker
    • JBoss A-MQ 6.2
    • JBoss A-MQ 6.1, JBoss A-MQ 6.2
    • broker, kahadb

    Description

      Hi,

      We recently implemented ENTMQ-512 which allows us to pre-allocate journals to recognize performance gains on GFS2:

       <persistenceAdapter>
                  <kahaDB directory="${activemq.data}/kahadb" preallocationStrategy="zeros" preallocationScope="entire_journal" />
              </persistenceAdapter>
      

      To reproduce:

      1. Start Master/Slave pair
      2. Push some persistent messages:

      ant producer -Ddurable=true -Dmax=100

      3. Stop the Master
      4. Try to read the messages:

      ERROR | org.apache.activemq.broker.region.cursors.QueueStorePrefetch@533db7a7:TEST.FOO,batchResetNeeded=false,size=100,cacheEnabled=false,maxBatchSize:100,hasSpace:true,pendingCachedIds.size:0,lastSyncCachedId:null,lastSyncCachedId-seq:null,lastAsyncCachedId:null,lastAsyncCachedId-seq:null,store=permits:10000,sd=nextSeq:100,lastRet:MessageOrderCursor:[def:0, low:0, high:0],pending:0 - Failed to fill batch
      java.io.IOException: Invalid location: 1:622, : java.lang.NegativeArraySizeException
      	at org.apache.activemq.store.kahadb.disk.journal.DataFileAccessor.readRecord(DataFileAccessor.java:91)
      	at org.apache.activemq.store.kahadb.disk.journal.Journal.read(Journal.java:618)
      	at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:1010)
      	at org.apache.activemq.store.kahadb.KahaDBStore.loadMessage(KahaDBStore.java:1122)
      	at org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore$5.execute(KahaDBStore.java:595)
      	at org.apache.activemq.store.kahadb.disk.page.Transaction.execute(Transaction.java:779)
      	at org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore.recoverNextMessages(KahaDBStore.java:584)
      	at org.apache.activemq.store.ProxyMessageStore.recoverNextMessages(ProxyMessageStore.java:106)
      	at org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:109)
      	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:385)
      	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.reset(AbstractStoreCursor.java:142)
      	at org.apache.activemq.broker.region.cursors.StoreQueueCursor.reset(StoreQueueCursor.java:159)
      	at org.apache.activemq.broker.region.Queue.doPageInForDispatch(Queue.java:1904)
      	at org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:2125)
      	at org.apache.activemq.broker.region.Queue.iterate(Queue.java:1606)
      	at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
      	at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      Caused by: java.lang.NegativeArraySizeException
      	at org.apache.activemq.store.kahadb.disk.journal.DataFileAccessor.readRecord(DataFileAccessor.java:86)
      	... 19 more
      ERROR | org.apache.activemq.broker.region.cursors.QueueStorePrefetch@533db7a7:TEST.FOO,batchResetNeeded=false,size=100,cacheEnabled=false,maxBatchSize:100,hasSpace:true,pendingCachedIds.size:0,lastSyncCachedId:null,lastSyncCachedId-seq:null,lastAsyncCachedId:null,lastAsyncCachedId-seq:null,store=permits:10000,sd=nextSeq:100,lastRet:MessageOrderCursor:[def:0, low:0, high:0],pending:0 - Failed to fill batch
      java.lang.RuntimeException: java.io.IOException: Invalid location: 1:622, : java.lang.NegativeArraySizeException
      	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:388)
      	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.reset(AbstractStoreCursor.java:142)
      	at org.apache.activemq.broker.region.cursors.StoreQueueCursor.reset(StoreQueueCursor.java:159)
      	at org.apache.activemq.broker.region.Queue.doPageInForDispatch(Queue.java:1904)
      	at org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:2125)
      	at org.apache.activemq.broker.region.Queue.iterate(Queue.java:1606)
      	at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
      	at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      Caused by: java.io.IOException: Invalid location: 1:622, : java.lang.NegativeArraySizeException
      	at org.apache.activemq.store.kahadb.disk.journal.DataFileAccessor.readRecord(DataFileAccessor.java:91)
      	at org.apache.activemq.store.kahadb.disk.journal.Journal.read(Journal.java:618)
      	at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:1010)
      	at org.apache.activemq.store.kahadb.KahaDBStore.loadMessage(KahaDBStore.java:1122)
      	at org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore$5.execute(KahaDBStore.java:595)
      	at org.apache.activemq.store.kahadb.disk.page.Transaction.execute(Transaction.java:779)
      	at org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore.recoverNextMessages(KahaDBStore.java:584)
      	at org.apache.activemq.store.ProxyMessageStore.recoverNextMessages(ProxyMessageStore.java:106)
      	at org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:109)
      	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:385)
      	... 10 more
      Caused by: java.lang.NegativeArraySizeException
      	at org.apache.activemq.store.kahadb.disk.journal.DataFileAccessor.readRecord(DataFileAccessor.java:86)
      	... 19 more
      ERROR | Failed to page in more queue messages 
      java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: Invalid location: 1:622, : java.lang.NegativeArraySizeException
      	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.reset(AbstractStoreCursor.java:145)
      	at org.apache.activemq.broker.region.cursors.StoreQueueCursor.reset(StoreQueueCursor.java:159)
      	at org.apache.activemq.broker.region.Queue.doPageInForDispatch(Queue.java:1904)
      	at org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:2125)
      	at org.apache.activemq.broker.region.Queue.iterate(Queue.java:1606)
      	at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
      	at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      Caused by: java.lang.RuntimeException: java.io.IOException: Invalid location: 1:622, : java.lang.NegativeArraySizeException
      	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:388)
      	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.reset(AbstractStoreCursor.java:142)
      	... 9 more
      Caused by: java.io.IOException: Invalid location: 1:622, : java.lang.NegativeArraySizeException
      	at org.apache.activemq.store.kahadb.disk.journal.DataFileAccessor.readRecord(DataFileAccessor.java:91)
      	at org.apache.activemq.store.kahadb.disk.journal.Journal.read(Journal.java:618)
      	at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:1010)
      	at org.apache.activemq.store.kahadb.KahaDBStore.loadMessage(KahaDBStore.java:1122)
      	at org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore$5.execute(KahaDBStore.java:595)
      	at org.apache.activemq.store.kahadb.disk.page.Transaction.execute(Transaction.java:779)
      	at org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore.recoverNextMessages(KahaDBStore.java:584)
      	at org.apache.activemq.store.ProxyMessageStore.recoverNextMessages(ProxyMessageStore.java:106)
      	at org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:109)
      	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:385)
      	... 10 more
      Caused by: java.lang.NegativeArraySizeException
      	at org.apache.activemq.store.kahadb.disk.journal.DataFileAccessor.readRecord(DataFileAccessor.java:86)
      	... 19 more
      

      Attachments

        Activity

          People

            gtully@redhat.com Gary Tully
            rhn-support-sjavurek Susan Javurek
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: