Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-12457

Single action timer is not triggered automatically after a DB outage, requires server restart

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • 18.0.0.Final
    • None
    • EJB
    • None
    • Hide

      See attached zip file for a reproducer:

      1. Configure EAP 7.2 with EJB timers backed by a DB
      2. Deploy the test application - it creates a single timer upon application startup, and a 10s single action timer on each subsequent timer execution
      3. Stop the DB
      4. Check that the next timer execution results in an ERROR
      5. Restart the DB
        => The timer execution is not resumed automatically, a server restart is required
      Show
      See attached zip file for a reproducer: Configure EAP 7.2 with EJB timers backed by a DB Deploy the test application - it creates a single timer upon application startup, and a 10s single action timer on each subsequent timer execution Stop the DB Check that the next timer execution results in an ERROR Restart the DB => The timer execution is not resumed automatically, a server restart is required

    Description

      A timer is created with the timerService.createSingleActionTimer(..) method. When the database is down at the time this timer expires, the timer execution fails with the following ERROR:

      09:01:32,968 ERROR [org.jboss.as.ejb3.timer] (EJB timer-thread-pool - 4) WFLYEJB0502: Exception checking if timer [id=7ac2a79a-daa8-46fb-afa1-9e72f39a5b01 timedObjectId=ejb-timer-db-outage.ejb-timer-db-outage.TimeoutExample auto-timer?:false persistent?:true timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl@7071c87c initialExpiration=Fri Aug 09 09:01:32 MDT 2019 intervalDuration(in milli sec)=0 nextExpiration=Fri Aug 09 09:01:32 MDT 2019 timerState=ACTIVE info=timer_1565362882955_10000] should run: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:jboss/datasources/ejb_timer
	at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:159)
	at org.jboss.as.connector.subsystems.datasources.WildFlyDataSource.getConnection(WildFlyDataSource.java:64)
	at org.jboss.as.ejb3.timerservice.persistence.database.DatabaseTimerPersistence.shouldRun(DatabaseTimerPersistence.java:405)
	at org.jboss.as.ejb3.timerservice.TimerServiceImpl.shouldRun(TimerServiceImpl.java:1148)
	at org.jboss.as.ejb3.timerservice.TimerTask.run(TimerTask.java:133)
	at org.jboss.as.ejb3.timerservice.TimerServiceImpl$Task$1.run(TimerServiceImpl.java:1304)
	at org.wildfly.extension.requestcontroller.RequestController$QueuedTask$1.run(RequestController.java:494)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
	at org.jboss.threads.JBossThread.run(JBossThread.java:485)
Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:jboss/datasources/ejb_timer
	at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:690)
	at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:440)
	at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:789)
	at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:151)
	... 10 more

      EXPECTATION: The timer should fire once the DB is accessible again.
      RESULT: The timer is not executed even after the DB connection has been restored.

      After a server restart, the timer is executed again.

      Note: This issue has originally been reported in the context of RHPAM as RHPAM-2180 where processes with timers get stuck after a DB outage. The relevant jBPM code calls timerService.createSingleActionTimer(..).

      Attachments

        Issue Links

          is cloned by

          Bug - A problem that impairs or prevents the functions of the product. JBEAP-17376 [GSS](7.2.z) Single action timer is not triggered automatically after a DB outage, requires server restart

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Verified

          Activity

            People

              cfang@redhat.com Cheng Fang
              elguardian@gmail.com Enrique González Martínez (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: