Uploaded image for project: 'Hawkular'
  1. Hawkular
  2. HAWKULAR-848

Hawkular doesn't shutdown with Ctrl-C

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • 1.0.0.Alpha8
    • 1.0.0.Alpha7
    • None
    • None

    Description

      Trying to shutdown Hawkular (Mac OS X, Standalone Cassandra) using Ctrl-C does not shuts the server down, something seems to be hanging. Complete stopping log is attached.

      Some relevant parts:

      2015-11-30 12:09:31,222 WARN  [org.hawkular.agent.monitor.service.MonitorService] (MSC service thread 1-6) HAWKMONITOR010055: Agent encountered errors during shutdown.: java.lang.IllegalStateException: [org.hawkular.agent.monitor.protocol.dmr.DMREndpointService] must be in state [RUNNING] rather than [RUNNING] to perfrom [removeInventoryListener()]
      	at org.hawkular.agent.monitor.service.ServiceStatus.assertInitialOrStopped(ServiceStatus.java:42)
      	at org.hawkular.agent.monitor.protocol.EndpointService.removeInventoryListener(EndpointService.java:328)
      	at org.hawkular.agent.monitor.protocol.ProtocolService.removeInventoryListener(ProtocolService.java:91)
      	at org.hawkular.agent.monitor.protocol.ProtocolServices.removeInventoryListener(ProtocolServices.java:221)
      	at org.hawkular.agent.monitor.service.MonitorService.stopMonitorService(MonitorService.java:522)
      	at org.hawkular.agent.monitor.service.MonitorService.stop(MonitorService.java:399)
      	at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:2056)
      	at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:2017)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      
      (...)
      
      2015-11-30 12:09:31,286 ERROR [org.apache.activemq.ra.ActiveMQEndpointWorker] (ActiveMQ Connection Executor: vm://org.hawkular.bus.broker.bucephalus#2) Connection to broker failed: peer (vm://org.hawkular.bus.broker.bucephalus#3) stopped.: javax.jms.JMSException: peer (vm://org.hawkular.bus.broker.bucephalus#3) stopped.
      	at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:54)
      	at org.apache.activemq.ActiveMQConnection.onAsyncException(ActiveMQConnection.java:1998)
      	at org.apache.activemq.ActiveMQConnection.onException(ActiveMQConnection.java:2017)
      	at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:101)
      	at org.apache.activemq.transport.ResponseCorrelator.onException(ResponseCorrelator.java:126)
      	at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:101)
      	at org.apache.activemq.transport.vm.VMTransport.stop(VMTransport.java:206)
      	at org.apache.activemq.transport.TransportFilter.stop(TransportFilter.java:65)
      	at org.apache.activemq.transport.TransportFilter.stop(TransportFilter.java:65)
      	at org.apache.activemq.transport.ResponseCorrelator.stop(ResponseCorrelator.java:132)
      	at org.apache.activemq.broker.TransportConnection.doStop(TransportConnection.java:1102)
      	at org.apache.activemq.broker.TransportConnection$4.run(TransportConnection.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: org.apache.activemq.transport.TransportDisposedIOException: peer (vm://org.hawkular.bus.broker.bucephalus#3) stopped.
      	... 9 more
      
      (...)
      
      2015-11-30 12:09:42,006 ERROR [org.hawkular.agent.monitor.storage.MetricsOnlyStorageAdapter] (OkHttp http://127.0.0.1:8080/hawkular/metrics/metrics/data) HAWKMONITOR010010: Failed to store metric data: {"counters":[],"availabilities":[],"gauges":[{"data":[{"value":3.3524988E8,"timestamp":1448885381998}],"id":"MI~R~[bucephalus~Local~~]~MT~WildFly Memory Metrics~Heap Used"}]}: java.io.IOException: status-code=[404], reason=[Not Found], url=[http://127.0.0.1:8080/hawkular/metrics/metrics/data]
      	at org.hawkular.agent.monitor.storage.MetricsOnlyStorageAdapter$1.onResponse(MetricsOnlyStorageAdapter.java:118)
      	at com.squareup.okhttp.Call$AsyncCall.execute(Call.java:170)
      	at com.squareup.okhttp.internal.NamedRunnable.run(NamedRunnable.java:33)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      
      2015-11-30 12:09:42,011 ERROR [org.hawkular.agent.monitor.protocol.EndpointService] (Hawkular-Monitor-Scheduler-Metrics-2) HAWKMONITOR010049: Could not access resources of endpoint [DMREndpointService[Endpoint[Local]:[null]]]: org.hawkular.agent.monitor.protocol.ProtocolException: Unsuccessful fetching DMR attribute [TimedOut]
      	at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:151)
      	at org.hawkular.agent.monitor.protocol.EndpointService.measureMetrics(EndpointService.java:299)
      	at org.hawkular.agent.monitor.scheduler.IntervalBasedScheduler$MetricsJob.run(IntervalBasedScheduler.java:94)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      	at org.jboss.threads.JBossThread.run(JBossThread.java:320)
      Caused by: org.hawkular.dmr.api.OperationFailureException: Could not perform operation [read-attribute]: WFLYCTL0030: No resource definition is registered for address [
          ("subsystem" => "datasources"),
          ("data-source" => "HawkularDS"),
          ("statistics" => "pool")
      ]
      	at org.hawkular.dmr.api.OperationBuilder$OperationResult.assertSuccess(OperationBuilder.java:422)
      	at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:148)
      	... 10 more
      
      2015-11-30 12:09:42,035 INFO  [com.squareup.okhttp.OkHttpClient] (OkHttp http://127.0.0.1:8080/hawkular/metrics/metrics/data) Callback failure for call to http://127.0.0.1:8080/...: java.io.IOException: status-code=[404], reason=[Not Found], url=[http://127.0.0.1:8080/hawkular/metrics/metrics/data]
      	at org.hawkular.agent.monitor.storage.MetricsOnlyStorageAdapter$1.onResponse(MetricsOnlyStorageAdapter.java:118)
      	at com.squareup.okhttp.Call$AsyncCall.execute(Call.java:170)
      	at com.squareup.okhttp.internal.NamedRunnable.run(NamedRunnable.java:33)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      
      (...)
      

      It seems the agent fails to stop and continues to try collecting and storing metrics. Not sure if it's the cause of the problem being reported.

      Attachments

        Activity

          People

            jmazzitelli John Mazzitelli
            alexandrem_jira Alexandre Mendonça (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: