Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-1891

Fluentd flush logs when Elasticsearch is not healthy overwhelming the logstore

    XMLWordPrintable

Details

    • False
    • False
    • NEW
    • NEW

    Description

      Description of problem:
      Fluentd continue to push logs even if the Elasticsearch cluster is unhealthy

      Version-Release number of selected component (if applicable):
      cluster-logging.5.2.2-21

      How reproducible:
      100%

      Steps to Reproduce:
      1. Deploy a default Clusterlogging instance
      2. Delete the ES pods "oc delete pod -l component=elasticsearch"
      3. Verify on the fluentd logs that the log collector component was flushing while the ES cluster wasn't ready.

      Actual results:
      In the customer environment, this caused ES to never start properly because overwhelmed by logs. Until they completely stoped all Fluentd instances.

      Expected results:
      Fluentd pods should not push data until ES is healthy.

      Additional info:
      A workaround exists and is to stop all the Fluentd instances to allow the ES cluster to start properly.

      Attachments

        Activity

          People

            Unassigned Unassigned
            rhn-support-cpassare Christian Passarelli
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: