Uploaded image for project: 'Hawkular Metrics'
  1. Hawkular Metrics
  2. HWKMETRICS-613

Add mechanism to automatically delete inactive metrics

    XMLWordPrintable

    Details

    • Type: Enhancement
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 0.26.0
    • Component/s: None
    • Labels:
      None

      Description

      This is a follow up to HWKMETRICS-612. Having APIs to delete metrics is necessary but not sufficient because we cannot guarantee that clients will use the APIs. The goal here is to prevent situations like HWKMETRICS-606. We can use a simple heuristic to decide if a metric is no longer active. For example, if it has been at least N days since we received any data points for a metrics and if the metric has no data points, then the metric is inactive and it can be deleted.

      As per the suggestion from Stefan Negrea we will need a new index table to track which metrics are active. We will update it whenever we store data points for a metric. The schema will look something like:

      CREATE TABLE metrics_activity_idx (
          tenant_id text,
          type tinyint,
          metric text,
          last_active timestamp,
         PRIMARY KEY ((tenant_id, type), metric)
      )
      

      We do not need to store millisecond timestamps. It should be sufficient to store just the day in which case we could use 4 or even 2 bytes instead of an 8 byte timestamp.

      We will have a scheduled job that will check this table to find inactive metrics. When we have determined that a metric can be deleted, we will first have to query metrics_idx to get the set of tags so we know which partitions in metrics_tags_idx to update. Lastly, we need to remove the metric from metrics_activity_idx.

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  nstefan Stefan Negrea
                  Reporter:
                  john.sanda John Sanda
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  1 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: