Spike: Mobile app level and SDK level metrics

Possible architectures: https://docs.google.com/drawings/d/16bVwXE2OV0M6RicihAyX97DFDY3ycEVzyRdySXKi7gI/edit

Before starting the discussion, here is a good summary of what time series data and regular data means:

ElasticSearch: Supports both.
InfluxDB: Time series data only. No update operations
PostgreSQL: supports both (no hands-on experiment done by the team)
Prometheus: time series data only. Not viable as a storage option anyway.

Another note is that, the trying to building a knowledge base data using queries is not natural thing to do on a time series database. Aggregations would be super-complex (if even possible) and slow (due to very long time range and too many matching documents).

Possible architectures we are going to pursue:

1. As we’re going to store time series and regular data in the same storage, we can use ElasticSearch or PostgreSQL.
Grafana is able to talk to both storage options.
Very very little business logic in the service component. Acts more like a proxy.

Although using Kibana is perfectly fine, we are going to investigate using Grafana as well.
Sub-spikes for these are AEROGEAR-2002 and AEROGEAR-2004

2. skipped
3. skipped

4. Our service writes time series data to Prometheus and Prometheus writes them to a PostgreSQL.
Our service also talks to PostgreSQL directly for adding regular data.
PostgreSQL is the only option here as it is the only DB that supports time series data and regular data and Prometheus supports it.

Sub-spikes: AEROGEAR-2003 and AEROGEAR-2005

We also need to check out the resource consumption of these architectures: AEROGEAR-2006

Another thing to check is if it makes sense to use fluentd/logstash: AEROGEAR-2007

Things eliminated

MongoDB:

• can be used as a time series database, but a bit painful
• no support on Grafana or Kibana

InfluxDB:

• as it doesn't support update operations, it can only be used for time series data
• it might still be possible to use InfluxDB for time series data, and some other DB for regular data. However we don't want to pursue this kind of architecture at the moment.