Data Sync Proof of Concept

Data Sync Server - Chat Proof of Concept

Server repo: https://github.com/aerogear/aerogear-data-sync-server
Fork of Android Apollo Client repo: https://github.com/aerogear/apollo-android

Phase 1 - Initial End to End

Technical Goals

• get a working end to end graphql client & server, storing data in a real database
• become familiar with graphql concepts such as 'schema', 'queries', 'mutations', 'subscriptions'
• understand how a resolver integrates between a graphql request and a database

Server

• The server has a hardcoded graphql schema file with a single entity 'Messages'
  • id (some unique server generated id)
  • content (text content of a message)
  • user_id (unique user id, can be the device client id initially)
  • timestamp (server timestamp for when the message was saved to the server)
• The schema file has queries defined for:
  • create Message
  • list Message
• Queries have hardcoded resolver implementations for postgreSQL

Client

• The Android showcase app has a new menu item, 'Chat'
• The 'Chat' screen shows a list view of all messages, ordered by timestamp, and the user_id beside each message
• The screen also has a text input and 'Send' button at the bottom, which creates a new 'Message'
  • a new 'Message' has the following fields set by the client:
    • content
    • user_id
• The 'Message' model, and associated 'create' and 'list' queries will be generated using the apollo gradle plugin, based on the graphql schema file
• The app has a 'subscription' to the 'Message' create query. When a new 'Message' is created by any app user, connected apps will be notified, and their UI updated with the latest messages on the 'Chat' screen.

Phase 2 - Dynamic Resolvers

Technical Goals

• develop generic postgreSQL resolver(s) that don't require schema knowledge ahead of time for 'create' and 'list' queries.
• have an API for setting what shema the server should use

Server

• Add a new API to the server that allows setting the schema from a file at runtime
  • the API should accept a graphql schema file, with entities, queries etc.. defined
  • only queries with certain names should be accepted at this time i.e. create_<entity> or list_<entity> e.g. create_Message, list_Message. This is to facilitate the ability to dynamically resolve data
• Based on defined queries in the schema file set via the API, the server should resolve them dynamically against postgreSQL.

Client

• no changes

NOTE: 'update' is actually a 'mutation', but for simplicity its referred to as a 'query' here.

Phase 3 - Offline updates

Technical Goals

• Develop a way for applying updates on the Client when its offline, and have them be applied on the server when online again.

Server

• potentially no changes

Client

• Modify the Apollo GraphQL Client to add a mutation strategy for applying updates in the App first, to a local db, in cases where the App is not connected to the server
• Updates should be considered applied from the clients point of view as soon as they are applied locally
• Detect when the App is reconnected to the server, and send the locally applied updates to the server.
  • This needs a lot of though on how best to manage the merging of new server changes with local db changes. e.g. should they be merged on the clients local db, or should they be nuked/overwritten with the latest changes from the server? Is a change/transaction log needed to be kept/replicated?

Previous description content below

• Research on the current offering by other vendors:
  • Firebase
  • Amazon AppSync
  • Apollo Graphql
  • DeepStream.io

We want to find out:

• high level architecture of their products
• how data gets synced
• offline support and how it is implemented
• query support (done on the client or the server)
• what type of data store they are using on clients
• authentication & authorisation support and how it's implemented
• integrating with other enterprise backend datastore and how they are implemented
• server-to-server sync support

We have done this for couchbase mobile (https://issues.jboss.org/browse/RHMAP-19616?jql=text%20~%20%22couchbase%20db%22). We can use it as an example.

Branch of Android Showcase App for graphql spikes

• https://github.com/aerogear/android-showcase-template/tree/graphql-spike

Branch of a graphql server for server spikes

• https://github.com/david-martin/starwars-server/tree/notes-spike

Some open questions

• Can you link directly to a specific Grafana dashboard (e.g. for linking to grafana from openshift ui)?
• Is there a default prometheus exporter for graphql?
• Can metadata be associated with a schema? (e.g. so you could define a role users should have to access an entity)