Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-561

Provide option to flatten structs in MongoDB unwrapping SMT

    XMLWordPrintable

Details

    • Feature Request
    • Resolution: Done
    • Major
    • 0.8.0.Beta1
    • None
    • mongodb-connector
    • None
    • 0
    • 0% 0%

    Description

      If there's a complex id type or other struct within a MongoDB document, the UnwrapFromMongoDbEnvelope will correctly restore a Struct from that property, but e.g. the JDBC sink connector only can deal with flat structures.

      There should be an option for the SMT for flattening out such structs by concatenating the fields into plain properties, using a (configurable) character. E.g. when receiving a document with an ide like this:

      _id : { comp: 32, cust: NumberLong("1008") }
      

      The SMT should optionally emit the following two fields:

      id_comp : 32, id_cust: 1008
      

      So we'd have two new options for the SMT:

      • flatten_structs: boolean, defaults to false
      • flattened_structs_field_concat_char: string, defaults to "_"

      Note that this option cannot be used to deal with arrays.

      Attachments

        Issue Links

          Activity

            People

              spolavarapu53 Sairam Polavarapu (Inactive)
              gunnar.morling Gunnar Morling
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: