Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-561

Provide option to flatten structs in MongoDB unwrapping SMT

    XMLWordPrintable

    Details

      Description

      If there's a complex id type or other struct within a MongoDB document, the UnwrapFromMongoDbEnvelope will correctly restore a Struct from that property, but e.g. the JDBC sink connector only can deal with flat structures.

      There should be an option for the SMT for flattening out such structs by concatenating the fields into plain properties, using a (configurable) character. E.g. when receiving a document with an ide like this:

      _id : { comp: 32, cust: NumberLong("1008") }
      

      The SMT should optionally emit the following two fields:

      id_comp : 32, id_cust: 1008
      

      So we'd have two new options for the SMT:

      • flatten_structs: boolean, defaults to false
      • flattened_structs_field_concat_char: string, defaults to "_"

      Note that this option cannot be used to deal with arrays.

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  spolavarapu53 Sairam Polavarapu
                  Reporter:
                  gunnar.morling Gunnar Morling
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  3 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: