Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7615

Enhanced event timestamp precision combined with ExtractNewRecordState not working

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • 2.6.0.CR1
    • 2.6.0.Beta1
    • core-library
    • None

    Description

      What Debezium connector do you use and what version?

      Oracle Connector 2.6.0.Beta1

      What is the connector configuration?

      "transforms": "changes,moveHeadersToValue,unwrap",
      "transforms.changes.type": "io.debezium.transforms.ExtractChangedRecordState", 
      "transforms.changes.header.changed.name": "Changed", 
      "transforms.moveHeadersToValue.type": "io.debezium.transforms.HeaderToValue", 
      "transforms.moveHeadersToValue.headers": "Changed", 
      "transforms.moveHeadersToValue.fields": "changes", 
      "transforms.moveHeadersToValue.operation": "move", 
      "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState", 
      "transforms.unwrap.add.fields": "source.table:META_SRC_TABLENAME,source.ts_ms:META_SRC_TS,source.ts_us:META_SRC_US,ts_us:META_US,ts_ns:META_NS,op:META_SRC_OP,source.scn:META_SRC_SCN,source.snapshot:META_SRC_SNAPSHOT,source.user_name:META_SRC_USER,changes:META_SRC_CHANGES", 
      "transforms.unwrap.delete.handling.mode": "rewrite", 
      "transforms.unwrap.drop.tombstones": "false",

      I think the rest of the configuration is not required.

      What is the captured database version and mode of depoyment?

      Oracle 19c CDB on prem
      In Kafka Connect we run Java OpenJDK11 (so nanoseconds aren't supported yet)

      What behaviour do you expect?

      "ts_ms": 1709889596702, 
      "ts_us": 1709889596702080, 
      "ts_ns": 1709889596702080000

      The envelope's new ts_us should deliver timestamps with microseconds when used with ExtractNewRecordState. 

      What behaviour do you see?

      "__deleted": "false", 
      "__META_SRC_TABLENAME": "ANTRFRAGENANTW", 
      "__META_SRC_TS_MS": 1709891779000,
      "__META_SRC_TS_US": 1709891779000000,
      "__META_SRC_TS_NS": 1709891779000000000, 
      "__META_TS_MS": 1709891800550, 
      "__META_TS_US": 1709891779000000, 
      "__META_TS_NS": 1709891779000000000, 
      "__META_SRC_OP": "r", 
      "__META_SRC_SCN": "328638752686", 
      "__META_SRC_SNAPSHOT": "true"

      Combined with the ExtractNewRecordState SMT the envelope's ts_us and ts_ns have the same values as in the source's ts_us and ts_ns. 

      The envelope's ts_ms seems to be correct. Without the ExtractNewRecordState all works fine.

       

      Attachments

        Activity

          People

            ccranfor@redhat.com Chris Cranford
            renerh René Rütter
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: