Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-500

TIME columns are emitted as NanoTime instead of Time/MicroTime

    XMLWordPrintable

Details

    • Hide

      Integration tests already show that behavior:
      https://github.com/debezium/debezium/blob/master/debezium-connector-postgres/src/test/java/io/debezium/connector/postgresql/AbstractRecordsProducerTest.java#L201
      is a test that runs in the default `time.precision.mode=adaptive`. The related table structure is based on:
      https://github.com/debezium/debezium/blob/master/debezium-connector-postgres/src/test/resources/postgres_create_tables.ddl#L13
      `ti TIME` should have a precision of *6* (max possible precision and the default when not specified). the value 6 is in scale(), but not in length().

      This is why in AbstractRecordsProducerTest.java#L201 it uses NanoTime.builder() instead of MicroTime which would be appropriate for precision = 6.

      Show
      Integration tests already show that behavior: https://github.com/debezium/debezium/blob/master/debezium-connector-postgres/src/test/java/io/debezium/connector/postgresql/AbstractRecordsProducerTest.java#L201 is a test that runs in the default `time.precision.mode=adaptive`. The related table structure is based on: https://github.com/debezium/debezium/blob/master/debezium-connector-postgres/src/test/resources/postgres_create_tables.ddl#L13 `ti TIME` should have a precision of * 6 * (max possible precision and the default when not specified). the value 6 is in scale(), but not in length(). This is why in AbstractRecordsProducerTest.java#L201 it uses NanoTime.builder() instead of MicroTime which would be appropriate for precision = 6.

    Description

      Postgres always uses NanoTime implementation in adaptive mode for time fields because Column#length and Column#scale aren't assigned and used consistently for time fields.

      The reason for this is, that `io.debezium.relational.Column#length` and `io.debezium.relational.Column#scale` are assigned and used in an inconsistent way for time fields by different connector implementations.
      ValueConverters like the generic JDBCValueConverters use `io.debezium.relational.Column#length` to get the precision of time fields. That's correct for MySQL connector implementation, but Postgres seems to always have a value of 15 for length and the fractional part size / precision is stored in scale. This leads to the issue that Postgres time fields in adaptive mode are always encoded using `io.debezium.time.NanoTime` instead of working adaptively, like expected and suggested.

      imho postgres is eventually behaving well, because it could make sense to have the precision in scale. but according to the javadoc in io.debezium.relational.Column interface,it should go to #length().

      Other types like timestamp could also be affected, because JDBCValueConverters always use column.length() there too and probably value assignment is broken for these Column instances too.

      Attachments

        Activity

          People

            Unassigned Unassigned
            rk3rn3r René Kerner
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: