Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1412

MS SQL Server Debezium Connector fails on adding columns to the source tables

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • None
    • 0.9.5.Final
    • sqlserver-connector
    • None
    • Hide

      1. Source DB: MS SQL Server
      2. CDC enabled on a table
      3. A new column added to the table without disabling the CDC on that table
      4. The below exception occurs

      Even though the CDC table is not refreshed to have the newly added column, the change events were getting captured in the CDC table without the new column. But Debezium connector stopped working with the below exception - looks like the new schema got updated in the audit topic and that caused the failure. Shouldn't the connector continue with the old schema, until the CDC table is refreshed to reflect the new schema changes?

      If we disable and enable the CDC for the table the issue gets resolved.

      Show
      1. Source DB: MS SQL Server 2. CDC enabled on a table 3. A new column added to the table without disabling the CDC on that table 4. The below exception occurs Even though the CDC table is not refreshed to have the newly added column, the change events were getting captured in the CDC table without the new column. But Debezium connector stopped working with the below exception - looks like the new schema got updated in the audit topic and that caused the failure. Shouldn't the connector continue with the old schema, until the CDC table is refreshed to reflect the new schema changes? If we disable and enable the CDC for the table the issue gets resolved.

    Description

      Connector fails with the below exception . Details in steps to reproduce

      [2019-07-19 22:33:26,736] ERROR Producer failure (io.debezium.pipeline.ErrorHandler)
      java.lang.ArrayIndexOutOfBoundsException: 23
      at io.debezium.relational.TableSchemaBuilder.lambda$createValueGenerator$2(TableSchemaBuilder.java:216)
      at io.debezium.relational.TableSchema.valueFromColumnData(TableSchema.java:145)
      at io.debezium.relational.RelationalChangeRecordEmitter.emitUpdateRecord(RelationalChangeRecordEmitter.java:89)
      at io.debezium.relational.RelationalChangeRecordEmitter.emitChangeRecords(RelationalChangeRecordEmitter.java:46)
      at io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:125)
      at io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.lambda$execute$1(SqlServerStreamingChangeEventSource.java:203)
      at io.debezium.jdbc.JdbcConnection.prepareQuery(JdbcConnection.java:493)
      at io.debezium.connector.sqlserver.SqlServerConnection.getChangesForTables(SqlServerConnection.java:143)
      at io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.execute(SqlServerStreamingChangeEventSource.java:137)
      at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:86)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)

      Attachments

        Activity

          People

            Unassigned Unassigned
            tiginsamuel Tigin Samuel (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: