Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7765

Sql-Server connector fails after initial start / processed record on subsequent starts

    XMLWordPrintable

Details

    • Critical

    Description

      What Debezium connector do you use and what version?

      • debezium-server-dist-2.6.0.Final-runner.jar
      • debezium-connector-sqlserver-2.6.0.Final.jar

      What is the connector configuration?

      Nothing special
       

      debezium.sink.type=eventhubs debezium.sink.eventhubs.hubname=cdc-3 debezium.sink.eventhubs.connectionstring=... debezium.sink.eventhubs.partitionid=0 
      debezium.source.topic.prefix=cdc debezium.source.connector.class=io.debezium.connector.sqlserver.SqlServerConnector 
      debezium.source.offset.storage.file.filename=data/offsets.dat debezium.source.offset.flush.interval.ms=0 
      debezium.source.schema.history.internal=io.debezium.storage.file.history.FileSchemaHistory debezium.source.schema.history.internal.file.filename=data/schema_history.dat 
      debezium.source.database.hostname=10.x.x.x debezium.source.database.port=1433 debezium.source.database.user=cdc_debezium debezium.source.database.password=... debezium.source.database.names=... debezium.source.database.encrypt=false 
      debezium.source.snapshot.mode=initial debezium.source.snapshot.lock.timeout.ms=20000 
      debezium.source.table.include.list=dbo.T1,cst.T2 
      debezium.source.snapshot.select.statement.overrides=dbo.T1,cst.T2 
      debezium.source.snapshot.select.statement.overrides.dbo.T1=SELECT * FROM [dbo].[T1] WHERE Flags & 524288 = 0 ORDER BY CustomerNumber debezium.source.snapshot.select.statement.overrides.cst.T2=SELECT * FROM [cst].[T2] ORDER BY CustomerNumber 
      quarkus.log.console.json=false quarkus.log.file.enable=true quarkus.log.level=INFO

       
      What is the captured database version and mode of depoyment?
      MS Sql Server on premise

      What behaviour do you expect?

      DBZ server restart should not fail after it started successfully

      What behaviour do you see?

      After initial startup and handling some cdc events (one?), DBZ server cannot be restarted anymore as it fails with some internal sql error. 

      Do you see the same behaviour using the latest relesead Debezium version?

      Yes. Just upgraded to 2.6.0

      Do you have the connector logs, ideally from start till finish?

      2024-04-09 13:30:07,311 INFO  [io.deb.con.CommonConnectorConfig] (pool-18-thread-1) Loading the custom source info struct maker plugin: io.debezium.connector.sqlserver.SqlServerSourceInfoStructMaker2024-04-09 13:30:07,326 INFO  [io.deb.con.CommonConnectorConfig] (pool-18-thread-1) Loading the custom topic naming strategy plugin: io.debezium.schema.SchemaTopicNamingStrategy2024-04-09 13:30:07,502 INFO  [io.deb.con.com.BaseSourceTask] (pool-18-thread-1) Found previous partition offset SqlServerPartition [sourcePartition={server=cdc_xxx, database=XXX}]: {event_serial_no=2, commit_lsn=0053b6f6:0000db28:0009, change_lsn=0053b6f6:0000db28:0002}2024-04-09 13:30:07,564 ERROR [io.deb.emb.asy.AsyncEmbeddedEngine] (pool-13-thread-1) 1 task(s) out of 1 failed to start.2024-04-09 13:30:07,564 ERROR [io.deb.emb.asy.AsyncEmbeddedEngine] (pool-13-thread-1) Engine has failed with : java.util.concurrent.ExecutionException: io.debezium.DebeziumException: Unable to get last available log position    at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)    at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)    at io.debezium.embedded.async.AsyncEmbeddedEngine.startSourceTasks(AsyncEmbeddedEngine.java:411)    at io.debezium.embedded.async.AsyncEmbeddedEngine.run(AsyncEmbeddedEngine.java:202)    at io.debezium.server.DebeziumServer.lambda$start$1(DebeziumServer.java:178)    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)    at java.base/java.lang.Thread.run(Thread.java:1583)Caused by: io.debezium.DebeziumException: Unable to get last available log position    at io.debezium.connector.sqlserver.SqlServerConnection.validateLogPosition(SqlServerConnection.java:756)    at io.debezium.jdbc.JdbcConnection.isLogPositionAvailable(JdbcConnection.java:1663)    at io.debezium.connector.common.BaseSourceTask.validateAndLoadSchemaHistory(BaseSourceTask.java:102)    at io.debezium.connector.sqlserver.SqlServerConnectorTask.start(SqlServerConnectorTask.java:108)    at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:227)    at io.debezium.embedded.async.AsyncEmbeddedEngine.lambda$startSourceTasks$2(AsyncEmbeddedEngine.java:394)    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)    ... 3 moreCaused by: com.microsoft.sqlserver.jdbc.SQLServerException: The SELECT permission was denied on the object 'fn_dblog', database 'mssqlsystemresource', schema 'sys'.    at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:259)    at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1695)    at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:920)    at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:814)    at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7675)    at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:4137)    at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:272)    at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:246)    at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeQuery(SQLServerStatement.java:736)    at io.debezium.jdbc.JdbcConnection.queryAndMap(JdbcConnection.java:645)    at io.debezium.jdbc.JdbcConnection.queryAndMap(JdbcConnection.java:514)    at io.debezium.connector.sqlserver.SqlServerConnection.singleOptionalValue(SqlServerConnection.java:761)    at io.debezium.connector.sqlserver.SqlServerConnection.validateLogPosition(SqlServerConnection.java:746)    ... 11 more 

      Attachments

        Activity

          People

            rh-ee-mvitale Mario Fiore Vitale
            devployment Daniel Weinand
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: