-
Bug
-
Resolution: Unresolved
-
Major
-
2.5.4.Final, 2.6.1.Final
-
None
-
False
-
None
-
False
Bug report
An incremental snapshot will fail with a NullPointerException if the provided surrogate key ("surrogate-key") doesn't exist in the provided tables ("data-collections").
io.debezium.DebeziumException: Database error while executing incremental snapshot for table 'DataCollection{id=schemaname.tablename, additionalCondition=, surrogateKey=SURROGATE_KEY}' Caused by: java.lang.NullPointerException at io.debezium.pipeline.source.snapshot.incremental.AbstractChunkQueryBuilder.lambda$buildMaxPrimaryKeyQuery$3(AbstractChunkQueryBuilder.java:225)
What Debezium connector do you use and what version?
Debezium for MySQL 2.6.1.Final
What is the connector configuration?
-
What is the captured database version and mode of depoyment?
Database: Amazon RDS for MariaDB 10.5.21
Deployment of Debezium: MSK Connect
What behaviour do you expect?
The snapshot process should emit a proper error message and not start if the provided surrogate key doesn't exist.
What behaviour do you see?
The snapshot process fails with a NullPointerException if the provided surrogate key doesn't exist.
Do you see the same behaviour using the latest released Debezium version?
Yes, also happens on main branch.
Do you have the connector logs, ideally from start till finish?
io.debezium.DebeziumException: Database error while executing incremental snapshot for table 'DataCollection{id=schemaname.tablename, additionalCondition=, surrogateKey=SURROGATE_KEY}' at io.debezium.pipeline.source.snapshot.incremental.AbstractIncrementalSnapshotChangeEventSource.readChunk(AbstractIncrementalSnapshotChangeEventSource.java:337) at io.debezium.pipeline.source.snapshot.incremental.AbstractIncrementalSnapshotChangeEventSource.addDataCollectionNamesToSnapshot(AbstractIncrementalSnapshotChangeEventSource.java:451) at io.debezium.pipeline.signal.actions.snapshotting.ExecuteSnapshot.arrived(ExecuteSnapshot.java:78) at io.debezium.pipeline.signal.SignalProcessor.processSignal(SignalProcessor.java:191) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:720) at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:274) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177) at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) at io.debezium.pipeline.signal.SignalProcessor.lambda$processSourceSignal$4(SignalProcessor.java:155) at io.debezium.pipeline.signal.SignalProcessor.executeWithSemaphore(SignalProcessor.java:165) at io.debezium.pipeline.signal.SignalProcessor.processSourceSignal(SignalProcessor.java:149) at io.debezium.pipeline.EventDispatcher$2.changeRecord(EventDispatcher.java:290) at io.debezium.relational.RelationalChangeRecordEmitter.emitCreateRecord(RelationalChangeRecordEmitter.java:79) at io.debezium.relational.RelationalChangeRecordEmitter.emitChangeRecords(RelationalChangeRecordEmitter.java:47) at io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:271) at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.lambda$handleInsert$5(MySqlStreamingChangeEventSource.java:713) at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleChange(MySqlStreamingChangeEventSource.java:781) at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleInsert(MySqlStreamingChangeEventSource.java:710) at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.lambda$execute$20(MySqlStreamingChangeEventSource.java:854) at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleEvent(MySqlStreamingChangeEventSource.java:285) at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.lambda$execute$30(MySqlStreamingChangeEventSource.java:880) at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1263) at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1089) at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:648) at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:949) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.NullPointerException at io.debezium.pipeline.source.snapshot.incremental.AbstractChunkQueryBuilder.lambda$buildMaxPrimaryKeyQuery$3(AbstractChunkQueryBuilder.java:225) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) at java.base/java.util.Collections$2.tryAdvance(Collections.java:4747) at java.base/java.util.Collections$2.forEachRemaining(Collections.java:4755) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) eyQuery(AbstractChunkQueryBuilder.java:226) at io.debezium.pipeline.source.snapshot.incremental.AbstractIncrementalSnapshotChangeEventSource.readChunk(AbstractIncrementalSnapshotChangeEventSource.java:276) ... 34 more