[DBZ-3193] DML parser IndexOutOfRangeException with where-clause using "IS NULL"

Type: Bug
Resolution: Done
Priority: Blocker
Fix Version/s: 1.4.2.Final, 1.5.0.Beta2
Affects Version/s: 1.5.0.Beta1
Component/s: oracle-connector
Labels:
None

Blocked:
False
Ready:
False
Release Note Text:
Undefined
Git Pull Request:
https://github.com/debezium/debezium-incubator/pull/272
Market:

SFDC Cases Counter:
SFDC Cases Open:
SFDC Cases Links:

The parseUpdate method of logminerdmlparser class throws indexoutofrange exception.

If the update statement where clause contains NULL column values, the values collection do not contains those values but the column name collection does contains the names.

So in my case the column names collection has 32 columns and column values collection has 18 values as NULLS are filtered out.

Both collections are iterated under the same and indexer i in for loop and since they have different sizes - this results into the error. Please refer the screen shot.

Either we should populate the NULL Values in the column values collection or we should not add the column names in names collection with null values.

Example DML:

'update "SCOTT"."MTL_MATERIAL_TRANSACTIONS" set "SUBINVENTORY_CODE" = 'TEST2' 
where "TRANSACTION_ID" = '22805' and "CREATION_DATE" = '01-MAR-12 20:29:47' 
and "LAST_UPDATE_DATE" = '01-MAR-12 20:33:07' and "TRANSACTION_DATE" = '01-MAR-12 20:28:26' 
and "INVENTORY_ITEM_ID" = '3150' and "ORGANIZATION_ID" = '7274'
and "SUBINVENTORY_CODE" = 'TEST1' and "LOCATOR_ID" = '3544' and "TRANSACTION_TYPE_ID" = '41' 
and "TRANSACTION_ACTION_ID" = '27' and "TRANSACTION_SOURCE_TYPE_ID" = '6' 
and "TRANSACTION_SOURCE_NAME" IS NULL and "TRANSACTION_QUANTITY" = '1' 
and "TRANSACTION_UOM" = 'EA' and "TRANSACTION_REFERENCE" IS NULL and "REASON_ID" IS NULL 
and "TRX_SOURCE_LINE_ID" IS NULL and "RCV_TRANSACTION_ID" IS NULL 
and "COMPLETION_TRANSACTION_ID" IS NULL and "SOURCE_CODE" = 'Manual' 
and "SOURCE_LINE_ID" = '121056' and "TRANSFER_ORGANIZATION_ID" IS NULL 
and "TRANSFER_SUBINVENTORY" IS NULL and "TRANSFER_LOCATOR_ID" IS NULL 
and "SHIPMENT_NUMBER" IS NULL and "LPN_ID" IS NULL and "TRANSFER_LPN_ID" IS NULL 
and "CONTENT_LPN_ID" IS NULL and "PICK_SLIP_NUMBER" IS NULL and "MOVE_ORDER_LINE_ID" IS NULL;'

- - Sort By Name
  - Sort By Date
  - Ascending
  - Descending
  - Thumbnails
  - List
  - Download All

Capture.PNG
87 kB
2021/02/24 2:37 PM

is duplicated by

DBZ-3214 Mining session stops with DmlParserException: Failed to parse update DML

Closed

is related to

DBZ-3235 LogMiner parse failure with Update DML with no where condition

Closed

Chris Cranford added a comment - 2021/03/04 9:03 AM

jpechane and ant0nk, I've logged ~~DBZ-3235~~, lets discuss as well as add the DDL there.

Chris Cranford added a comment - 2021/03/04 9:03 AM jpechane and ant0nk , I've logged DBZ-3235 , lets discuss as well as add the DDL there.

Jiri Pechanec added a comment - 2021/03/04 1:46 AM

ant0nk Also could you please create a separate Jira for it?

Jiri Pechanec added a comment - 2021/03/04 1:46 AM ant0nk Also could you please create a separate Jira for it?

Jiri Pechanec added a comment - 2021/03/04 1:46 AM

ant0nk Could you please share DDL for the table? Do you know which DML operation has triggered this log mining record?

Jiri Pechanec added a comment - 2021/03/04 1:46 AM ant0nk Could you please share DDL for the table? Do you know which DML operation has triggered this log mining record?

Anton Kondratev (Inactive) added a comment - 2021/03/04 1:41 AM

ccranfor@redhat.com one of our databases sent in logminer update without WHERE clause and parser could not process it:

ERROR Mining session stopped due to the {} (io.debezium.connector.oracle.logminer.LogMinerHelper:561)
io.debezium.connector.oracle.logminer.parser.DmlParserException: DML statement couldn't be parsed. Please open a Jira issue with the statement 'update "IBS"."Z#M_D" set "SN" = '11', "SU" = '66', "C_ACC_KT" = NULL, "C_NUM_KT" = NULL;'. You can set internal.log.mining.dml.parser='legacy' as a workaround until the parse error is fixed.
                at io.debezium.connector.oracle.logminer.LogMinerQueryResultProcessor.parse(LogMinerQueryResultProcessor.java:293)
                at io.debezium.connector.oracle.logminer.LogMinerQueryResultProcessor.processResult(LogMinerQueryResultProcessor.java:199)
                at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:182)
                at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:140)
                at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:113)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
                at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at java.lang.Thread.run(Thread.java:748)
Caused by: io.debezium.connector.oracle.logminer.parser.DmlParserException: Failed to parse update DML: 'update "IBS"."Z#M_D" set "SN" = '11', "SU" = '66', "C_ACC_KT" = NULL, "C_NUM_KT" = NULL;'
                at io.debezium.connector.oracle.logminer.parser.LogMinerDmlParser.parseUpdate(LogMinerDmlParser.java:176)
                at io.debezium.connector.oracle.logminer.parser.LogMinerDmlParser.parse(LogMinerDmlParser.java:81)
                at io.debezium.connector.oracle.logminer.LogMinerQueryResultProcessor.parse(LogMinerQueryResultProcessor.java:283)
                ... 9 more
Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: 108
                at java.lang.String.substring(String.java:1963)
                at io.debezium.connector.oracle.logminer.parser.LogMinerDmlParser.parseWhereClause(LogMinerDmlParser.java:454)
                at io.debezium.connector.oracle.logminer.parser.LogMinerDmlParser.parseUpdate(LogMinerDmlParser.java:149)
                ... 11 more
[2021-03-03 18:55:21,984] ERROR Producer failure (io.debezium.pipeline.ErrorHandler:31)
io.debezium.connector.oracle.logminer.parser.DmlParserException: DML statement couldn't be parsed. Please open a Jira issue with the statement 'update "IBS"."Z#M_D" set "SN" = '11', "SU" = '66', "C_ACC_KT" = NULL, "C_NUM_KT" = NULL;'. You can set internal.log.mining.dml.parser='legacy' as a workaround until the parse error is fixed.
                at io.debezium.connector.oracle.logminer.LogMinerQueryResultProcessor.parse(LogMinerQueryResultProcessor.java:293)
                at io.debezium.connector.oracle.logminer.LogMinerQueryResultProcessor.processResult(LogMinerQueryResultProcessor.java:199)
                at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:182)
                at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:140)
                at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:113)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
                at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at java.lang.Thread.run(Thread.java:748)
Caused by: io.debezium.connector.oracle.logminer.parser.DmlParserException: Failed to parse update DML: 'update "IBS"."Z#M_D" set "SN" = '11', "SU" = '66', "C_ACC_KT" = NULL, "C_NUM_KT" = NULL;'
                at io.debezium.connector.oracle.logminer.parser.LogMinerDmlParser.parseUpdate(LogMinerDmlParser.java:176)
                at io.debezium.connector.oracle.logminer.parser.LogMinerDmlParser.parse(LogMinerDmlParser.java:81)
                at io.debezium.connector.oracle.logminer.LogMinerQueryResultProcessor.parse(LogMinerQueryResultProcessor.java:283)
                ... 9 more
Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: 108
                at java.lang.String.substring(String.java:1963)
                at io.debezium.connector.oracle.logminer.parser.LogMinerDmlParser.parseWhereClause(LogMinerDmlParser.java:454)
                at io.debezium.connector.oracle.logminer.parser.LogMinerDmlParser.parseUpdate(LogMinerDmlParser.java:149)
                ... 11 more

Isn't it suprising that V$LOGMINER_CONTENTS can output such updates at all given that Debezium can collect Before-state only using WHERE-clause?

Anton Kondratev (Inactive) added a comment - 2021/03/04 1:41 AM ccranfor@redhat.com one of our databases sent in logminer update without WHERE clause and parser could not process it: ERROR Mining session stopped due to the {} (io.debezium.connector.oracle.logminer.LogMinerHelper:561) io.debezium.connector.oracle.logminer.parser.DmlParserException: DML statement couldn 't be parsed. Please open a Jira issue with the statement ' update "IBS" . "Z#M_D" set "SN" = '11' , "SU" = '66' , "C_ACC_KT" = NULL, "C_NUM_KT" = NULL; '. You can set internal.log.mining.dml.parser=' legacy' as a workaround until the parse error is fixed. at io.debezium.connector.oracle.logminer.LogMinerQueryResultProcessor.parse(LogMinerQueryResultProcessor.java:293) at io.debezium.connector.oracle.logminer.LogMinerQueryResultProcessor.processResult(LogMinerQueryResultProcessor.java:199) at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:182) at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:140) at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:113) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748) Caused by: io.debezium.connector.oracle.logminer.parser.DmlParserException: Failed to parse update DML: 'update "IBS" . "Z#M_D" set "SN" = ' 11 ', "SU" = ' 66 ', "C_ACC_KT" = NULL, "C_NUM_KT" = NULL;' at io.debezium.connector.oracle.logminer.parser.LogMinerDmlParser.parseUpdate(LogMinerDmlParser.java:176) at io.debezium.connector.oracle.logminer.parser.LogMinerDmlParser.parse(LogMinerDmlParser.java:81) at io.debezium.connector.oracle.logminer.LogMinerQueryResultProcessor.parse(LogMinerQueryResultProcessor.java:283) ... 9 more Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: 108 at java.lang. String .substring( String .java:1963) at io.debezium.connector.oracle.logminer.parser.LogMinerDmlParser.parseWhereClause(LogMinerDmlParser.java:454) at io.debezium.connector.oracle.logminer.parser.LogMinerDmlParser.parseUpdate(LogMinerDmlParser.java:149) ... 11 more [2021-03-03 18:55:21,984] ERROR Producer failure (io.debezium.pipeline.ErrorHandler:31) io.debezium.connector.oracle.logminer.parser.DmlParserException: DML statement couldn 't be parsed. Please open a Jira issue with the statement ' update "IBS" . "Z#M_D" set "SN" = '11' , "SU" = '66' , "C_ACC_KT" = NULL, "C_NUM_KT" = NULL; '. You can set internal.log.mining.dml.parser=' legacy' as a workaround until the parse error is fixed. at io.debezium.connector.oracle.logminer.LogMinerQueryResultProcessor.parse(LogMinerQueryResultProcessor.java:293) at io.debezium.connector.oracle.logminer.LogMinerQueryResultProcessor.processResult(LogMinerQueryResultProcessor.java:199) at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:182) at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:140) at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:113) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748) Caused by: io.debezium.connector.oracle.logminer.parser.DmlParserException: Failed to parse update DML: 'update "IBS" . "Z#M_D" set "SN" = ' 11 ', "SU" = ' 66 ', "C_ACC_KT" = NULL, "C_NUM_KT" = NULL;' at io.debezium.connector.oracle.logminer.parser.LogMinerDmlParser.parseUpdate(LogMinerDmlParser.java:176) at io.debezium.connector.oracle.logminer.parser.LogMinerDmlParser.parse(LogMinerDmlParser.java:81) at io.debezium.connector.oracle.logminer.LogMinerQueryResultProcessor.parse(LogMinerQueryResultProcessor.java:283) ... 9 more Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: 108 at java.lang. String .substring( String .java:1963) at io.debezium.connector.oracle.logminer.parser.LogMinerDmlParser.parseWhereClause(LogMinerDmlParser.java:454) at io.debezium.connector.oracle.logminer.parser.LogMinerDmlParser.parseUpdate(LogMinerDmlParser.java:149) ... 11 more Isn't it suprising that V$LOGMINER_CONTENTS can output such updates at all given that Debezium can collect Before-state only using WHERE-clause?

Chris Cranford added a comment - 2021/03/01 9:50 AM

integratemukesh, I believe Jiri included this in 1.4.2.Final which has been released. If you want to use 1.5, then you'll need to grab the nightly snapshot until we release the next 1.5 release, which will likely be 1.5.0.CR1 in a week or two.

Chris Cranford added a comment - 2021/03/01 9:50 AM integratemukesh , I believe Jiri included this in 1.4.2.Final which has been released. If you want to use 1.5, then you'll need to grab the nightly snapshot until we release the next 1.5 release, which will likely be 1.5.0.CR1 in a week or two.

Mukesh Sharma (Inactive) added a comment - 2021/03/01 8:57 AM

is this available in 1.5.0.Beta1 now?

in that case i do not have to use the internal.log.mining.dml.parser property as workaround.

can you please confirm?

Mukesh Sharma (Inactive) added a comment - 2021/03/01 8:57 AM is this available in 1.5.0.Beta1 now? in that case i do not have to use the internal.log.mining.dml.parser property as workaround. can you please confirm?

Debezium Builder added a comment - 2021/03/01 2:34 AM

Released

Debezium Builder added a comment - 2021/03/01 2:34 AM Released

Gunnar Morling added a comment - 2021/02/26 3:18 PM

Doh, sorry for not being more explicit about the option name. That said, I think we can resolve this issue, correct, ccranfor@redhat.com?

Gunnar Morling added a comment - 2021/02/26 3:18 PM Doh, sorry for not being more explicit about the option name. That said, I think we can resolve this issue, correct, ccranfor@redhat.com ?

Chris Cranford added a comment - 2021/02/26 1:19 PM

Hi remember1202, the option that Gunnar referenced is an internal option, so you need to use it as internal.log.mining.dml.parser. Its designated as internal as its simply a shortcut around the new parser for the interim until we know all the bugs have been worked out.

Chris Cranford added a comment - 2021/02/26 1:19 PM Hi remember1202 , the option that Gunnar referenced is an internal option, so you need to use it as internal.log.mining.dml.parser . Its designated as internal as its simply a shortcut around the new parser for the interim until we know all the bugs have been worked out.

shuang liu (Inactive) added a comment - 2021/02/26 3:01 AM

gunnar.morling , the connector option "log.mining.dml.parser": "legacy" is not take effect

shuang liu (Inactive) added a comment - 2021/02/26 3:01 AM gunnar.morling , the connector option "log.mining.dml.parser": "legacy" is not take effect

Assignee:: Chris Cranford

Reporter:: Mukesh Sharma (Inactive)

Votes:: 0 Vote for this issue

Watchers:: 7 Start watching this issue

Created:: 2021/02/24 2:42 PM

Updated:: 2023/01/24 3:42 PM

Resolved:: 2021/03/01 12:21 AM

Details

Description

Attachments

Attachments

Issue Links

Easy Agile Planning Poker

Activity

Collapse comment: Chris Cranford added a comment - 2021/03/04 9:03 AM

Expand comment: Chris Cranford added a comment - 2021/03/04 9:03 AM

Collapse comment: Jiri Pechanec added a comment - 2021/03/04 1:46 AM

Expand comment: Jiri Pechanec added a comment - 2021/03/04 1:46 AM

Collapse comment: Jiri Pechanec added a comment - 2021/03/04 1:46 AM

Expand comment: Jiri Pechanec added a comment - 2021/03/04 1:46 AM

Collapse comment: Anton Kondratev (Inactive) added a comment - 2021/03/04 1:41 AM

Expand comment: Anton Kondratev (Inactive) added a comment - 2021/03/04 1:41 AM

Collapse comment: Chris Cranford added a comment - 2021/03/01 9:50 AM

Expand comment: Chris Cranford added a comment - 2021/03/01 9:50 AM

Collapse comment: Mukesh Sharma (Inactive) added a comment - 2021/03/01 8:57 AM

Expand comment: Mukesh Sharma (Inactive) added a comment - 2021/03/01 8:57 AM

Collapse comment: Debezium Builder added a comment - 2021/03/01 2:34 AM

Expand comment: Debezium Builder added a comment - 2021/03/01 2:34 AM

Collapse comment: Gunnar Morling added a comment - 2021/02/26 3:18 PM

Expand comment: Gunnar Morling added a comment - 2021/02/26 3:18 PM

Collapse comment: Chris Cranford added a comment - 2021/02/26 1:19 PM

Expand comment: Chris Cranford added a comment - 2021/02/26 1:19 PM

Collapse comment: shuang liu (Inactive) added a comment - 2021/02/26 3:01 AM

Expand comment: shuang liu (Inactive) added a comment - 2021/02/26 3:01 AM

People

Dates