Details
-
Feature Request
-
Resolution: Unresolved
-
Major
-
None
-
None
-
None
-
0
-
0%
Description
The setting for my.cnf are:
log-bin=/var/log/mysql/mysql-bin
binlog_format=row
binlog_row_image=full
As per the mysql documentation , if the engine type is `NDB`, mysql logs only `minimal` changes to log files. Even if the binlog_row_image is set to `full`.
This causes no issues, if the Insert or Update query affects all columns. But, if you have an insert or update to only partial columns, the `minimal` feature means that partial data is posted on the log.
Full insert:
### INSERT INTO `user_registrations` ### SET ### @1=824 ### @2='+190554937263' ### @3='56DA9A2FD4D91320D01514ED497C719E5DC9323F' ### @4=1 ### @5=NULL ### @6='2019-06-18 06:20:23' ### @7='2019-06-18 06:20:23' ### @8=NULL
Partial insert on the same (as in the bin log):
### INSERT INTO `user_registrations` ### SET ### @1=824 ### @4=2 ### @6='2019-06-18 06:20:26'
This results in casting exception in TableSchemaBuilder, since this is expecting a full row from the mysql connector.
Table Structure:
'id', 'int(10) unsigned', 'NO', 'PRI', NULL, 'auto_increment' 'msisdn', 'varchar(50)', 'NO', '', NULL, '' 'device_id', 'varchar(50)', 'NO', '', NULL, '' 'status', 'enum(\'waiting_for_validation\',\'approved\',\'rejected\',\'expired\')', 'NO', '', NULL, '' 'rejection_reason', 'enum(\'max_otp_validations_reached\',\'canceled_by_user\')', 'YES', '', NULL, '' 'updated_at', 'datetime', 'YES', '', 'CURRENT_TIMESTAMP', 'on update CURRENT_TIMESTAMP' 'created_at', 'datetime', 'NO', '', NULL, '' 'channel', 'varchar(50)', 'YES', '', NULL, '
- * Log trace looks as below:*
[2019-06-18 11:32:38,175] ERROR Failed to properly convert data value for 'wallet_db_v3.user_registrations.msisdn' of type VARCHAR for *row [824, 2, 2019-06-18T06:20:26]*: (io.debezium.relational.TableSchemaBuilder:253) java.lang.IllegalArgumentException: Unexpected value for JDBC type 12 and column msisdn VARCHAR(50) CHARSET utf8 NOT NULL: class=class java.lang.Integer at io.debezium.jdbc.JdbcValueConverters.handleUnknownData(JdbcValueConverters.java:1164) at io.debezium.jdbc.JdbcValueConverters.convertValue(JdbcValueConverters.java:1207) at io.debezium.connector.mysql.MySqlValueConverters.convertString(MySqlValueConverters.java:380) at io.debezium.connector.mysql.MySqlValueConverters.lambda$converter$13(MySqlValueConverters.java:293) at io.debezium.relational.TableSchemaBuilder.lambda$createValueGenerator$2(TableSchemaBuilder.java:249) at io.debezium.relational.TableSchema.valueFromColumnData(TableSchema.java:145) at io.debezium.connector.mysql.RecordMakers$1.insert(RecordMakers.java:244) at io.debezium.connector.mysql.RecordMakers$RecordsForTable.create(RecordMakers.java:456) at io.debezium.connector.mysql.BinlogReader.handleInsert(BinlogReader.java:833) at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:501) at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1095) at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:943) at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:580) at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:825) at java.lang.Thread.run(Thread.java:748)
