Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-649

SMT - DataException with io.debezium.connector.mongodb.transforms.UnwrapFromMongoDbEnvelope

    Details

      Description

      debezium-connector-mongodb-0.7.4-plugin.tar.gz
      Confluent Platform 4.0
      

      {
        "name": "mongodb-connector-json-a04",
        "config": {
          "connector.class": "io.debezium.connector.mongodb.MongoDbConnector",
              "value.converter":"org.apache.kafka.connect.json.JsonConverter",
              "key.converter":"org.apache.kafka.connect.json.JsonConverter",
              "key.converter.schemas.enable":"true",
              "value.converter.schemas.enable":"true",
          "mongodb.hosts": "rs0/proxmox01.moffatt.me:27017",
          "mongodb.name": "ubnt",
          "database.whitelist": "ace",
          "collection.whitelist": "ace.device",
          "transforms": "unwrap,changeTopic",
          "transforms.unwrap.type" :"io.debezium.connector.mongodb.transforms.UnwrapFromMongoDbEnvelope",
      "transforms.changeTopic.type":"org.apache.kafka.connect.transforms.RegexRouter",
      "transforms.changeTopic.regex":"(.*)",
      "transforms.changeTopic.replacement":"$1-json-a04"
        }
      }
      

      Error:

      [2018-03-07 17:35:28,504] ERROR WorkerSourceTask{id=mongodb-connector-json-a04-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:172)
      org.apache.kafka.connect.errors.DataException: is_11ac is not a valid field name
              at org.apache.kafka.connect.data.Struct.lookupField(Struct.java:252)
              at org.apache.kafka.connect.data.Struct.put(Struct.java:202)
              at io.debezium.connector.mongodb.transforms.MongoDataConverter.convertFieldValue(MongoDataConverter.java:188)
              at io.debezium.connector.mongodb.transforms.MongoDataConverter.lambda$convertFieldValue$0(MongoDataConverter.java:176)
              at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1380)
              at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
              at io.debezium.connector.mongodb.transforms.MongoDataConverter.convertFieldValue(MongoDataConverter.java:138)
              at io.debezium.connector.mongodb.transforms.MongoDataConverter.convertRecord(MongoDataConverter.java:32)
              at io.debezium.connector.mongodb.transforms.UnwrapFromMongoDbEnvelope.apply(UnwrapFromMongoDbEnvelope.java:103)
              at org.apache.kafka.connect.runtime.TransformationChain.apply(TransformationChain.java:38)
              at org.apache.kafka.connect.runtime.WorkerSourceTask.sendRecords(WorkerSourceTask.java:211)
              at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:187)
              at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:170)
              at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:214)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  jpechanec Jiri Pechanec
                  Reporter:
                  rmoff Robin Moffatt
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  3 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: