Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1165

Debezium fails connecting to Mongo Atlas

    XMLWordPrintable

Details

    • Bug
    • Resolution: Won't Do
    • Major
    • None
    • 0.6.2
    • mongodb-connector
    • None
    • Hide

      Mongo debezium connector has worked on our dev and qa env, however once we moved to perf which has Mongo Atlas, tasks are not getting created when we start the debezium connector.

      FYI:
      Worker IP whitelisted on Mongo Atas
      Monga Atlas version 3.4, Mongo java driver 3.4
      User has Atlas admin permission

      Config:
      {
      "connector.class": "io.debezium.connector.mongodb.MongoDbConnector",
      "schema.registry.url": "http://testing-schemareg01.cloud.operative.com:8081,http://testing-schemareg02.cloud.operative.com:8081",
      "zookeeper.connect": "testing-zookeeper01.cloud.operative.com:2181,testing-zookeeper02.cloud.operative.com:2181,testing-zookeeper03.cloud.operative.com:2181",
      "tasks.max": "1",
      "name": "mongo_debezium_testing",
      "bootstrap.servers": "testing-kafka01.cloud.operative.com:9091,testing-kafka02.cloud.operative.com:9091,testing-kafka03.cloud.operative.com:9091",
      "mongodb.name": "perftest",
      "mongodb.hosts": "testing-shard-0/testing-shard-00-00-mxmsa.mongodb.net:27017,testing-shard-0/testing-shard-00-01-mxmsa.mongodb.net:27017,testing-shard-0/testing-shard-00-02-mxmsa.mongodb.net:27017",
      "mongo.ssl.enabled":"true",
      "mongodb.password": "testing",
      "mongodb.user": "testuser",
      "mongodb.members.auto.discover": "true",
      "collection.whitelist": "testdb.testcollection"
      }

      Below is the exception we are seeing,

      2019-02-27 06:22:43 [DistributedHerder] INFO o.a.k.c.r.d.DistributedHerder - Finished starting connectors and tasks
      2019-02-27 06:23:03 [Thread-14] ERROR i.d.c.mongodb.ReplicaSetDiscovery - Error while reading the 'shards' collection in the 'config' database: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=testing-shard-00-02-mxmsa.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}, {address=testing-shard-00-00-mxmsa.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}, {address=testing-shard-00-01-mxmsa.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}]
      com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=testing-shard-00-02-mxmsa.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}, {address=testing-shard-00-00-mxmsa.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}, {address=testing-shard-00-01-mxmsa.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}]
      at com.mongodb.connection.BaseCluster.getDescription(BaseCluster.java:167)
      at com.mongodb.Mongo.getConnectedClusterDescription(Mongo.java:885)
      at com.mongodb.Mongo.createClientSession(Mongo.java:877)
      at com.mongodb.Mongo$3.getClientSession(Mongo.java:866)
      at com.mongodb.Mongo$3.execute(Mongo.java:823)
      at com.mongodb.MongoIterableImpl.execute(MongoIterableImpl.java:130)
      at com.mongodb.MongoIterableImpl.iterator(MongoIterableImpl.java:77)
      at com.mongodb.MappingIterable.iterator(MappingIterable.java:40)
      at io.debezium.connector.mongodb.MongoUtil.contains(MongoUtil.java:181)
      at io.debezium.connector.mongodb.MongoUtil.contains(MongoUtil.java:170)
      at io.debezium.connector.mongodb.MongoUtil.onDatabase(MongoUtil.java:114)
      at io.debezium.connector.mongodb.MongoUtil.onCollection(MongoUtil.java:129)
      at io.debezium.connector.mongodb.MongoUtil.onCollectionDocuments(MongoUtil.java:148)
      at io.debezium.connector.mongodb.ReplicaSetDiscovery.getReplicaSets(ReplicaSetDiscovery.java:66)
      at io.debezium.connector.mongodb.ReplicaSetMonitorThread.run(ReplicaSetMonitorThread.java:62)
      2019-02-27 06:23:03 [Thread-14] INFO org.mongodb.driver.cluster - Cluster description not yet available. Waiting for 30000 ms before timing out

      Show
      Mongo debezium connector has worked on our dev and qa env, however once we moved to perf which has Mongo Atlas, tasks are not getting created when we start the debezium connector. FYI: Worker IP whitelisted on Mongo Atas Monga Atlas version 3.4, Mongo java driver 3.4 User has Atlas admin permission Config: { "connector.class": "io.debezium.connector.mongodb.MongoDbConnector", "schema.registry.url": "http://testing-schemareg01.cloud.operative.com:8081, http://testing-schemareg02.cloud.operative.com:8081 ", "zookeeper.connect": "testing-zookeeper01.cloud.operative.com:2181,testing-zookeeper02.cloud.operative.com:2181,testing-zookeeper03.cloud.operative.com:2181", "tasks.max": "1", "name": "mongo_debezium_testing", "bootstrap.servers": "testing-kafka01.cloud.operative.com:9091,testing-kafka02.cloud.operative.com:9091,testing-kafka03.cloud.operative.com:9091", "mongodb.name": "perftest", "mongodb.hosts": "testing-shard-0/testing-shard-00-00-mxmsa.mongodb.net:27017,testing-shard-0/testing-shard-00-01-mxmsa.mongodb.net:27017,testing-shard-0/testing-shard-00-02-mxmsa.mongodb.net:27017", "mongo.ssl.enabled":"true", "mongodb.password": "testing", "mongodb.user": "testuser", "mongodb.members.auto.discover": "true", "collection.whitelist": "testdb.testcollection" } Below is the exception we are seeing, 2019-02-27 06:22:43 [DistributedHerder] INFO o.a.k.c.r.d.DistributedHerder - Finished starting connectors and tasks 2019-02-27 06:23:03 [Thread-14] ERROR i.d.c.mongodb.ReplicaSetDiscovery - Error while reading the 'shards' collection in the 'config' database: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers= [{address=testing-shard-00-02-mxmsa.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}, {address=testing-shard-00-00-mxmsa.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}, {address=testing-shard-00-01-mxmsa.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}] com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers= [{address=testing-shard-00-02-mxmsa.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}, {address=testing-shard-00-00-mxmsa.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}, {address=testing-shard-00-01-mxmsa.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}] at com.mongodb.connection.BaseCluster.getDescription(BaseCluster.java:167) at com.mongodb.Mongo.getConnectedClusterDescription(Mongo.java:885) at com.mongodb.Mongo.createClientSession(Mongo.java:877) at com.mongodb.Mongo$3.getClientSession(Mongo.java:866) at com.mongodb.Mongo$3.execute(Mongo.java:823) at com.mongodb.MongoIterableImpl.execute(MongoIterableImpl.java:130) at com.mongodb.MongoIterableImpl.iterator(MongoIterableImpl.java:77) at com.mongodb.MappingIterable.iterator(MappingIterable.java:40) at io.debezium.connector.mongodb.MongoUtil.contains(MongoUtil.java:181) at io.debezium.connector.mongodb.MongoUtil.contains(MongoUtil.java:170) at io.debezium.connector.mongodb.MongoUtil.onDatabase(MongoUtil.java:114) at io.debezium.connector.mongodb.MongoUtil.onCollection(MongoUtil.java:129) at io.debezium.connector.mongodb.MongoUtil.onCollectionDocuments(MongoUtil.java:148) at io.debezium.connector.mongodb.ReplicaSetDiscovery.getReplicaSets(ReplicaSetDiscovery.java:66) at io.debezium.connector.mongodb.ReplicaSetMonitorThread.run(ReplicaSetMonitorThread.java:62) 2019-02-27 06:23:03 [Thread-14] INFO org.mongodb.driver.cluster - Cluster description not yet available. Waiting for 30000 ms before timing out

    Description

      Mongo debezium connector fails to create tasks while trying to connec to Mongo Atlas

      Attachments

        Activity

          People

            Unassigned Unassigned
            arrajeev87 Rajeev AR (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: