Details
-
Bug
-
Resolution: Obsolete
-
Major
-
None
-
0.10.0.Beta2
-
None
Description
Running an instance with postgres as source, with debezium connect pulling changes into kafka. Everything in docker. Runs fine until restarting the postgres container, afterwards no changes appears.
Tried restarting connector and task running, but to no effect.
Downgraded to version 0.9.5.Final, and it works fine.
No exceptions in being logged either, connector logs as usual "Commit offsets".
Postgres also logs "Streaming transactions committing after ****" after restart.
Config
source.json
{
"name": "postgres-connector",
"config": {
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
"tasks.max": "1",
"database.hostname": "postgres",
"database.port": "5432",
"database.user": "user",
"database.password": "ps",
"database.dbname": "db",
"database.server.name": "dbServer",
"schema.blacklist": "public",
"transforms": "changeTopic",
"transforms.changeTopic.type": "org.apache.kafka.connect.transforms.RegexRouter",
"transforms.changeTopic.regex": ".*",
"transforms.changeTopic.replacement": "db_changes"
}
}
Docker compose part
zookeeper:
image: debezium/zookeeper:${DEBEZIUM_VERSION}
container_name: ${CONTAINER_PREFIX}-zookeeper
restart: always
ports:
- 2181:2181
- 2888:2888
- 3888:3888
networks:
- network1
kafka:
image: debezium/kafka:${DEBEZIUM_VERSION}
container_name: ${CONTAINER_PREFIX}-kafka
restart: always
ports:
- 9092:9092
links:
- zookeeper
environment:
ZOOKEEPER_CONNECT: "zookeeper:2181"
networks:
- network1
postgres:
image: debezium/postgres:${POSTGRES_VERSION}
container_name: ${CONTAINER_PREFIX}-postgres
restart: always
stop_grace_period: 15s
ports:
- 5000:5432
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=ps
- POSTGRES_DB=db
command: '-c shared_buffers=512MB -c work_mem=16MB'
volumes:
- postgresdata:/var/lib/postgresql/data
networks:
- network1
connect:
image: debezium/connect:${DEBEZIUM_VERSION}
container_name: ${CONTAINER_PREFIX}-connect
restart: always
stop_grace_period: 15s
ports:
- 8083:8083
links:
- kafka
- postgres
environment:
BOOTSTRAP_SERVERS: "kafka:9092"
GROUP_ID: 1
OFFSET_FLUSH_TIMEOUT_MS: 30000
OFFSET_FLUSH_INTERVAL_MS: 20000
MAX_BATCH_SIZE: 4096
MAX_QUEUE_SIZE: 12288
LOG_LEVEL: INFO
CONFIG_STORAGE_TOPIC: my_connect_configs
OFFSET_STORAGE_TOPIC: my_connect_offsets
STATUS_STORAGE_TOPIC: my_connect_statuses
networks:
- network1
