After the initial deployment of Debezium, if a new table is added to MSSQL, its schema is was captured

What Debezium connector do you use and what version?

 - Debezium connector for SQL Server version 2.5.3 Final

What is the connector configuration?

{
  "name": "debezium-Source-Connector-MsSql-test_internalTopic-002",
  "config": {
  "connector.class":
  "io.debezium.connector.sqlserver.SqlServerConnector",

  "database.encrypt": "false",
  "database.hostname": "mssql.xyz",
  "database.port": "1433",
  "database.names": "BAY_POC",
  "database.user": "user",
  "database.password": "password",
  "table.include.list": "dbo.WorkProfile",

  "topic.prefix": "test_internalTopic-002",
  "schema.history.internal.kafka.bootstrap.servers": "broker.xyz:9092", 
  "schema.history.internal.kafka.topic": "test_internalTopic-002_schemahistory",

  "tombstones.on.delete": "false",  
  "time.precision.mode": "connect",

  "key.converter": "io.confluent.connect.avro.AvroConverter",
  "value.converter" : "io.confluent.connect.avro.AvroConverter",
  "value.converter.schema.registry.url" : "http://schema.xyz:8081",
  "key.converter.schema.registry.url" : "http://schema.xyz:8081",  

  "transforms": "unwrap",
  "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState",
  "transforms.unwrap.delete.handling.mode": "rewrite"  
 }
} 

What is the captured database version?

 - Microsoft SQL Server 2019

What behaviour do you expect?

 - After the initial deployment of Debezium, if a new table is added to MSSQL, its schema is not captured.

What behaviour do you see?

 let me explain what I've tested:

  1. On MSSQL, I have tables WorkProfile and WorkProfile_2, both with CDC enabled.
  2. I deployed Debezium with table.include.list set to table WorkProfile only and left schema.history.internal.store.only.captured.tables.ddl at its default value, which is false.
  3. After deploying Debezium in step 2, I found that the schema history topic had two record, which described the schema of table WorkProfile and WorkProfile_2 (Which is the correct behavior according to what we want.)
  4. I created a new table WorkProfile_3 on MSSQL and enabled CDC.
  5. I updated table.include.list to include tables WorkProfile and WorkProfile_3 and then re-deployed Debezium.
  6. After re-deploying Debezium in step 5, I found that the schema history topic had one more record, which described the schema of table WorkProfile_3. So, at this point, the schema history topic had a total of 3 records.

 The bugs I encountered were:

  - After the initial deployment of Debezium, if a new table is added to MSSQL, its schema is not captured. However, in the case I encountered, the schema of the newly added table was captured.

Note : From 5., if I don't add WorkProfile_3 to the table.include.list, Debezium won't capture the schema of the WorkProfile_3 table. This means that in 6., the schema history topic won't have a new record added. That is, there will be only 2 records in the schema history topic, which describe the schema of WorkProfile and WorkProfile_2.

Do you see the same behaviour using the latest relesead Debezium version?

 -  Yes, I encountered this behavior in the Debezium connector for SQL Server version 2.5.3 Final, which is the latest stable version.

For further information

https://debezium.zulipchat.com/#narrow/stream/348252-community-sqlserver/topic/Adding.20a.20table.20.20after.20deploy.20debezium