Details
-
Enhancement
-
Resolution: Done
-
Minor
-
1.1.0.Beta1
-
None
Description
"Update" change events for MongoDB do not include all of the fields needed when the event is from a sharded collection that uses multiple fields for the shard key. Here is what an "update" oplog entry looks like where the shard key is made up of dc, acctId and _id:
{
"ts" : Timestamp(1570559743, 2),
"t" : NumberLong(180),
"h" : NumberLong(-2190864266344322319),
"v" : 2,
"op" : "u",
"ns" : "Asset",
"ui" : UUID("8667a4d5-051f-4257-bfa4-451b6836b651"),
"o2" : {
"dc" : "west",
"acctId" : NumberLong(1234567),
"_id" : NumberLong(902345287483)
},
"wall" : ISODate("2019-10-08T18:35:43.431Z"),
"o" : {
"$v" : 1,
"$set" : {...}
}
}
The current code will just use the _id field as the key so the other two fields will be lost:
{
"schema": {
"type": "struct",
"fields": [
{
"type": "string",
"optional": false,
"field": "id"
}
],
"optional": false,
"name": "Asset.Key"
},
"payload": {
"id": "902345287483"
}
}
I propose that when multiple fields are in o2 the key should use all of them as the id like so:
{
"schema": {
"type": "struct",
"fields": [
{
"type": "string",
"optional": false,
"field": "id"
}
],
"optional": false,
"name": "Asset.Key"
},
"payload": {
"id": "{ \"dc" : \"west\" , \"acctId\" : 1234567, \"_id\" : 902345287483}"
}
}