Details

    • Type: Enhancement
    • Status: Coding In Progress (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: sqlserver-connector
    • Labels:
      None

      Description

      This feature has been already implemented in MySQL connector.

      Motivation:
      1) Some columns might contain sensitive data which shouldn't be captured

      2) SQL Sever CDC mechanism has a limitation on column size

      According to docs:

      By default, the maximum size that can be added to a captured column in a single INSERT, UPDATE, WRITETEXT, or UPDATETEXT statement is 65,536 bytes or 64 KB. To increase this size to support larger LOB data, use the Configure the max text repl size Server Configuration Option to specify a larger maximum size. For more information, see Configure the max text repl size Server Configuration Option.

      We might want to skip huge columns if they are not needed.

      CDC mechanism already supports columns white-listing:

      [ @captured_column_list = ] 'captured_column_list'
      Identifies the source table columns that are to be included in the change table. captured_column_list is nvarchar(max) and can be NULL. If NULL, all columns are included in the change table.

      Column names must be valid columns in the source table. Columns defined in a primary key index, or columns defined in an index referenced by index_name must be included.

      captured_column_list is a comma-separated list of column names. Individual column names within the list can be optionally quoted by using either double quotation marks ("") or square brackets ([]). If a column name contains an embedded comma, the column name must be quoted.

      captured_column_list cannot contain the following reserved column names: __$start_lsn, __$end_lsn, __$seqval, __$operation, and __$update_mask.

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  grzegorzk Grzegorz Kołakowski
                  Reporter:
                  grzegorzk Grzegorz Kołakowski
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  2 Start watching this issue

                  Dates

                  • Created:
                    Updated: