Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1232

Oracle connnector capture data from XStreams is very slow in certain scenario

    XMLWordPrintable

Details

    • Bug
    • Resolution: Obsolete
    • Major
    • None
    • 0.9.3.Final
    • oracle-connector
    • None

    Description

      Hi, having a trouble when using oracle connector ,description below:

      It happens in a certain scenario.
      I create a connector, it can capture data from Oracle very fast however when i delete the connector then re-create a new one using same config, the connector will become very slow i mean VERY VERY slow.
      Then i try to debug the code i found the line 70 in "OracleStreamingChangeEventSource.java" is very slow

                  // 2. receive events while running
                  while(context.isRunning()) {
                      LOGGER.trace("Receiving LCR");
                      xsOut.receiveLCRCallback(handler, XStreamOut.DEFAULT_MODE);
                  }
      

      the code above basically process the LCR from xstreams.
      So i wrote a sample code which is similar behaviour like the code above

              try {
                  Class.forName("oracle.jdbc.driver.OracleDriver");
                  String xStreamOutServerName = "DBZXOUT_TEST";
      
                  Connection connection = DriverManager.getConnection("jdbc:oracle:oci:@10.177.83.9:1521/ycdb", "xstrmadmin" , "xsa");
      
                  byte[] scn = XStreamUtility.convertSCNToPosition(new NUMBER(15600869L));
                  LCRHandler lcrHandler = new LCRHandler();
                  XStreamOut xout = XStreamOut.attach((OracleConnection) connection, xStreamOutServerName, scn, XStreamOut.DEFAULT_MODE);
                  System.out.println("start to run!");
                  while (true) {
                      whileCount++;
                      System.out.println(whileCount);
                      if(whileCount > 30) {
                          System.out.println("xout detached");
                          xout.detach(XStreamOut.DEFAULT_MODE);
                          break;
                      } else {
                          xout.receiveLCRCallback(lcrHandler , XStreamOut.DEFAULT_MODE);
                      }
      
                  }
              } catch (Exception e) {
                  e.printStackTrace();
              }
      
      public class LCRHandler implements XStreamLCRCallbackHandler {
          private int count = 0;
          public void processLCR(LCR lcr) throws StreamsException {
      
              if(lcr.getCommandType().equals("COMMIT")){
                  return;
              }
              try {
                  count++;
                  long pos = XStreamUtility.getSCNFromPosition(lcr.getPosition()).longValue();
                  System.out.println(String.format("count:%s ,transcationId:%s , sourcetime: %s , commandtype: %s , pos:%s", count ,
                          lcr.getTransactionId(),
                          lcr.getSourceTime(),
                          lcr.getCommandType(),
                          pos));
              } catch (SQLException e) {
                  e.printStackTrace();
              }
      
          }
      
          public void processChunk(ChunkColumnValue chunkColumnValue) throws StreamsException {
      
          }
      
          public LCR createLCR() throws StreamsException {
              return null;
          }
      
          public ChunkColumnValue createChunk() throws StreamsException {
              return null;
          }
      }
      

      What i found is when i first time attach to the XStream outbound server , it processes very fast , but if i stop the programme and attach second time or more using different scn position, it processes VERY VERY slow.

      If i use a NEW xstream outbound server name it becomes faster again. i have absolutely no idea why it happens.
      Please help me verify this and provide some potential solution
      Thanks a lot.

      Attachments

        Activity

          People

            Unassigned Unassigned
            dreamcatchernick Nick Gu (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: