Uploaded image for project: 'Teiid'
  1. Teiid
  2. TEIID-2383

InputStreamReader throws MalformedInputException when handling multi-byte characters

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • 8.3, 7.7.6
    • 7.7.1
    • Common
    • None
    • Hide

      1. unzip sample.zip
      2. copy files in sample.zip

      sample 
      |-- datafile  -> $JBOSS_HOME/servers/$profile
      |   `-- postal_utf8.csv
      |-- file-test-vdb.xml -> $JBOSS_HOME/servers/$profile/deploy
      `-- test-ds.xml       -> $JBOSS_HOME/servers/$profile/deploy
      

      3. run simple client
      ex)

      ./run.sh
      

      4. After 256 raws fetched, MalformedInputException occurs as follow:-

      11:42:16,219 DEBUG [PROCESSOR] Request Thread x8XReh9a7kIY.0 - error occurred
      org.teiid.core.TeiidProcessingException: Input length = 1
      	at org.teiid.query.processor.relational.TextTableNode.readChar(TextTableNode.java:267)
      	at org.teiid.query.processor.relational.TextTableNode.readLine(TextTableNode.java:207)
      	at org.teiid.query.processor.relational.TextTableNode.nextBatchDirect(TextTableNode.java:166)
      	at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280)
      	at org.teiid.query.processor.BatchIterator.finalRow(BatchIterator.java:70)
      	at org.teiid.common.buffer.AbstractTupleSource.getCurrentTuple(AbstractTupleSource.java:69)
      	at org.teiid.query.processor.BatchIterator.getCurrentTuple(BatchIterator.java:82)
      	at org.teiid.common.buffer.AbstractTupleSource.hasNext(AbstractTupleSource.java:91)
      	at org.teiid.query.processor.relational.NestedTableJoinStrategy.process(NestedTableJoinStrategy.java:120)
      	at org.teiid.query.processor.relational.JoinNode.nextBatchDirect(JoinNode.java:199)
      	at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280)
      	at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:155)
      	at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280)
      	at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:176)
      	at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:139)
      	at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:105)
      	at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:147)
      	at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:375)
      	at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:288)
      	at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:49)
      	at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:216)
      	at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:240)
      	at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:122)
      	at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:292)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      	at java.lang.Thread.run(Thread.java:662)
      Caused by: java.nio.charset.MalformedInputException: Input length = 1
      	at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
      	at org.teiid.core.util.InputStreamReader.read(InputStreamReader.java:84)
      	at java.io.BufferedReader.fill(BufferedReader.java:136)
      	at java.io.BufferedReader.read(BufferedReader.java:157)
      	at org.teiid.query.processor.relational.TextTableNode.readChar(TextTableNode.java:245)
      	... 26 more
      
      Show
      1. unzip sample.zip 2. copy files in sample.zip sample |-- datafile -> $JBOSS_HOME/servers/$profile | `-- postal_utf8.csv |-- file-test-vdb.xml -> $JBOSS_HOME/servers/$profile/deploy `-- test-ds.xml -> $JBOSS_HOME/servers/$profile/deploy 3. run simple client ex) ./run.sh 4. After 256 raws fetched, MalformedInputException occurs as follow:- 11:42:16,219 DEBUG [PROCESSOR] Request Thread x8XReh9a7kIY.0 - error occurred org.teiid.core.TeiidProcessingException: Input length = 1 at org.teiid.query.processor.relational.TextTableNode.readChar(TextTableNode.java:267) at org.teiid.query.processor.relational.TextTableNode.readLine(TextTableNode.java:207) at org.teiid.query.processor.relational.TextTableNode.nextBatchDirect(TextTableNode.java:166) at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280) at org.teiid.query.processor.BatchIterator.finalRow(BatchIterator.java:70) at org.teiid.common.buffer.AbstractTupleSource.getCurrentTuple(AbstractTupleSource.java:69) at org.teiid.query.processor.BatchIterator.getCurrentTuple(BatchIterator.java:82) at org.teiid.common.buffer.AbstractTupleSource.hasNext(AbstractTupleSource.java:91) at org.teiid.query.processor.relational.NestedTableJoinStrategy.process(NestedTableJoinStrategy.java:120) at org.teiid.query.processor.relational.JoinNode.nextBatchDirect(JoinNode.java:199) at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280) at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:155) at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280) at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:176) at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:139) at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:105) at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:147) at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:375) at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:288) at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:49) at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:216) at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:240) at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:122) at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang. Thread .run( Thread .java:662) Caused by: java.nio.charset.MalformedInputException: Input length = 1 at java.nio.charset.CoderResult.throwException(CoderResult.java:260) at org.teiid.core.util.InputStreamReader.read(InputStreamReader.java:84) at java.io.BufferedReader.fill(BufferedReader.java:136) at java.io.BufferedReader.read(BufferedReader.java:157) at org.teiid.query.processor.relational.TextTableNode.readChar(TextTableNode.java:245) ... 26 more

    Description

      My VDB uses File translator to handle a csv file which contains multi-byte characters. Reading some csv file via the VDB, MalformedInputException always occurs.

      Attachments

        1. sample.zip
          6 kB
        2. simple_client.tar
          620 kB

        Activity

          People

            rhn-engineering-shawkins Steven Hawkins
            rhn-support-hokuda Hisanobu Okuda
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: