Uploaded image for project: 'Teiid'
  1. Teiid
  2. TEIID-5368

JSONTOXML fails on parsing too long numbers

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Blocker
    • Resolution: Done
    • Affects Version/s: 10.2
    • Fix Version/s: 11.0, 10.3.2, 10.2.3
    • Component/s: Query Engine
    • Labels:
      None
    • Environment:

      teiid-10.2.0 on WildFly Full 11.0.0.Final (WildFly Core 3.0.8.Final)

    • Steps to Reproduce:
      Hide

      Run the following query:

      Unable to find source-code formatter for language: noformat. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml
      select jsontoxml('root','{
               "items" : [{
      			"ordernumber" : "NL0000026010",
      			"orderdate" : "2016-01-01T01:13:39Z",
      			"orderdate_ts" : "01-01-2016 01:13:39",
      			"totalprice" : 24.8,
      			"total_order_price" : 24.8,
      			"vatvalue" : 4.3,
      			"vatrate" : 21,
      			"vatcustomernumber" : null,
      			"odr_description" : null,
      			"odr_note" : null,
      			"rest_status" : "pending",
      			"shipping_minimum_date" : null,
      			"shipping_maximum_date" : null,
      			"order_source" : "Magento",
      			"oss_id" : 131458290409912116350633678684356778292}, {
      			"ordernumber" : "NL0000026011",
      			"orderdate" : "2016-01-01T06:52:05Z",
      			"orderdate_ts" : "01-01-2016 06:52:05",
      			"totalprice" : 27.85,
      			"total_order_price" : 27.85,
      			"vatvalue" : 4.84,
      			"vatrate" : 21,
      			"vatcustomernumber" : null,
      			"odr_description" : null,
      			"odr_note" : null,
      			"rest_status" : "pending",
      			"shipping_minimum_date" : null,
      			"shipping_maximum_date" : null,
      			"order_source" : "Magento",
      			"oss_id" : 131458290409912116350633678684356778292
      		}
      	],
      	"hasMore" : true,
      	"limit" : 10,
      	"offset" : 0,
      	"count" : 10
      }') ;;
      
      Show
      Run the following query: Unable to find source-code formatter for language: noformat. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml select jsontoxml( 'root' ,'{ "items" : [{ "ordernumber" : "NL0000026010" , "orderdate" : "2016-01-01T01:13:39Z" , "orderdate_ts" : "01-01-2016 01:13:39" , "totalprice" : 24.8, "total_order_price" : 24.8, "vatvalue" : 4.3, "vatrate" : 21, "vatcustomernumber" : null , "odr_description" : null , "odr_note" : null , "rest_status" : "pending" , "shipping_minimum_date" : null , "shipping_maximum_date" : null , "order_source" : "Magento" , "oss_id" : 131458290409912116350633678684356778292}, { "ordernumber" : "NL0000026011" , "orderdate" : "2016-01-01T06:52:05Z" , "orderdate_ts" : "01-01-2016 06:52:05" , "totalprice" : 27.85, "total_order_price" : 27.85, "vatvalue" : 4.84, "vatrate" : 21, "vatcustomernumber" : null , "odr_description" : null , "odr_note" : null , "rest_status" : "pending" , "shipping_minimum_date" : null , "shipping_maximum_date" : null , "order_source" : "Magento" , "oss_id" : 131458290409912116350633678684356778292 } ], "hasMore" : true , "limit" : 10, "offset" : 0, "count" : 10 }') ;;

      Description

      Running the following query:

      Unable to find source-code formatter for language: noformat. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml
      select jsontoxml('root','{
               "items" : [{
      			"ordernumber" : "NL0000026010",
      			"orderdate" : "2016-01-01T01:13:39Z",
      			"orderdate_ts" : "01-01-2016 01:13:39",
      			"totalprice" : 24.8,
      			"total_order_price" : 24.8,
      			"vatvalue" : 4.3,
      			"vatrate" : 21,
      			"vatcustomernumber" : null,
      			"odr_description" : null,
      			"odr_note" : null,
      			"rest_status" : "pending",
      			"shipping_minimum_date" : null,
      			"shipping_maximum_date" : null,
      			"order_source" : "Magento",
      			"oss_id" : 131458290409912116350633678684356778292}, {
      			"ordernumber" : "NL0000026011",
      			"orderdate" : "2016-01-01T06:52:05Z",
      			"orderdate_ts" : "01-01-2016 06:52:05",
      			"totalprice" : 27.85,
      			"total_order_price" : 27.85,
      			"vatvalue" : 4.84,
      			"vatrate" : 21,
      			"vatcustomernumber" : null,
      			"odr_description" : null,
      			"odr_note" : null,
      			"rest_status" : "pending",
      			"shipping_minimum_date" : null,
      			"shipping_maximum_date" : null,
      			"order_source" : "Magento",
      			"oss_id" : 131458290409912116350633678684356778292
      		}
      	],
      	"hasMore" : true,
      	"limit" : 10,
      	"offset" : 0,
      	"count" : 10
      }') ;;
      

      fails with the following error message:

      Unable to find source-code formatter for language: noformat. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml
      15:30:44,531 WARN  [org.teiid.PROCESSOR] (Worker0_QueryProcessorQueue5) p69Z3QNLg3hP TEIID30020 Processing exception for request p69Z3QNLg3hP.0 'TEIID30328 Unable to evaluate jsontoxml('root', ?): TEIID30384 Error while evaluating function jsontoxml'. Originally ExpressionEvaluationException 'For input string: "131458290409912116350633678684356778292"' NumberFormatException.java:65.: org.teiid.api.exception.query.ExpressionEvaluationException: TEIID30328 Unable to evaluate jsontoxml('root', ?): TEIID30384 Error while evaluating function jsontoxml
              at org.teiid.query.eval.Evaluator.evaluate(Evaluator.java:709)
              at org.teiid.query.rewriter.QueryRewriter.evaluate(QueryRewriter.java:2295)
              at org.teiid.query.rewriter.QueryRewriter.rewriteExpressionDirect(QueryRewriter.java:2285)
              at org.teiid.query.rewriter.QueryRewriter.access$000(QueryRewriter.java:105)
              at org.teiid.query.rewriter.QueryRewriter$2.replaceExpression(QueryRewriter.java:767)
              at org.teiid.query.sql.visitor.ExpressionMappingVisitor.visit(ExpressionMappingVisitor.java:158)
              at org.teiid.query.sql.symbol.ExpressionSymbol.acceptVisitor(ExpressionSymbol.java:59)
              at org.teiid.query.sql.navigator.AbstractNavigator.visitVisitor(AbstractNavigator.java:50)
              at org.teiid.query.sql.navigator.PreOrPostOrderNavigator.postVisitVisitor(PreOrPostOrderNavigator.java:57)
              at org.teiid.query.sql.navigator.PreOrPostOrderNavigator.visit(PreOrPostOrderNavigator.java:179)
              at org.teiid.query.sql.symbol.ExpressionSymbol.acceptVisitor(ExpressionSymbol.java:59)
              at org.teiid.query.sql.navigator.AbstractNavigator.visitNode(AbstractNavigator.java:59)
              at org.teiid.query.sql.navigator.AbstractNavigator.visitNodes(AbstractNavigator.java:72)
              at org.teiid.query.sql.navigator.PreOrPostOrderNavigator.visit(PreOrPostOrderNavigator.java:338)
              at org.teiid.query.sql.lang.Select.acceptVisitor(Select.java:164)
              at org.teiid.query.sql.navigator.PostOrderNavigator.doVisit(PostOrderNavigator.java:36)
              at org.teiid.query.rewriter.QueryRewriter.rewriteExpressions(QueryRewriter.java:774)
              at org.teiid.query.rewriter.QueryRewriter.rewriteQuery(QueryRewriter.java:559)
              at org.teiid.query.rewriter.QueryRewriter.rewriteCommand(QueryRewriter.java:269)
              at org.teiid.query.rewriter.QueryRewriter.rewrite(QueryRewriter.java:172)
              at org.teiid.query.rewriter.QueryRewriter.rewrite(QueryRewriter.java:176)
              at org.teiid.dqp.internal.process.Request.generatePlan(Request.java:436)
              at org.teiid.dqp.internal.process.Request.processRequest(Request.java:486)
              at org.teiid.dqp.internal.process.RequestWorkItem.processNew(RequestWorkItem.java:660)
              at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:339)
              at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:47)
              at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:276)
              at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:277)
              at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:115)
              at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:206)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      Caused by: org.teiid.api.exception.query.FunctionExecutionException: TEIID30384 Error while evaluating function jsontoxml
              at org.teiid.query.function.FunctionDescriptor.invokeFunction(FunctionDescriptor.java:289)
              at org.teiid.query.eval.Evaluator.evaluate(Evaluator.java:1309)
              at org.teiid.query.eval.Evaluator.internalEvaluate(Evaluator.java:741)
              at org.teiid.query.eval.Evaluator.evaluate(Evaluator.java:707)
              ... 32 more
      Caused by: java.lang.NumberFormatException: For input string: "131458290409912116350633678684356778292"
              at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
              at java.lang.Long.parseLong(Long.java:592)
              at java.lang.Long.valueOf(Long.java:803)
              at org.teiid.json.simple.Yylex.yylex(Yylex.java:662)
              at org.teiid.json.simple.JSONParser.nextToken(JSONParser.java:125)
              at org.teiid.json.simple.JSONParser.parse(JSONParser.java:257)
              at org.teiid.query.function.source.XMLSystemFunctions$JsonToXmlContentHandler.nextEvent(XMLSystemFunctions.java:354)
              at org.codehaus.stax2.ri.Stax2EventWriterImpl.add(Stax2EventWriterImpl.java:171)
              at org.teiid.query.function.source.XMLSystemFunctions$8.translate(XMLSystemFunctions.java:1012)
              at org.teiid.query.function.source.XMLSystemFunctions.saveToBufferManager(XMLSystemFunctions.java:1087)
              at org.teiid.query.function.source.XMLSystemFunctions.jsonToXml(XMLSystemFunctions.java:1003)
              at org.teiid.query.function.source.XMLSystemFunctions.jsonToXml(XMLSystemFunctions.java:983)
              at org.teiid.query.function.source.XMLSystemFunctions.jsonToXml(XMLSystemFunctions.java:979)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.teiid.query.function.FunctionDescriptor.invokeFunction(FunctionDescriptor.java:275)
              ... 35 more
      

      I know that Teiid now contains totally json-simple library source code and the bug can be fixed quickly, something similar was reported here: https://github.com/fangyidong/json-simple/issues/57, later merged to https://github.com/fangyidong/json-simple/issues/73 issue but there is also another fork of the json-simple lib, here is the link: https://github.com/cliftonlabs/json-simple, author of the library claimed that he had already fixed the bug there, here is list of fixed bugs and features: https://cliftonlabs.github.io/json-simple/ but the main problem is that the forked library is not compatible anymore with original json-simple library but maybe we can take some changes from there to fix the original bug described here.

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                shawkins Steven Hawkins
                Reporter:
                dalex005 Dmitrii Pogorelov
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: