-
Bug
-
Resolution: Done
-
Major
-
8.12.4
-
None
In the Impala translator, pushdown doesn't happen when two columns are multiplied in the THEN portion of a CASE statement with decimal data type.
//this should push down but does not
CASE WHEN column1 IS NOT NULL
THEN column1 * column2
ELSE column3 END
//but you can multiply the columns by a constant and it will push down
CASE WHEN column1 IS NOT NULL
THEN column1 * 34567
ELSE column3 END
//or
CASE WHEN column1 IS NOT NULL
THEN 34567 * column2
ELSE column3 END
//or casting to consistent datatypes with DECIMAL output
CASE WHEN column1 IS NOT NULL
THEN cast(column1 as DECIMAL ) * cast(column2 as DECIMAL )
ELSE cast(column3 as DECIMAL ) END
//BUT output it as double with the cast statement above changed and it does push down (so floating point works)
CASE WHEN column1 IS NOT NULL
THEN cast(column1 as double ) * cast(column2 as double)
ELSE cast(column3 as double) END
Source data types:
Column1 is DECIMAL
Column2 is BIGINT/LONG
Column3 is DECIMAL
Output column is DECIMAL
[TEIID-3937] Multiplying two columns in CASE statement overrides pushdown (Impala)
Steven Hawkins
added a comment - This is now addressed in 9.0
> It's not very straightforward, for what that may be worth, even using the overridden settings.xml
Can you describe in more detail what you are seeing (ideally in a forum post) so that we can get that captured / addressed?
Steven Hawkins
added a comment - This is now addressed in 9.0
> It's not very straightforward, for what that may be worth, even using the overridden settings.xml
Can you describe in more detail what you are seeing (ideally in a forum post) so that we can get that captured / addressed? 
Donald Krapohl (Inactive)
added a comment - Thanks! I was just getting my project set up. It's not very straightforward, for what that may be worth, even using the overridden settings.xml. Will push through so I can contribute. Thanks again!
Donald Krapohl (Inactive)
added a comment - Thanks! I was just getting my project set up. It's not very straightforward, for what that may be worth, even using the overridden settings.xml. Will push through so I can contribute. Thanks again! Steven Hawkins
added a comment - After reproducing this locally, I see what is happening. The translator is not marked as requiring a connection for the capabilities, so it is not looking automatically at the version. You can workaround this by manually setting the databaseVersion property on the translator. I'll fix the code for this as well.
Steven Hawkins
added a comment - After reproducing this locally, I see what is happening. The translator is not marked as requiring a connection for the capabilities, so it is not looking automatically at the version. You can workaround this by manually setting the databaseVersion property on the translator. I'll fix the code for this as well. Steven Hawkins
added a comment - > If you tell me how you call I may be able to see how it comes back.
We are using DatabaseMetaData.getDatabaseProductVersion() from Connection.getMetaData()
> Issue is the same with DECIMAL data type.
Decimal and BigDecimal are synonymous for Teiid.
Steven Hawkins
added a comment - > If you tell me how you call I may be able to see how it comes back.
We are using DatabaseMetaData.getDatabaseProductVersion() from Connection.getMetaData()
> Issue is the same with DECIMAL data type.
Decimal and BigDecimal are synonymous for Teiid. Donald Krapohl (Inactive)
added a comment - Corrected data type descriptions. Issue is the same with DECIMAL data type.
Donald Krapohl (Inactive)
added a comment - Corrected data type descriptions. Issue is the same with DECIMAL data type. Donald Krapohl (Inactive)
added a comment - We're on the standard parcel distro of Cloudera CDH 5.4.9 (latest supported), which should show as Impala 2.2.0+cdh5.4.9+0. Not sure if the version comes back as 2.2.0 when you call. If you tell me how you call I may be able to see how it comes back.
Donald Krapohl (Inactive)
added a comment - We're on the standard parcel distro of Cloudera CDH 5.4.9 (latest supported), which should show as Impala 2.2.0+cdh5.4.9+0. Not sure if the version comes back as 2.2.0 when you call. If you tell me how you call I may be able to see how it comes back. Steven Hawkins
added a comment - We don't have a type mapping for bigdecimal unless impala reports version 2 or greater. What version of impala are you using?
If it's an older version of impala we could add an option to map bigdecimal to double.
Steven Hawkins
added a comment - We don't have a type mapping for bigdecimal unless impala reports version 2 or greater. What version of impala are you using?
If it's an older version of impala we could add an option to map bigdecimal to double. Donald Krapohl (Inactive)
added a comment - Added detail. Summary:
Impala translator
Can cast all to double and output as double and the formula pushes down.
If output is DECIMAL or BIGDECIMAL doesn't matter whether you cast or not, it doesn't send it down.
Our example has two BIGDECIMAL and one BIGINT source columns.
Donald Krapohl (Inactive)
added a comment - Added detail. Summary:
Impala translator
Can cast all to double and output as double and the formula pushes down.
If output is DECIMAL or BIGDECIMAL doesn't matter whether you cast or not, it doesn't send it down.
Our example has two BIGDECIMAL and one BIGINT source columns. Steven Hawkins
added a comment - > We're ouputting as bigdecimal so that may be a factor in the decision to push down as double datatype does work.
Yes the types can matter. What specific translator are you using? And what are the types of column1, column2, and column3?
Steven Hawkins
added a comment - > We're ouputting as bigdecimal so that may be a factor in the decision to push down as double datatype does work.
Yes the types can matter. What specific translator are you using? And what are the types of column1, column2, and column3? 
Will put it in a forum post. Thanks for quick work on this as always!