Search code examples
hadoopapache-sparkjdbchivethrift

org.apache.hive.jdbc.HiveDriver: HiveBaseResultSet has not implemented absolute()?


I just started using the driver org.apache.hive.jdbc.HiveDriver (Version 1.2.1 for spark2) with a Spark Thrift Server (STS) (reference here)

java.sql.ResultSet defines the method absolute() (JavaDoc here)

but HiveBaseResultSet seems to have chosen not to implement the method (source code here)

So now my application (built on top of SmartGWT) was doing a simple operation and I got the following error message:

=== 2017-05-13 18:06:16,980 [3-47] WARN  RequestContext - dsRequest.execute() failed:
java.sql.SQLException: Method not supported
        at org.apache.hive.jdbc.HiveBaseResultSet.absolute(HiveBaseResultSet.java:70)
        at org.apache.commons.dbcp.DelegatingResultSet.absolute(DelegatingResultSet.java:373)
        at com.isomorphic.sql.SQLDataSource.executeWindowedSelect(SQLDataSource.java:2970)
        at com.isomorphic.sql.SQLDataSource.SQLExecute(SQLDataSource.java:2024)

What is the reason that the driver chose not to implement absolute()?

Are there any workaround for the limitation?


Solution

  • Thanks for the hint from Mark Rotteveel. Now I understand better and let me post an answer to my own question.

    Implementation of absolute() is optional

    As specified by the Interface of ResultSet#absolute() (link), the implementation for absolute() is optional -- especially when the result set type is TYPE_FORWARD_ONLY.

    Workaround

    In my case, the result set comes from a Spark Thrift Server (STS) so I guess it is indeed forward-only. So the question became how to instruct my application to NOT making a call to absolute(), which is basically for cursor movement.

    SmartGWT-specific answer

    For SmartGWT, this is controlled by a property called sqlPaging, which we can specified for an OperationBinding. The right value to use seems to be dropAtServer (more reference here). So I set my SmartGWT DataSource XML file to something like this

    <operationBindings>
        <operationBinding   operationType="fetch"  progressiveLoading="false"
                    sqlPaging="dropAtServer" 
            >
    

    After that I saw another error, which is now related to HiveConnection#commit():

    java.sql.SQLException: Method not supported
        at org.apache.hive.jdbc.HiveConnection.commit(HiveConnection.java:742)
        at org.apache.commons.dbcp.DelegatingConnection.commit(DelegatingConnection.java:334)
        at com.isomorphic.sql.SQLTransaction.commitTransaction(SQLTransaction.java:307)
        at com.isomorphic.sql.SQLDataSource.commit(SQLDataSource.java:4673) 
    

    After more digging, I realized that the right property for SmartGWT to control the commit behavior is autoJoinTransactions and I should set it to false (more reference here). After these two changes, I could get may application to talk to STS via jdbc.HiveDriver

    For anyone out there who are also trying this, here is my full settings for the driver in SmartGWT's server.properties (more reference here)

    sql.defaultDatabase: perf2 # this name is picked by me, but it can be anyname
    
    sql.perf2.driver.networkProtocol: tcp
    sql.perf2.driver: org.apache.hive.jdbc.HiveDriver   # important
    sql.perf2.database.type: generic                    # important
    sql.perf2.autoJoinTransactions: false               # important
    sql.perf2.interface.type: driverManager             # important
    sql.perf2.driver.url: jdbc:hive2://host:port        # important -- pick your host:port
    sql.perf2.driver.user: someuser                     # important -- pick your username
    sql.perf2.interface.credentialsInURL: true
    sql.perf2.driver.databaseName: someDb
    sql.perf2.driver.context: