I have made a java script that will connect to hive using Hiveserver2 and will create table and manage tables, for simple create, drop, insert data works fine.
I want to create external table with partition, for this I need to change the value for the following hive property,
hive.exec.dynamic.partition = true
hive.exec.dynamic.partition.mode = nonstrict
In hive cli I can do it using SET and the property name, but how can this be done in java code.
Here is my Java code:
public class HiveJdbcClient {
private static String strDriverName = "org.apache.hive.jdbc.HiveDriver";
public static void main(String[] args) throws SQLException {
try{
Class.forName(strDriverName);
} catch (ClassNotFoundException e){
e.printStackTrace();
System.out.println("No class found");
System.exit(1);
}
Connection con = DriverManager.getConnection("jdbc:hive2://172.11.1.11:10000/default","root","root123");
Statement stmt = con.createStatement();
String strTableName = "testtable";
//stmt.execute("drop table " + strTableName);
//creating staging table that will load the data to partition data
String strStagingTableSql = "create table if not exists "+strTableName+"_staging "+ " (SEQUENCE_NO DECIMAL, DATE_KEY INT, ACTIVITY_TIME_KEY INT, Ds_KEY INT, Ds_VALUE DECIMAL, TL_DATE_KEY INT) ROW FORMAT DELIMITED FIELDS TERIMANTED BY '~'";
String strMainTableSql = "create external table if not exists "+strTableName+" (SEQUENCE_NO DECIMAL, ACTIVITY_TIME_KEY INT, Ds_KEY INT, Ds_VALUE DECIMAL, TL_DATE_KEY INT) PARTITIONED BY (DATE_KEY INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY '~' LOCATION '/informatica/dwh/teradata/testtable'";
String strCreateSql = "create external table if not exists "+ strTableName + " (key int, value string) row format delimited fields terminated by ','";
boolean res = stmt.execute(strCreateSql);
//show tables
String sql = "show tables '" + strTableName + "'";
ResultSet res1 = stmt.executeQuery(sql);
if (res1.next()){
System.out.println(res1.getString(1));
}
sql = "describe "+ strTableName;
System.out.println("Running: "+ sql);
res1 = stmt.executeQuery(sql);
while (res1.next()){
System.out.println(res1.getString(1) + "\t" + res1.getString(2));
}
// load data into table
// NOTE: filepath has to be local to the hive server
// NOTE: /tmp/a.txt is a ctrl-A separated file with two fields per line
String strFilepath = "/informatica/testing_hive_client_java.txt";
sql = "load data inpath '" + strFilepath + "' into table " + strTableName;
System.out.println("Running: " + sql);
res = stmt.execute(sql);
sql = "select count(1) from "+ strTableName;
System.out.println("Running: "+ sql);
res1 = stmt.executeQuery(sql);
while(res1.next()){
System.out.println(res1.getString(1));
}
}// end of main
}// end of class
Experts please pour in your thoughts.
I was able to solve my problem by following code.
boolean resHivePropertyTest = stmt
.execute("SET hive.exec.dynamic.partition = true");
resHivePropertyTest = stmt
.execute("SET hive.exec.dynamic.partition.mode = nonstrict");
As the code is JDBC client code , so the execute will just go and execute this in hive and so that worked for me.