How do I install Custom Jar in HDInsight

Question

I am new to Hadoop/HDInsight.

I have followed the steps here to create the jar package of SerDe. After the package json-serde-1.1.9.9-Hive13-jar-with-dependencies.jar is created the post says that I need to upload it to the head-node.

Does it mean that I have to RDP into the HDInsight VM and then manually upload the file?

If I don't have remote connection enabled to that VM what else can I do?

PS: The HDInsight Cluster is already provisioned.

Answer 1

You don't have to add it to the head-node for HDInsight. If you upload the jar to the storage account associated with your cluster, you can access it using the add jar command used in your example.

add jar wasb://<storageaccount>@<containername>/<jarfolder>/json-serde-1.1.9.9-Hive13-jar-with-dependencies.jar;

For example:

add jar wasb://andrewsstorage@datacontainer/myjars/json-serde-1.1.9.9-Hive13-jar-with-dependencies.jar

This is a more scaleable approach because the jar asset will remain after the HDI cluster is destroyed.