HDFS path in user defined function of Pig Latin

Question

I have the following User Defined Function programmed in Java language:

I defined FileWriter but an error message appear after the execution.

The program:

outputFile = new FileWriter("hdfs://NaeNode:9000/input/SG.csv",true); fw = new BufferedWriter(outputFile);

Caught error from UDF: trial.obvious_guess [hdfs:/NaemNode:9000/input/SG.csv (No such file or directory)]

How can I fix this issue, since I am using [pig -x MapReduce fie.pig] on execution

Answer 1

To store in HDFS, you must use FSDataOutputStream.

FileSystem.create returns FSDataOutputStream.

See the code below.

public static void main(String[] args) throws Exception {
    FSDataOutputStream fout = null;
    try {
        Path path = new Path("hdfs://NaeNode:9000/input/SG.csv");
        String data = "data";
        FileSystem fileSystem = FileSystem.get(new Configuration());
        if (!fileSystem.exists(path)) {
            fout = fs.create(path); 
        } else {
            fout = fs.append(path)
        }
        BufferedWriter bufferedWriter = new BufferedWriter(
            new OutputStreamWriter(fout)
        );
        bufferedWriter.write(data);
        bufferedWriter.close();
    } catch (Exception e) {
        e.printStackTrace();
    }
}