Search code examples
hadooplog4japache-pigoozie

Pass JVM options to point to log4j.properties file in Pig Action in Oozie Workflow

In my Oozie workflow, there is a pig action.
While running, it is looking for log4j.properties file in CDH as I have not provided the file in my jars.
Now, I have the log4j.properties file with me and I just need to pass it as JVM option in the Pig action.
Is there any way to do this?

Solution

  • It is not possible to pass the custom log4j properties file to Pig action in an Oozie workflow.

    The PigMain.java file has disallowed the following parameters to be passed as arguments.

    static {
    DISALLOWED_PIG_OPTIONS.add("-4");
    DISALLOWED_PIG_OPTIONS.add("-log4jconf");
    DISALLOWED_PIG_OPTIONS.add("-e");
    DISALLOWED_PIG_OPTIONS.add("-execute");
    DISALLOWED_PIG_OPTIONS.add("-f");
    DISALLOWED_PIG_OPTIONS.add("-file");
    DISALLOWED_PIG_OPTIONS.add("-l");
    DISALLOWED_PIG_OPTIONS.add("-logfile");
    DISALLOWED_PIG_OPTIONS.add("-r");
    DISALLOWED_PIG_OPTIONS.add("-dryrun");
    DISALLOWED_PIG_OPTIONS.add("-P");
    DISALLOWED_PIG_OPTIONS.add("-propertyFile");
}

    This is possible if we run pig script via CLI, but not via Oozie Pig action.

    Reference: https://github.com/apache/oozie/blob/master/sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java

    The piglog4j.properties file is created during application runtime by reading the basic log4j.properties file from yarn container and placing the created file in the Hadoop job appCache.