Search code examples
hadoophdfsoozieoozie-coordinator

Passing variable based on date in oozie email action for attachments


I am using oozie to send out emails with attachments. I am doing like below.

<workflow-app name="Email" xmlns="uri:oozie:workflow:0.5">
    <start to="email-0fdf"/>
    <kill name="Kill">
        <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <action name="email-0fdf">
        <email xmlns="uri:oozie:email-action:0.2">
            <to>xxxxxxxxxxxxxxx@xxxxx</to>
            <subject>job success</subject>
            <content_type>text/plain</content_type>
            <attachment>/user/XXXX/logs/2017-05-03/exec.log</attachment>
        </email>
        <ok to="End"/>
        <error to="Kill"/>
    </action>
    <end name="End"/>
</workflow-app>

Now in the workflow near <attachment>/user/XXXX/logs/2017-05-03/exec.log</attachment> the date always changes.

How can I pass the variable that when the work flow is invoked then I want to send attachment for that particular day.

edited question.

My shell script:

#!/bin/bash

TIMESTAMP=`date "+%Y-%m-%d"`
path=/user/$USER/logging/${TIMESTAMP}/status/${TIMESTAMP}.fail_log

path1=/user/$USER/logging/`date -d "-1 days" '+%Y-%m-%d'`/status/`date -d "-1 days" '+%Y-%m-%d'`.fail_log

echo filePath=$path
echo filePath1=$path1

My new workflow:

<workflow-app name="My_Workflow" xmlns="uri:oozie:workflow:0.5">
<start to="shell-05e6"/>
<kill name="Kill">
    <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<action name="shell-05e6">
    <shell xmlns="uri:oozie:shell-action:0.1">
        <job-tracker>${jobTracker}</job-tracker>
        <name-node>${nameNode}</name-node>
        <exec>shell.sh</exec>
        <file>/user/xxxxx/oozie/email/lib/shell.sh#shell.sh</file>
          <capture-output/>
    </shell>
    <ok to="email-66c2"/>
    <error to="Kill"/>
</action>
<action name="email-66c2">
    <email xmlns="uri:oozie:email-action:0.2">
        <to>[email protected]</to>
        <subject>job status</subject>
        <body>job status ${wf:actionData('shell-05e6')['filePath']}</body>
        <content_type>text/plain</content_type> 
       <attachment>${wf:actionData('shell-05e6')['filePath']},${wf:actionData('shell-05e6')['filePath1']}</attachment>
    </email>
    <ok to="End"/>
    <error to="Kill"/>
</action>
<end name="End"/>

Now if there is no file at one of the location say either filepath or filepath1 then the email action is failing.

What I want is irrespective of whether the file is present or not I want the email action to be successful


Solution

  • There could be two approach to solve new requriement.

    Approach #1 Add conditional Action between shell Action and email action

    Shell Action would be like as :

    path=/user/$USER/logging/${TIMESTAMP}/status/${TIMESTAMP}.fail_log
    path1=/user/$USER/logging/`date -d "-1 days" '+%Y-%m-%d'`/status/`date -d "-1 days" '+%Y-%m-%d'`.fail_log
    
    if [ -e "$path" ] && [ -e "$path1"]
    then
        echo filePath=$path,$path1
    elif [ -e "$path" ]
    then
        echo filePath=$path
    elif [ -e "$path1" ]
    then
        echo filePath=$path1
    else
        echo filePath=""
    fi
    

    conditional action would be like as :

    if filePath = "" then
      call email_0 action # which has NO attachment tag.
    else
      call email_2 action # which has attachment tag with two files.
    end if
    

    Below conditional action you will have two email actions.

    1. with attachment tag "<attachment>${wf:actionData('shell-05e6')['filePath']}</attachment>" and
    2. without attachment tag

    Approach#2 Without conditional action.

    Shell Action would be like as :

    path=/user/$USER/logging/${TIMESTAMP}/status/${TIMESTAMP}.fail_log
    path1=/user/$USER/logging/`date -d "-1 days" '+%Y-%m-%d'`/status/`date -d "-1 days" '+%Y-%m-%d'`.fail_log
    
    if [ -e "$path" ] && [ -e "$path1"]
    then
        echo filePath=$path,$path1
    elif [ -e "$path" ]
    then
        echo filePath=$path
    elif [ -e "$path1" ]
    then
        echo filePath=$path1
    else
        echo filePath="/user/$USER/logging/No_Status_log.fail_log" # this is default file with no data. You have to create it only one time.
    fi
    

    In this approach, there will alway one file attached eventhough no data avaliable.