AnsweredAssumed Answered

Handling loops in oozie workflow

Question asked by sachin on Jul 16, 2013
I have an oozie use case for checking input data availability and trigger mapreduce job based on availability of data. So I wrote a shell script for checking input data and created an ssh action for it in oozie,

The number of retries and and retry intervals of Input data checking should be configurable and after each retry if the data is still missing I got to send an alert, after specified number of retries mapreduce job can start with the available data

I wrote a workflow as follows :
<code>

<start to="datacheck" />
<action name="datacheck">
    <ssh xmlns="uri:oozie:ssh-action:0.1">
        <host>${sshUserHost}</host>
        <command>${Oozieutilsscript}</command>
    </ssh>
    <ok to="datacheckswitch" />
    <error to="fail" />
</action>
<decision name="datacheckswitch">
    <switch>
        <case to="mapreduce">${(wf:actionData('datacheck')['datatransfer'] == "complete" )}</case>
        <case to="retry">${(wf:actionData('datacheck')['datatransfer'] == "incomplete" )}</case>
        <default to="fail" />
    </switch>
</decision>
<action name="retry">
    <ssh xmlns="uri:oozie:ssh-action:0.1">
        <host>${sshUserHost}</host>
        <command>${Oozieutilsscript1}</command>
    </ssh>
    <ok to="retryswitch" />
    <error to="fail" />
</action>
<decision name="retryswitch">
    <switch>
        <case to="datacheck">${(wf:actionData('datacheck')['retry'] == "notfinished" )}</case>
        <case to="datacheck">${(wf:actionData('datacheck')['retry'] == "finished" )}</case>
        <default to="fail" />
    </switch>
</decision>
<action name="mapreduce">
    ...............
    </action>
<!--Kill and End portion-->
<kill name="fail">
    <message>Java failed, error message[${wf:errorMessage(wf:lastErrorNode())}</message>
</kill>
<end name="end" />

</code>

Only when I executed the workflow I came to know that oozie doesn't support DAG. got the error

<pre>
Error: E0707 : E0707: Loop detected at parsing, node [datacheck] while parsing workflow.xml

</pre>
Is there any different approach for handling this usecase ?

Any help is appreciated .

Outcomes