Search code examples
muledataweavemule4

Mule 4.4 DataWeave append counter to file

I am reading in a file (see below). The example file has 13 rows.

A|doe|chemistry|100|A|
B|shea|maths|90|A|
C|baba|physics|80|B|
D|doe|chemistry|100|A|
E|shea|maths|90|A|
F|baba|physics|80|B|
G|doe|chemistry|100|A|
H|shea|maths|90|A|
I|baba|physics|80|B|
J|doe|chemistry|100|A|
K|shea|maths|90|A|
L|baba|physics|80|B|
M|doe|chemistry|100|A|

Then iterating over these rows using a for each ( batch size 5 ) and then calling a REST API Depending on REST API response ( success or failure ) I am writing payloads to respective success / error files.

I have mocked the called API such that first batch of 5 records will fail and rest of the files will succeed.

While writing to success / error files am using the following transformation :

output application/csv quoteValues=true,header=false,separator="|"
---
payload

All of this works fine.
Success log file:

"F"|"baba"|"physics"|"80"|"B"
"G"|"doe"|"chemistry"|"100"|"A"
"H"|"shea"|"maths"|"90"|"A"
"I"|"baba"|"physics"|"80"|"B"
"J"|"doe"|"chemistry"|"100"|"A"
"K"|"shea"|"maths"|"90"|"A"
"L"|"baba"|"physics"|"80"|"B"
"M"|"doe"|"chemistry"|"100"|"A"

Error log file:

"A"|"doe"|"chemistry"|"100"|"A"
"B"|"shea"|"maths"|"90"|"A"
"C"|"baba"|"physics"|"80"|"B"
"D"|"doe"|"chemistry"|"100"|"A"
"E"|"shea"|"maths"|"90"|"A"

Now what I want to do is append the row/line number to each of these files so when this goes to production , whoever is monitoring these files can easily understand and correlate with the original file . So as an example in case of error log file ( the first batch failed which is rows 1 to 5 ) I want to append these numbers to each of the rows:

"1"|"A"|"doe"|"chemistry"|"100"|"A"
"2"|"B"|"shea"|"maths"|"90"|"A"
"3"|"C"|"baba"|"physics"|"80"|"B"
"4"|"D"|"doe"|"chemistry"|"100"|"A"
"5"|"E"|"shea"|"maths"|"90"|"A"

Not sure what I should write in DataWeave to achieve this?

Solution

  • Inside the ForEach scope, you have access to the counter vars.counter (or whatever name you've chosen since it's configurable). You will need to iterate over each chunk of records for adding the position for each one. You can use something like:

    %dw 2.0
output application/csv quoteValues=true,header=false,separator="|"
var batchSize = 5
---
payload map ({
    counter: batchSize * (vars.counter - 1) + ($$ + 1)
} ++ $
)

    Or if you prefer to use the update function (this will add the record counter at the last column instead though):

    %dw 2.0
output application/csv quoteValues=true,header=false,separator="|"
var batchSize = 5
---
payload map (
    $ update {
        case .counter! -> batchSize * (vars.counter - 1) + ($$ + 1)
    }
)

    Remember to replace the batchSize variable from this code with the same value you're using in the ForEach scope (if it's parameterised, it would be better).

    Edit 1 - Clarification: the - 1 and + 1 are because both indexes (the counter from the For Each scope and the $$ from the map) are zero-based.