I generate a flow file with this data format(text):
+0002150736YY+0701748361MN 0001-01-01SSD+000000000000000+000000000000000+000000000000000+000000000000000+000000000000000+000000000000000+000000000000000+000000000000000+000000000000000+000000000000000+000000000000000+000000000000000-000000000002860+000000000000000+000000000000000 1+0000000000000002023-05-23+0701748361
+0002152097XX+0701763467RR 0001-01-01SSF+000000000000000+000000000000000+000000000000000+000000000000000+000000000000000+000000000011845+000000000000000+000000000000000+000000000000000+000000000000000+000000000000000+000000000000000+000000000000000+000000000000000+000000000000000 1+0000000000000002023-05-23+0701763467
I'm trying to execute a groovy script with the ExecuteGroovyScript process in order to output a json file, and for each key has its value:
import groovy.json.JsonBuilder
import java.nio.charset.StandardCharsets
import java.time.LocalDate
def flowFile = session.get()
if (flowFile) {
def datePattern = DateTimeFormatter.ofPattern('yyyy-MM-dd')
def records = []
flowFile.read().withReader('ISO-8859-1') { reader ->
reader.eachLine { line ->
def record = [
SEQ_LD: 'ADMIN.SEQ_INFO.NEXTVAL',
NO: line[0..10]?.trim()?.toBigDecimal(),
PA: line[11..12]?.trim() ?: '',
PAR: line[13..23]?.trim()?.toBigDecimal(),
CD: line[24..27]?.trim() ?: '',
CFOUR: line[28..31]?.trim() ?: '',
DTIR: line[32..41]?.trim() != '0000-00-00' ? LocalDate.parse(line[32..41]?.trim(), datePattern) : null,
DEV: line[42..44]?.trim() ?: '',
EXBL: line[45..60]?.trim()?.toBigDecimal(),
NEXBL: line[61..76]?.trim()?.toBigDecimal(),
SUSP: line[77..92]?.trim()?.toBigDecimal(),
NEXBL_SUSP: line[93..108]?.trim()?.toBigDecimal(),
SOLC: line[109..124]?.trim()?.toBigDecimal(),
MXBL: line[125..140]?.trim()?.toBigDecimal(),
MMIX: line[141..156]?.trim()?.toBigDecimal(),
ABD: line[157..172]?.trim()?.toBigDecimal(),
EXBLFF: line[173..188]?.trim()?.toBigDecimal(),
MTGR: line[189..204]?.trim()?.toBigDecimal(),
MKLM: line[205..220]?.trim()?.toBigDecimal(),
MXYR: line[221..236]?.trim()?.toBigDecimal(),
MHHH: line[237..252]?.trim()?.toBigDecimal(),
MOPO: line[253..268]?.trim()?.toBigDecimal(),
MNBFR: line[269..284]?.trim()?.toBigDecimal(),
CGTRH: line[285..289]?.trim() ?: '',
IPOING: line[290..290]?.trim() ?: '',
BGFTU: line[291..306]?.trim()?.toBigDecimal(),
DLJHJ: line[307..316]?.trim() != '0000-00-00' ? LocalDate.parse(line[307..316]?.trim(), datePattern) : null,
NHJGY: line[317..327]?.trim()?.toBigDecimal(),
AACBT: line[328..328]?.trim() ?: '',
TRT: LocalDate.parse('2023-06-05', datePattern)
]
records << record
}
}
def json = new JsonBuilder(records)
def jsonBytes = json.toString().getBytes(StandardCharsets.UTF_8)
flowFile = session.write(flowFile, { outputStream ->
outputStream.write(jsonBytes)
} as OutputStreamCallback)
flowFile = session.putAttribute(flowFile, 'filename', 'output.json')
session.transfer(flowFile, REL_SUCCESS)
}
but every time I get this error, I don't understand where is the problem:
All NodesExecuteGroovyScript[id=afb7a99f-0188-1000-0000-00004af5fb62] java.lang.NumberFormatException: java.lang.NumberFormatException
Any help please? thank you
A NumberFormatException means some field you are trying to turn into a BigDecimal isn't a number. Probably likely that one of those fields is empty and that will trigger a NumberFormatException. But, you have it worse than that because EACH LINE could have an issue that the other lines don't. The issue could be on line 5434, but you won't know which line had the problem because a NFE isn't going to carry the line out of the data. So, you need some better debugging in there. Right now it's like finding a needle in a haystack.
My suggestion is to build a small method to do that logic rather than inlining it over and over. Turn the inline line[0..10]?.trim()?.toBigDecimal() into:
Number toNumber( String line, ObjectRange range) {
try {
String v = line[range]?.trim()
return v ? v.toBigDecimal() : null // this will handle v == null or v is empty
} catch( Exception ex ) {
throw new Exception("Encountered exception at ${range} in line ${line}: ${ex}", ex)
}
}
Then change:
def record = [
SEQ_LD: 'ADMIN.SEQ_INFO.NEXTVAL',
NO: toNumber(line, 0..10 ),
PA: line[11..12]?.trim() ?: '',
PAR: toNumber(line, 13..23 ),
CD: line[24..27]?.trim() ?: '',
...
]
If you still have trouble you could do something to identify which field was having trouble by trapping the exception (ie NFE), and throwing a new exception in the method with the range in the message to know which part had problems.
You then need to keep track of the line number as you parse things so you can show that back when you find a problem. You'll want to trap exceptions in the loop and write out the line number of the offending line. Probably best to halt the entire process by throwing an exception with the line number:
int lineNumber = 1
reader.eachLine { line ->
try {
....your logic here....
lineNumber++
} catch( Exception ex ) {
throw new IOException( "Exception encountered at line ${lineNumber}: ${ex.getMessage()}", ex )
}
}