Search code examples
apache-nifi

ConvertJSONtoSQL and Putsql - Error when timestamp is null


Reading a CSV and inserting the data to a table. Using a convertJsonToSQL and put sql processor.

To handle the null values in the timestamps I am using

1) EvaluateJsonPath (strip out all the timestamp fields and assign as sql.args.7.original) and

2)then Use a ConvertJSONToSQL after which

3) I use an UpdateAttribute -- Where I check for nulls and assign null values and if they original values are not null then I assign the corresponding value. (IF NULL sql.args.7.value = NULL)

The flow works perfecting if there is a non-null value.

If there is a null value It errors out on the PUTSQL processor with the following error "the value of sql.args.7.value is NULL which cannot be converted to timestamp".

PS: I did try to pass empty string/-1/null with and without quotes. These does not seem to work.

UPDATE:

expression used if the value is null name : sql.args.7.value, value : null.

IF not Null then ${sql.args.7.original:toDate("yyyy-MM-dd"):toNumber()}

I did try without using assigning a null value as well.(For instance process only if the value is not null. That use case also fails)


Solution

  • I think because your values are in attributes (for PreparedStatement parameter values) and attributes have string values, you might not be able to have a real null as a value, but I'll look into it. In the meantime, maybe try PutDatabaseRecord, that should alleviate the need to convert the JSON to SQL and thus doesn't use attributes for the column values.