Search code examples
c#jsoncsvchoetl

How to output JSON array as a single field in CSV using ChoETL


I'm using ChoETL to convert JSON to CSV. Currently, if a property in the JSON object is an array it is output into separate fields in JSON. Example:

{
    "id", 1234,
    "states": [
        "PA",
        "VA"
     ]
},
{
    "id", 1235,
    "states": [
        "CA",
        "DE",
        "MD"
     ]
},

This results in CSV like this (using pipe as a delimeter)

"id"|"states_0"|"states_1"|"states_2"
"1234"|"PA"|"VA"
"1235"|"CA"|"DE"|"MD"

What I would like is for the array to be displayed in a single states field as a comma separated string

"id"|"states"
"1234"|"PA,VA"
"1235"|"CA,DE,MD"

Here is the code I have in place to perform the parsing and transformation.

public static class JsonCsvConverter
{
    public static string ConvertJsonToCsv(string json)
    {
        var csvData = new StringBuilder();
        using (var jsonReader = ChoJSONReader.LoadText(json))
        {
            using (var csvWriter = new ChoCSVWriter(csvData).WithFirstLineHeader())
            {
                csvWriter.WithMaxScanRows(1000);
                csvWriter.Configuration.Delimiter = "|";
                csvWriter.Configuration.QuoteAllFields = true;
                csvWriter.Write(jsonReader);
            }
        }

        return csvData.ToString();
    }
}

Edited: Removed test code that wasn't useful


Solution

  • This is how you can produce the expected output using the code below

    var csvData = new StringBuilder();
    using (var jsonReader = ChoJSONReader.LoadText(json))
    {
        using (var csvWriter = new ChoCSVWriter(csvData)
            .WithFirstLineHeader()
            .WithDelimiter("|")
            .QuoteAllFields()
            .Configure(c => c.UseNestedKeyFormat = false)
            .WithField("id")
            .WithField("states",  m => m.ValueConverter(o => String.Join(",", ((Array)o).OfType<string>())))
            )
        {
            csvWriter.Write(jsonReader);
        }
    }
    
    Console.WriteLine(csvData.ToString());
    

    Output:

    id|states
    "1234"|"PA,VA"
    "1235"|"CA,DE,MD"
    

    PS: on the next release, this issue will be handled automatically without using valueconverters