Search code examples
jsonrubycsvdatetimeepoch

Convert JSON Epoch to DateTime in Ruby for Conversion to CSV File


Hello Stack Overflow Friends,

We use an email metric API that returns data requested in JSON. What I am attempting to do is convert the JSON to a CSV file in Ruby (which I have been able to successfully do using the json-converter gem). But, obviously JSON returns date/time fields in Epoch which is useless to most people. Therefore, I'm trying to figure out 2 things:

1) How to appropriately convert Epoch into a more human-readable date/time for the CSV file, and

2) Where along the way in my Ruby code do I need to attempt to do this - before the JSON to CSV conversion or after?

Here's the sample code so far:

JSON Response

{
  "data": [
    {
      "id": 571823,
      "subCommunityId": 2,
      "emailName": "Email Name",
      "fromName": "John Doe",
      "fromAddress": "[email protected]",
      "subjectLine": "This is my subject",
      "preHeader": "",
      "categoryName": "General Communication",
      "sentCount": 15678,
      "scheduledDateTimestamp": 1504889858000,
      "actualSendTimestamp": 1504889858000,
      "dateAdded": 1504889859576
    }
  ],
  "startAt": 0,
  "maxResults": 1000,
  "total": 1
}

Ruby Script - JSON to CSV

require 'json_converter'
converter = JsonConverter.new
json = File.read("emailresponse.json")
converter.write_to_csv(json, "emailreporting.csv")

Any thoughts on how best to do this are welcome.


Solution

  • Actually, Ruby already can build CSV.

    require 'json'
    require 'csv'
    
    json = '{
      "data": [
        {
          "id": 571823,
          "subCommunityId": 2,
          "emailName": "Email Name",
          "fromName": "John Doe",
          "fromAddress": "[email protected]",
          "subjectLine": "This is my subject",
          "preHeader": "",
          "categoryName": "General Communication",
          "sentCount": 15678,
          "scheduledDateTimestamp": 1504889858000,
          "actualSendTimestamp": 1504889858000,
          "dateAdded": 1504889859576
        }
      ],
      "startAt": 0,
      "maxResults": 1000,
      "total": 1
    }'
    hash = JSON.parse(json)
    
    CSV.open('emailreporting.csv', 'w') do |csv| 
      # write headers row
      csv << hash['data'].first.keys 
    
      # write data rows
      hash['data'].each do |data| 
        data['scheduledDateTimestamp'] = Time.at(data['scheduledDateTimestamp'] / 1000)
        data['actualSendTimestamp'] = Time.at(data['actualSendTimestamp'] / 1000) 
        data['dateAdded'] = Time.at(data['dateAdded'] / 1000) 
        csv << data.values  
      end
    end
    

    Building CSV manually gives some advantages:

    1. using Ruby's standard library rather than some little-known gem
    2. more control during building csv