Parsing JSON block by block

Question

I have a JSON file with a list of customers and date.

The file looks like this:

{
"Customers": [
{
  "Customer": "Customer Name Here",
  "Company": "Super Coffee",
  "First Name": "First Name Here",
  "Main Phone": "777-777-7777",
  "Fax": "777-777-7777",
  "Bill to 1": "Billing Address One",
  "Bill to 2": "Billing Address Two",
  "Bill to 3": "Billing Address Three",
  "Ship to 1": "Shipping Address One",
  "Ship to 2": "Shipping Address Two",
  "Ship to 3": "Shipping Address Three",
  "Customer Type": "Dealer/Retail"
},
{
  "Customer": "Customer Name Here",
  "Company": "Turtle Mountain Welding",
  "First Name": "First Name Here",
  "Main Phone": "777-777-7777",
  "Fax": "777-777-7777",
  "Bill to 1": "Billing Address One",
  "Bill to 2": "Billing Address Two",
  "Bill to 3": "Billing Address Three",
  "Ship to 1": "Shipping Address One",
  "Ship to 2": "Shipping Address Two",
  "Ship to 3": "Shipping Address Three",
  "Customer Type": "Dealer/Retail"
},
{
  "Customer": "Customer Name Here",
  "Company": "Mountain Equipment Coop",
  "First Name": "First Name Here",
  "Main Phone": "777-777-7777",
  "Fax": "777-777-7777",
  "Bill to 1": "Billing Address One",
  "Bill to 2": "Billing Address Two",
  "Bill to 3": "Billing Address Three",
  "Ship to 1": "Shipping Address One",
  "Ship to 2": "Shipping Address Two",
  "Ship to 3": "Shipping Address Three",
  "Customer Type": "Dealer/Retail"
},
{
  "Customer": "Customer Name Here",
  "Company": "Best Soup Inc.",
  "First Name": "First Name Here",
  "Main Phone": "777-777-7777",
  "Fax": "777-777-7777",
  "Bill to 1": "Billing Address One",
  "Bill to 2": "Billing Address Two",
  "Bill to 3": "Billing Address Three",
  "Ship to 1": "Shipping Address One",
  "Ship to 2": "Shipping Address Two",
  "Ship to 3": "Shipping Address Three",
  "Customer Type": "Dealer/Retail"
}
]
}

I need to be able to extract data from the file block by block, instead of line by line.

I'm used to parsing files line by line to get the data, but with JSON, I need to somehow read it block by block (or more precisely, object by object?). I need to read it by whats inside the brackets for each customer. That way I can write a script that extracts the data I need, and builds a CSV file from it.

For example:

i="1"
for file in *.json; do
     customername=$(jsonblock$i:customername);
     customerAddress=$(jsonblock$i:customeraddress);
     etc...
     i=$[i+1]
done

I understand how this is done when reading a file line by line, but how can I read each JSON block as if it was a line so to speak?

Answer 1

If your intention is simply to print the JSON data in CSV format then you are asking the wrong question. You should parse the entire JSON document and process the Customers array item by item.

Using Perl's JSON and Text::CSV modules, that would look like this

use strict;
use warnings;

use JSON 'from_json';
use Text::CSV ();

my @columns = (
  'Bill to 1',  'Bill to 2',     'Bill to 3', 'Company',
  'Customer',   'Customer Type', 'Fax',       'First Name',
  'Main Phone', 'Ship to 1',     'Ship to 2', 'Ship to 3',
);

my $out_fh = \*STDOUT;
my $json_file = 'customers.json';

my $data = do {
  open my $fh, '<', $json_file or die qq{Unable to open "$json_file" for input: $!};
  local $/;
  from_json(<$fh>);
};
my $customers = $data->{Customers};

my $csv = Text::CSV->new({ eol => $/ });
$csv->print($out_fh, \@columns);

for my $customer ( @$customers ) {
  $csv->print($out_fh, [ @{$customer}{@columns} ]);
}

output

"Bill to 1","Bill to 2","Bill to 3",Company,Customer,"Customer Type",Fax,"First Name","Main Phone","Ship to 1","Ship to 2","Ship to 3"
"Billing Address One","Billing Address Two","Billing Address Three","Super Coffee","Customer Name Here",Dealer/Retail,777-777-7777,"First Name Here",777-777-7777,"Shipping Address One","Shipping Address Two","Shipping Address Three"
"Billing Address One","Billing Address Two","Billing Address Three","Turtle Mountain Welding","Customer Name Here",Dealer/Retail,777-777-7777,"First Name Here",777-777-7777,"Shipping Address One","Shipping Address Two","Shipping Address Three"
"Billing Address One","Billing Address Two","Billing Address Three","Mountain Equipment Coop","Customer Name Here",Dealer/Retail,777-777-7777,"First Name Here",777-777-7777,"Shipping Address One","Shipping Address Two","Shipping Address Three"
"Billing Address One","Billing Address Two","Billing Address Three","Best Soup Inc.","Customer Name Here",Dealer/Retail,777-777-7777,"First Name Here",777-777-7777,"Shipping Address One","Shipping Address Two","Shipping Address Three"