Parsehub API PHP

Question

How can I dump the results into a MySql db? Specially decode gzip and parse it to an HP array then dump into a db.

<?php

$connection = mysqli_connect("localhost","root","Aruns@cal");
if (!$connection) {
    die("Database connection failed: " . mysqli_error());
}

$db_select = mysqli_select_db($connection, "arunslocal");
if (!$db_select) {
    die("Database selection failed: " . mysqli_error());
}

$params = http_build_query(array(
  "api_key" => "xxxxxxx",
  "format" => "JSON"
));

$result = file_get_contents(
    'https://www.parsehub.com/api/v2/projects/tmnx0bN0Ty0U/last_ready_run/data?'.$params,
    false,
    stream_context_create(array(
        'http' => array(
            'method' => 'GET'
        )
    ))
);

echo "1"; 
$arr = json_decode($result, true);
echo "2"; 
foreach((array)$arr as $item){
    echo "3"; 
    $name = $item['name'];
    $value = $item['value'];
    echo "4"; 

    $sql = "INSERT INTO `arunslocal`.`scrap_yielddata`( `yield_name`, `yield_value`) VALUES(1,3)";
    //$sql = "INSERT INTO `arunslocal`.`scrap_yielddata`( `yield_name`, `yield_value`) VALUES('$name','$value')";

        if(!mysqli_query($connection,$sql))
        {
            die('Error : ' . mysqli_error($connection));
        }



    }
echo "5"; 
?>

How can I dump the results into a MySql db? Specially decode gzip and parse it to an HP array then dump into a db.

Answer 1

The data is gzip encoded

Your problem is the response data is gzip encoded.

As per parsehub's API documentation,

The Content-Encoding of this response is always gzip.

Solution

Try decoding the gzip response.

You can use gzdecode to decode the gzip response.

Also fetch the API with format json instead.

You can then json_decode to convert it into an array and use it.

Example

<?php
    $params = http_build_query(array(
      "api_key" => "xxxxxx",
      "format" => "json"
    ));

    $result = file_get_contents(
        'https://www.parsehub.com/api/v2/projects/tmnx0bN0Ty0U/last_ready_run/data?'.$params,
        false,
        stream_context_create(array(
            'http' => array(
                'method' => 'GET'
            )
        ))
    );
    $result = gzdecode($result);
    echo $result;
    $parsed_result = json_decode($result, true);
    $yield_data = $parsed_result['yield_data'];
    foreach($yield_data as $item){
        $name = $item['name'];
        $value = $item['value'];

        $sql = "INSERT INTO `arunslocal`.`scrap_yielddata`( `yield_name`, `yield_value`) VALUES('{$name}',{$value})";

        if(!mysqli_query($connection,$sql)) {
            die('Error : ' . mysqli_error($connection));
        }
    }
?>

Output

{
 "yield_data": [
  {
   "name": "Sri Lanka 3M",
   "value": "7.760"
  },
  {
   "name": "Sri Lanka 6M",
   "value": "8.500"
  },
  {
   "name": "Sri Lanka 1Y",
   "value": "9.054"
  },
  {
   "name": "Sri Lanka 2Y",
   "value": "9.283"
  },
  {
   "name": "Sri Lanka 3Y",
   "value": "9.588"
  },
  {
   "name": "Sri Lanka 4Y",
   "value": "9.850"
  },
  {
   "name": "Sri Lanka 5Y",
   "value": "10.000"
  },
  {
   "name": "Sri Lanka 6Y",
   "value": "10.100"
  },
  {
   "name": "Sri Lanka 7Y",
   "value": "10.150"
  },
  {
   "name": "Sri Lanka 8Y",
   "value": "10.200"
  },
  {
   "name": "Sri Lanka 9Y",
   "value": "10.250"
  },
  {
   "name": "Sri Lanka 10Y",
   "value": "10.300"
  },
  {
   "name": "Sri Lanka 15Y",
   "value": "10.575"
  }
 ]
}