Search code examples
phpamazon-web-servicesproxysquidimport.io

Accessing Import.io API through a proxy server


I am having issues using the import.io API.

Despite my app being used and deployed in the UK it will return (for certain stores) incorrect currencies and price data due toe Import.IO's servers being deployed in the US. I spoke with the support team there who helpfully informed me that I could host a proxy server to the Import API.

I managed to get an AWS instance running and have installed Squid as a proxy server. I changed my Firefox connection settings and have successfully managed to browse the web via this proxy server (also verified that my ip was the IP of my server)

However I am not entirely sure how exactly I am meant to call the Import library from within my application.

The application is built in PHP and a current example of how I would generate a URL to call would be:

public function generateCall( $import_key, $url )
{
    return sprintf(
        'https://api.import.io/store/data/%s/_query?input/webpage/url=%s&_user=XXXX&_apikey=%s',
        $import_key, urlencode( $url ), self::$apikey
    );
}

I am calling the api.import.io server directly.


Solution

  • You can use CURL and fetch the API. Then, You can find a proxy from certain country to fetch the API data by country.

    $user = 'User';
    $key = 'key';
    $url = 'https://api.import.io/store/data/%s/_query?input/webpage/url=%s&_user=XXXX&_apikey=%s';
    $proxy = '127.0.0.1:8888';
    //$proxyauth = 'user:password';
    
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL,$url);
    curl_setopt($ch, CURLOPT_PROXY, $proxy);
    //curl_setopt($ch, CURLOPT_PROXYUSERPWD, $proxyauth);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_HEADER, 1);
    $curl_scraped_page = curl_exec($ch);
    curl_close($ch);
    
    echo $curl_scraped_page;