“Search Queries” data from Google WMT

Question

I want to get the key words (aka "queries") that are used by users to find a webpage on google. The same keywords that are shown in Google Analytics. I want to get them using the API (Search Console / Webmaster Tool API). As mentioned in the accepted answere here there was a time when this part of the Google WebMaster tools API was not available to public.

I was wondering if this is still valid because I found no official google page which says so. Nevertheless I was not able to retrieve data from the API.

I tried to code a script (using Perl with Net::Google::WebmasterTools). I was able to authorize and send a request. I also got a reply with status 200 but containing no data (especially no keywords, which is what I want to get). Nevertheless, I can see the keywords when browsing to the Search WebmasterTools Analytics Report.

#!/usr/bin/perl

use Net::Google::WebmasterTools;
use Net::Google::WebmasterTools::OAuth2;
use Data::Dumper;
use URL::Encode 'url_encode';

my $site_url      = url_encode("http://www.example.com");
my $client_id     = "[ID]";
my $client_secret = "[SECRET]";
my $refresh_token = "[TOKEN]";

my $wmt = Net::Google::WebmasterTools->new;

my $oauth = Net::Google::WebmasterTools::OAuth2->new(
    client_id     => $client_id,
    client_secret => $client_secret,
);

my $token = $oauth->refresh_access_token($refresh_token);
print Dumper($token);

$wmt->token($token);

# Build request
my $req = $wmt->new_request(
    site_url            => "$site_url",
    report_name         => "searchAnalytics",
    method              => "query",
    dimensions          => ['Country','Device','Query'],
    #search_type        => 'web',
    start_date          => '2015-01-01',
    end_date            => '2015-09-30',
    row_limit           => 1000,
);
print $req;
# Send request
my $res = $wmt->retrieve($req);
die("GWMT error: " . $res->error_message) if !$res->is_success;
# Print results

print Dumper($res);

print
    "Results: 1 - ", $res->items_per_page,
    " of ", $res->total_results, "\n\n";

for my $row (@{ $res->rows }) {
    print
        $row->get_source,  ": ",
        $row->get_visits,  " visits, ",
        $row->get_bounces, " bounces\n";
}

print
    "\nTotal: ",
    $res->totals("visits"),  " visits, ",
    $res->totals("bounces"), " bounces\n";

I also tried using the API Explorer for webmasters.searchanalytics.query but as response I get 500 Internal Server Error.

I'm not sure if I use the api in a wrong way or if it is still not supported. Does anyone have "recent" experience about that? (maybe by using another programming language or library).

Answer 1

The API is fully operational.

Try ['country','device','query'] instead of the propercase equivalents.

...and the docs are wrong regarding paging -- it DOES support paging, but there is a caveat: stop when a page contains less than 5000 rows. Also you will have to deduplicate between pages since the output is sorted only on the 'clicks' column and below that is randomly/inconsistently sorted, which means you can get the same row in multiple pages if the page-break occurs when there are 2 or more rows with that number of clicks (particularly bad with 0 click rows).