Search code examples
rubysparqlwikidata

POST (not GET) wikidata query


I'm following these instructions to submit a SPARQL query, preferring the POST method as queries may be quite long. But it seems to fail even though the GET works - any way to make a POST query work?

sparql = <<END
SELECT ?item ?itemLabel 
WHERE 
{
    ?item wdt:P31 wd:Q146.
    SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
}
END

# Fails with 405 Not writable
resp = Excon.post('https://query.wikidata.org/sparql', body: "query=#{URI::encode(sparql)}")
puts resp.status, resp.body

# Works with 200
resp = Excon.get("https://query.wikidata.org/sparql?query=#{URI::encode(sparql)}")
puts resp.status, resp.body

I've tried sending "Content-Type" => "application/x-www-form-urlencoded" too, but no luck.


Solution

  • Thanks to the commenters above for showing working examples. I did some further checks based on all obvious combinations, as below.

    In summary:

    • Use GET if it's short and likely to be reused (since GET queries are cached)
    • Ensure you send a user-agent header and note that some libraries won't include one by default (otherwise it returns an unexplained 403)
    • For POST, best and simplest (in my opinion) is to send a SPARQL query as the entire body with "content-type" as "application/sparql-query", no encoding of the query is necessary here. You can alternatively use the form syntax with "content-type" as "application/x-www-form-urlencoded" and ensure the query is encoded.
    require 'excon'
    
    # any arbitrary query   
    sparql = 'SELECT ?item ?itemLabel WHERE { ?item wdt:P31 wd:Q146.  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". } }'
    
    # SUCCESS!
    resp = Excon.get("https://query.wikidata.org/sparql?query=#{URI::encode(sparql)}")
    puts "GET", resp.status, resp.body[0,100], "\n"
    
    # FAIL! 403 (need user agent)
    headers = { "Content-Type" => "application/sparql-query" }
    resp= Excon.post('https://query.wikidata.org/sparql', body: sparql, headers: headers)
    puts "POST sparql-query", resp.status, resp.body[0,100], "\n"
    
    # SUCCESS!
    headers = { "Content-Type" => "application/sparql-query", "User-Agent" => "Ruby 2.6.4" }
    resp= Excon.post('https://query.wikidata.org/sparql', body: sparql, headers: headers)
    puts "POST sparql-query with user-agent", resp.status, resp.body[0,100], "\n"
    
    # FAIL! 403 (need user agent)
    headers = { "Content-Type" => "application/x-www-form-urlencoded" }
    resp = Excon.post('https://query.wikidata.org/sparql', body: "query=#{URI::encode(sparql)}", headers: headers)
    puts "POST form", resp.status, resp.body[0,100], "\n"
    
    # SUCCESS!
    headers = { "Content-Type" => "application/x-www-form-urlencoded", "User-Agent" => "Ruby 2.6.4" }
    resp = Excon.post('https://query.wikidata.org/sparql', body: "query=#{URI::encode(sparql)}", headers: headers)
    puts "POST form with user-agent", resp.status, resp.body[0,100], "\n"
    
    # FAIL! 405 (need encoding)
    resp = Excon.post('https://query.wikidata.org/sparql', body: "query=#{URI::encode(sparql)}")
    puts "POST plain", resp.status, resp.body[0,100], "\n"
    
    # FAIL! 405 (need encoding)
    headers = { "User-Agent" => "Ruby 2.6.4" }
    resp = Excon.post('https://query.wikidata.org/sparql', body: "query=#{URI::encode(sparql)}", headers: headers)
    puts "POST plain with user-agent", resp.status, resp.body[0,100], "\n"
    

    More info