Elastic Cloud: API endpoint - 404 Not Found on search?

Question

Description

I want to query the data in our Elastic Cloud instance using the REST API with authorization via API Key.

Steps taken

I tried using the SQL API as well as the Search API. NB that I would much preferably use the SQL API.

Based on the following curl commands provided in the documentation:

curl -X POST "localhost:9200/_sql?format=txt&pretty" -H 'Content-Type: application/json' -d'
{
  "query": "SELECT * FROM library WHERE release_date < \u00272000-01-01\u0027"
}
'

[source]

curl -u elastic:password https://CLUSTER_ID.REGION.PLATFORM.found.io:9243/my_index/my_type -XPOST -d '{
"title": "One", "tags": ["ruby"]
}'
{"_index":"my_index","_type":"my_type","_id":"AV3ZeXsOMOVbmlCACuwj","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"created":true}

[source]

and the documentation about the REST API, I attempted the following:

import base64
import json
import requests

if __name__ == '__main__':
    response1 = requests.post(
        'https://{redacted-id}.northamerica-northeast1.gcp.elastic-cloud.com:9243/_sql?format=txt',
        headers={
            "Authorization": base64.standard_b64encode(bytes('{API_KEY_ID}:{API_KEY_KEY}', 'utf-8')),
            "Content-Type": 'application/json'
        },
        data={
            "query": """
                 ...
            """
        }
    )

    print('response1')
    print(response1)

    response2 = requests.get(
        'https://{redacted-id}.northamerica-northeast1.gcp.elastic-cloud.com:9243/logs-pubsub/_search',
        headers={
            "Authorization": base64.standard_b64encode(bytes('{API_KEY_ID}:{API_KEY_KEY}', 'utf-8')),
            "Content-Type": 'application/json'
        },
        data={
            "query": {
                 # ...
            }
        }
    )

    print('response2')
    print(response2)

But both queries answer with 404 - Not Found.

What did I miss? Am I missing a part of the path like /api/..., /rest/...? Or is this a misdirection from a 403 to a 404 and the issue is the API Key?

Thanks!

Answer 1

The kind folks at ElasticStack provided me with the following answer. There were two issues:

The URL above was pointing to the Kibana instance, not to the ElasticCloud instance

In order to get the appropriate URL:

1. Navigate to https://cloud.elastic.co/deployments/
2. Click on your deployment:

3. Click on Copy Endpoint for the Elasticsearch endpoint:

The Authorization header was malformed

The Authorization header for an ApiKey powered request is the following:

ApiKey {B64_ENCODE('API_KEY_ID:API_KEY_KEY')}

Which can be written in python as:

"ApiKey " + str(base64.standard_b64encode(bytes('API_KEY_ID:API_KEY_KEY', 'utf-8')), 'utf-8')

---

On a final note, the team also strongly suggested I look at their Free on-demand Elasticsearch training.

Hopefully this will be helpful to whoever passes by here!