My function can index documents in single and bulk to my AWS Elasticsearch from a local Jupyter notebook, but when I deploy to Lambda it keeps returning this error:
"errorMessage": "AuthorizationException(403, 'security_exception', 'no permissions for
[indices:data/write/bulk] and User [name=arn:aws:iam::xxxxxxxxxxxx:role/MyLambdaRole,
backend_roles=[arn:aws:iam::xxxxxxxxxxxx:role/MyLambdaRole], requestedTenant=null]')"
My Elasticsearch domain (v7.7) is configured as such:
Fine-grained access control: Enabled
Master user type: Internal user database
SAML authentication: Disabled
Amazon Cognito for authentication: Disabled
Require HTTPS: Enabled
Encryption at rest: Enabled
KMS master keyarn:aws:kms:us-east-1:xxxxxxxxxxxxx:key/<aws/es key>
Node-to-node encryption: Enabled
The domain's access policy contains:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Resource": "*"
}
]
}
The IAM policy for MyLambdaRole contains:
...
{
"Action": [
"es:*"
],
"Resource": [
"*"
],
"Effect": "Allow"
}
]
In Kibana I have mapped both my AWS admin IAM user and MyLambdaRole under Security -> Role Mappings -> all_access. I have tried different combinations of adding them to Backend roles and also adding them to security_manager.
The Lambda uses AWS Signature v4 authentication and the elasticsearch client is version 7.7.0:
import boto3
from elasticsearch import Elasticsearch, RequestsHttpConnection, helpers
from requests_aws4auth import AWS4Auth
session = boto3.Session()
credentials = session.get_credentials().get_frozen_credentials()
awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, \
session.region_name, 'es', session_token=credentials.token)
host = 'search-es-domain.us-east-1.es.amazonaws.com'
es = Elasticsearch(
hosts = [{'host': host, 'port': 443}],
http_auth = awsauth
use_ssl = True,
verify_certs = True,
connection_class = RequestsHttpConnection
)
# Single indexing call
document = { my data }
es.index(index="my_index", doc_type="_doc", id=doc_id, body=document)
# Bulk indexing call
k = ({ my data })
helpers.bulk(es, k)
If I replace http_auth = awsauth with my Kibana credentials http_auth = (kibana_username, kibana_password) it returns status 200 but then no new documents are created in the index which is weird.
I would like to know what I could be missing or where my configuration could be off.
This is because of fine grained access control being enabled. I ran into the exact same problem and fine grained access control causes issues. Your notebook might be using the master ARN you specified which has access and is always allowed basically.
I recreated my ES instance and disabled fine grained access control and used domain policy only as it suited our set up.
Read more here and notice the highlighted section re user / IAM mixing and not working correctly.
https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/fgac.html