Search code examples
amazon-s3amazon-sagemakeramazon-kms

Sagemaker API online feature store creation not using given kms key


I am calling Sagemaker API from python script inside EC2 instance to create online feature store. I gave required permission and its creating feature group. However I observed that key I'm passing in below program (online_store_kms_key_id = 'arn:aws:kms:us-east-1:1234:key/1111') is not being used to write objects to s3 bucket instead it's using default bucket key. I'm not sure what is causing this to happen? Why its not using key given in create feature group config? Any idea?

code snippet:

customer_data = pd.read_csv("data.csv",dtype={'customer_id': int,'city_code': int, 'state_code': int, 'country_code': int, 'eventtime': float })

    customers_feature_group_name = "customers-fg-01"
    customers_feature_group = FeatureGroup(name=customers_feature_group_name, sagemaker_session=sagemaker_session
                                           )

    current_time_sec = int(round(time.time()))

    record_identifier_feature_name = "customer_id"

    customers_feature_group.load_feature_definitions(data_frame=customer_data)

    customers_feature_group.create(
        s3_uri="s3://xxxx/sagemaker-featurestore/",
        record_identifier_name=record_identifier_feature_name,
        event_time_feature_name="eventtime",
        role_arn='arn:aws:iam::1234:role/role-1234',
        enable_online_store=True,
        online_store_kms_key_id = 'arn:aws:kms:us-east-1:1234:key/1111'
    )

Solution

  • For encryption of data stored in s3 ( offline store ) you need to add a field 'offline_store_kms_key_id ' to the create() method call, please refer the document below

    https://sagemaker.readthedocs.io/en/stable/api/prep_data/feature_store.html#sagemaker.feature_store.feature_group.FeatureGroup.create

    Also please go through the below document to check the policies and also to verify if you have a symmetric customer managed keys or asymmetric customer managed keys as feature store only supports symmetric keys.

    https://docs.aws.amazon.com/sagemaker/latest/dg/feature-store-security.html