Search code examples
amazon-sagemaker

Configuration local JupyterLab linked with SageMaker


I am following the manual at https://aws.amazon.com/blogs/machine-learning/schedule-your-notebooks-from-any-jupyterlab-environment-using-the-amazon-sagemaker-jupyterlab-extension/ to link our local JupyterLab instance with SageMaker facilities primarily to run jobs on powerful machines in the Amazon cloud, but cannot run a job because of error: [Errno 2] No such file or directory: '/opt/ml/metadata/resource-metadata.json'

I am aware of https://docs.aws.amazon.com/sagemaker/latest/dg/nbi-metadata.html but cannot understand how I am supposed to obtain ResourceArn and ResourceName. In order to to find out I have run an instance of the JL inside the SageMaker where I have revealed a resource-metadata.json:

{
    "AppType": "JupyterLab",
    "DomainId": "d-4xxxxxxxxxxx",
    "SpaceName": "test",
    "ExecutionRoleArn": "arn:aws:iam::000000000000:role/service-role/AmazonSageMaker-ExecutionRole-20240412T112060",
    "ResourceArn": "arn:aws:sagemaker:us-east-1: 000000000000:app/d-4xxxxxxxxxxx/test/JupyterLab/default",
    "ResourceName": "default",
    "AppImageVersion": ""
}

ResourceArn targets to the real cloud instance of the JL which I do not have in my scenario where JS is a local instance.

What am I missing? Thank you.


Solution

  • amazon-sagemaker-jupyter-scheduler 3.0.11 has an issue where it requires '/opt/ml/metadata/resource-metadata.json' file. It is available on SageMaker notebook environments, but not available on local machines, EC2, etc.

    Please downgrade the amazon-sagemaker-jupyter-scheduler to 2.* (as the blog wrote). You can delete the resource-metadata.json file you added to your environment.