Search code examples
amazon-s3docker-composemlflow

MlFlow - Unable to run with S3 as default-artifact-root


I am trying to store my model artifacts using mlflow to s3. In the API services, we use MLFLOW_S3_ENDPOINT_URL as the s3 bucket. In the mlflow service, we pass it as an environment variable. But, the mlflow container servicer fails with the below exception:

mflow_server  | botocore.exceptions.HTTPClientError: An HTTP Client raised an unhandled exception: Not supported URL scheme s3

docker-compose file as below:

version: "3.3"
services:
  prisim-api:
    image: prisim-api:latest
    container_name: prisim-api
    expose:
      - "8000"
    environment: 
    - S3_URL=s3://mlflow-automation-artifacts/
    - MLFLOW_SERVER=http://mlflow:5000
    - AWS_ID=xyz+
    - AWS_KEY=xyz

    networks:
      - prisim 
    depends_on:
      - mlflow
    links:
            - mlflow
    volumes:
      - app_data:/usr/data
  mlflow:
    image: mlflow_server:latest
    container_name: mflow_server
    ports:
      - "5000:5000"    
    environment:
      - AWS_ACCESS_KEY_ID=xyz+
      - AWS_SECRET_ACCESS_KEY=xyz
      - MLFLOW_S3_ENDPOINT_URL=s3://mlflow-automation-artifacts/
    healthcheck:
      test: ["CMD", "echo", "mlflow server is running"]
      interval: 1m30s
      timeout: 10s
      retries: 3
    networks:
       - prisim 
networks:
 prisim:
volumes:
  app_data:

Why the scheme s3 is not supported?


Solution

  • I found the solution.

    I have added ["AWS_DEFAULT_REGION"] to the environment variables and it worked.