AWS Databricks pricing - should we also pay for EC2 instances seperately, in addition to DBU costs?
am trying to do some cost comparison between AWS Glue and Databricks hosted on an AWS environment. For the comparison, I have chosen m4.xlarge which is equivalent of 1 DPU in AWS Glue (4 vCPUs/16GB memory).
Assuming I have an pyspark job thats expected to run for 1 hour daily for 30 days with 5DPUs. My cost estimator as per AWS is as follows:
glue cost estimator : 5 DPUs x 30.00 hours x 0.44 USD per DPU-Hour = 66.00 USD (Apache Spark ETL job cost)
Databricks cost estimator : This gives an monthly estimate of 74 USD
Am concerned if we have to pay any EC2 cost to AWS for the 6 nodes in addition to this 73 USD. This is due to the note added in the estimate "This Pricing Calculator provides only an estimate of your Databricks cost. Your actual cost depends on your actual usage. Also, the estimated cost doesn't include cost for any required AWS services (e.g. EC2 instances)."
That will be an additional 36 USD approximately for this instance type/count, in addition to databricks cost. Can someone please clarify so we can make a decision to go with AWS Glue or Databricks. I know in databricks we can choose any instance type, but the question is if i pay EC2 cost seperately. Thanks
The answer is yes.
You should pay for all the infrastructure used directly by Databricks.
As mentioned in the footnote you added: This Pricing Calculator provides only an estimate of your Databricks cost. Your actual cost depends on your actual usage. Also, the estimated cost doesn't include the cost for any required AWS services (e.g. EC2 instances).
Think of it as a software license on top of hardware costs that you would pay anyway, whether you use the software or not.
This point was verified with Databricks Solution Architect that accompanies our company while implementing the Databricks solution.
- Cannot Connect To AWS Elasticache Redis Cluster From Local Machine
- How do i access AWS SAM-CLI through bash on windows?
- Reporting AWS Tools RDS or Redshift?
- What does "eksctl create iamserviceaccount" do under the hood on an EKS cluster?
- Configure Selenium Nodes without a JSON file
- How to bypass expectation of S3 server 100-contiune response in Boto3 put_object method
- Deleting Perforce depot from AWS EC2 server does not free up space on EC2
- Middy is not getting a secret from Secret Manager in a NodeJS AWS Lambda
- Can I include a display name when sending email from the AWS SES Javascript v3 SDK?
- Is it possible to add fields to struct in an existing AWS Athena table?
- AWS SNS - Message missing a phone number
- How to run a one-off task on AWS ECS Fargate?
- AWS FIFO queues subscription with SNS: passing message group id
- Is it necessary to add health-check config for ecs.CfnTaskDefinition.ContainerDefinitionProperty?
- AWS CLI to get all matching CloudFronts using the "--query" option?
- What is the best way to read a csv and text file from S3 on AWS glue without having to read it as a Dynamic daataframe?
- Why is CloudFormation saying AlreadyExists when creating a AWS::ApiGateway::Authorizer
- AWS sts assume role in one command
- GetSecretValue, get identity: get credentials: failed to refresh cached credentials
- Does PutACLAsync make a copy of an object?
- How to use AWS CodeArtifact *within* A Dockerfile in AWSCodeBuild
- AWS CodeBuild: Accessing CodeCommit repository in another account?
- AWS Java SDK SSL Certificates
- How can I use AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to perform actions in AWS through a Jenkins pipeline?
- Syntax error when executing Invoke-DDBQuery to fetch dynamodb record using Powershell
- How can I delete a specific record from my AWS Glue table?
- Start cfn-init in Ubuntu instance with cloudformation (yaml)
- How to check ephemeral storage that is allocated to EKS node
- Permission Error Running Container in AWS CodeBuild
- retrieve list of urls from Amazon S3 bucket using R