amazon-web-servicesapache-sparkapache-spark-sqlemramazon-emr
Running an EMR Spark script, and the Spark UI SQL tab disappears
When running spark sql jobs on local mode, my Spark UI has a SQL tab.
When running that same job on AWS EMR, the Spark UI's SQL job is no longer there?
- I've SSH tunneled
- setup the FoxyProxy settings
- can view the various EMR UI's in the browser
- can view the Spark UI in the browser
is there a code reason why this would not be there in Amazon's EMR version of the Spark UI?
Solution
There are two different UI's for Spark in EMR: 1. the Spark UI, and 2. the Spark History Server.
They look the same, but they are served from different URLs, and one of them continues to be served even after an application completes, as long as the EMR cluster is still up and running.
The history server AFAIK does not have a SQL tab
- Email address is not verified (AWS SES)
- SpringBoot fails to deploy after adding .ebextensions for ngingx SSL -[An error occurred during execution of command [app-deploy]]
- What does 0.0.0.0/0 and ::/0 mean?
- How to query dynamoDB based on key + attribute?
- Getting "No AWS accounts are available to you" when using aws configure sso
- Terraform directory Structure for common infrastructure on aws for ADO pipeline
- Mysterious TransactionConflict in TransactionCanceledException
- Cloudwatch VPC interface endpoint times out
- AWS s3 V3 Javascript SDK stream file from bucket (GetObjectCommand)
- How to bypass expectation of S3 server 100-contiune response in Boto3 put_object method
- Disabling AWS X-ray in FastAPI Lambda application
- Cannot Connect To AWS Elasticache Redis Cluster From Local Machine
- How do i access AWS SAM-CLI through bash on windows?
- Reporting AWS Tools RDS or Redshift?
- What does "eksctl create iamserviceaccount" do under the hood on an EKS cluster?
- Configure Selenium Nodes without a JSON file
- Is there a CLI command on Azure to see console messages of a VM that is not booting up?
- Deleting Perforce depot from AWS EC2 server does not free up space on EC2
- Middy is not getting a secret from Secret Manager in a NodeJS AWS Lambda
- Can I include a display name when sending email from the AWS SES Javascript v3 SDK?
- Is it possible to add fields to struct in an existing AWS Athena table?
- Which resource needs a static route through a Transit Gateway API?
- AWS SNS - Message missing a phone number
- How to run a one-off task on AWS ECS Fargate?
- AWS FIFO queues subscription with SNS: passing message group id
- Is it necessary to add health-check config for ecs.CfnTaskDefinition.ContainerDefinitionProperty?
- AWS CLI to get all matching CloudFronts using the "--query" option?
- What is the best way to read a csv and text file from S3 on AWS glue without having to read it as a Dynamic daataframe?
- Why is CloudFormation saying AlreadyExists when creating a AWS::ApiGateway::Authorizer
- AWS sts assume role in one command