How can I connect Airflow Cloud Composer to Cloud SQL using Private IP?
I'm attempting to connect to a Potgres database running in Cloud SQL, and I have enabled Private IP as shown below with the "Default" settings. I have also added the IP addresses of this Cloud SQL instance as connections into the Airflow instance on Google Cloud Composer. Note I have also set up the Cloud Composer instance with private networking, and the same "Default" options as I did on the Cloud SQL instance when setting it to use Private IP.
I can test connect fine in the connections tab, however I can't run any queries since it says I don't have the correct permissions when I attempt a DAG run. Am I connecting in the correct way, or have I missed a step in connecting to a network in this way?
This was solved after correcting database permissions, after help from fellow users above
- Debugging Broken DAGs
- Apache airflow doens't display the DAG
- Seeking Advice on Efficient Data Unpivoting in SQL Server with SSIS or Alternative Tools
- How do we trigger single airflow dag multipler times using TriggerDagRunOperator?
- How to consider daylight savings time when using cron schedule in Airflow
- Airflow EMRServerlessCreateApplicationOpertor can't detect application name from airflow input parameter using jinja template
- Airflow Task Group Execution Order
- How to Configure Airflow Celery executor with redis sentinel?
- Running airflow tasks/dags in parallel
- Facing Issue in passing metadata field with DataprocCreateClusterOperator (Airflow 2.0)
- Airflow Helm Chart Worker Node Error - CrashLoopBackOff
- Airflow: how to delete a DAG?
- How to update airflow.cfg file in MWAA local Runner
- Airflow Scheduler Hanging When Run with Remote Database
- Connection refused when calling Postgres container from another docker container
- How to run Airflow Web Console on different port?
- Service account cannot get resource \"sparkapplications/status\" in API group \"sparkoperator.k8s.io\"
- Is there a way to pass the dataproc version via the Airflow DataprocCreateBatchOperator method?
- What is the difference between min_file_process_interval and dag_dir_list_interval in Apache Airflow 1.9.0?
- Apache Airflow unit and integration test
- Airflow it's running two jobs and returning error in Databricks but ran succesfully
- How to use AirFlow to run a folder of python files?
- Unexpected String Concatenation Issue in Airflow 2.7.2 DAG
- Not uploading file to postgres with bulk_load. Apache Airflow
- The task completes with the SKIPPED status if it was started outside the interval of the schedule_interval parameter
- Datadog exclude the logs
- Capture airflow run duration
- Airflow DAG throws GlueJobOperator is not JSON serializable
- Ariflow Tasks in TaskGroup run out of order
- Python doesn't see added module to the same dir