Search code examples
google-cloud-platformairflowdirected-acyclic-graphsgoogle-cloud-composergoogle-cloud-shell

Airflow: Is there a way to copy variables from one Composer version to another Composer version using Cloud Shell?


I am working on a migration task, which involves copying variables from one Cloud Composer version to another Cloud Composer version using Cloud Shell.

Though, is it possible to accomplish such a task in Cloud Shell?

I have read the Airflow and Composer documentation; though, I cannot seem to find a working command that will allow me to copy variables from one Composer version to another.


Solution

  • There isn't a single Airflow CLI command to "move" Variables from one Airflow environment to another; however, you can export Variables from source environment to a file and then import from the same file to the target environment.

    Something like this:

    Export from source environment

    gcloud composer environments run SOURCE_ENVIRONMENT_NAME \
        --location SOURCE_LOCATION \
        variables export \
        my_file.json
    

    Import to target environment

    gcloud composer environments run TARGET_ENVIRONMENT_NAME \
        --location TARGET_LOCATION \
        variables import \
        my_file.json