Not able to connect private github repository in apache ariflow helm chart in Kubernetes

Question

I am new working with Airflow and Kubernetes. I am trying to use apache Airflow in Kubernetes.

To deploy it I used this chart: https://github.com/apache/airflow/tree/master/chart.

I want to upload my dags in my github repository so:

gitSync:
    enabled: true
    # git repo clone url
    # ssh examples ssh://[email protected]/apache/airflow.git
    # [email protected]:apache/airflow.git
    # https example: https://github.com/apache/airflow.git
    repo: https://github.com/mygithubrepository.git
    branch: master
    rev: HEAD
    root: "/git"
    dest: "repo"
    depth: 1
    # the number of consecutive failures allowed before aborting
    maxFailures: 0
    # subpath within the repo where dags are located
    # should be "" if dags are at repo root
    subPath: ""

Then I see that to use a private github repository I have to create a secret as is specified in the value.yml file:

# if your repo needs a user name password
    # you can load them to a k8s secret like the one below
    #   ---
    #   apiVersion: v1
    #   kind: Secret
    #   metadata:
    #     name: git-credentials
    #   data:
    #     GIT_SYNC_USERNAME: <base64_encoded_git_username>
    #     GIT_SYNC_PASSWORD: <base64_encoded_git_password>
    # and specify the name of the secret below
    #credentialsSecret: git-credentials

I am creating the secret:

apiVersion: v1
data:
  GIT_SYNC_USERNAME: bXluYW1l
  GIT_SYNC_PASSWORD: bXl0b2tlbg==
kind: Secret
metadata:
  name: git-credentials
  namespace: default

Then I use the secret name in the value.yml file:

repo: https://github.com/mygithubrepository.git
    branch: master
    rev: HEAD
    root: "/git"
    dest: "repo"
    depth: 1
    # the number of consecutive failures allowed before aborting
    maxFailures: 0
    # subpath within the repo where dags are located
    # should be "" if dags are at repo root
    subPath: ""
    # if your repo needs a user name password
    # you can load them to a k8s secret like the one below
    #   ---
    #   apiVersion: v1
    #   kind: Secret
    #   metadata:
    #     name: git-credentials
    #   data:
    #     GIT_SYNC_USERNAME: <base64_encoded_git_username>
    #     GIT_SYNC_PASSWORD: <base64_encoded_git_password>
    # and specify the name of the secret below
    credentialsSecret: git-credentials

but seems not bee working.

Answer 1

I see that you are connecting to your github repo via https.

Try to use:

ssh://[email protected]/mygithubrepository.git

or simply

[email protected]/mygithubrepository.git

You can experience issues with connecting via https especially if you have two-factor authentication enabled on your github account. It's described more in detail in this article.

Also take a look at VonC's answer where he mentions:

As noted in Oliver's answer, an HTTPS URL would not use username/password if two-factor authentication (2FA) is activated.

In that case, the password should be a PAT (personal access token) as seen in "Using a token on the command line".

That applies only for HTTPS URLS, SSH is not affected by this limitation.

As well as at this one provided by rc0r.

But as I said, simply using ssh instead of https should resolve your problem easily.