How to exclude some files when transferring between source cloud repo?

Question

Currently I have two google cloud projects, one for staging environment and the other one for production environment. Each of them has a source cloud repo. Whenever I am done with the development, I will merge the code to the master branch of the repo in the staging environment, which will then fire up a cloud build trigger that copies all the code from the repo in the staging environment to the repo in the production environment.

There are some configuration files and credential files that should be different in different environments, so I don't want to copy them otherwise it will overwrite the environment related files in the production repo. Here is my attempt in the cloudbuild.yaml:

steps:
    - name: gcr.io/cloud-builders/git
      args: ['checkout', '--orphan', 'temp']
    - name: gcr.io/cloud-builders/git
      args: ['add', '-u']
    - name: gcr.io/cloud-builders/git
      entrypoint: "bash"
      args:
        - '-c'
        - |
        - git reset `cat .gitcopyignore`
    - name: gcr.io/cloud-builders/git
      args: ['config', '--global', 'user.name', '<name>']
    - name: gcr.io/cloud-builders/git
      args: ['config', '--global', 'user.email', '<email>']
    - name: gcr.io/cloud-builders/git
      args: ['commit', '-am', 'latest production commit']
    - name: gcr.io/cloud-builders/git
      args: ['branch', '-D', 'master']
    - name: gcr.io/cloud-builders/git
      args: ['branch', '-m', 'master']
    - name: gcr.io/cloud-builders/git
      args: ['push', '-f', 'https://source.developers.google.com/p/<production project>/r/<production repo>', 'master']

I put all the environment related files into the .gitcopyignore file and run git reset on them after the git add to exclude them from being copied into the new repo. However, each time this trigger executes, it says all steps are executed successfully but the environment related files in the production environment are still overwritten.

I don't want to add those files into .gitignore because I do want to keep a version control in the repo for the staging environment, just in case my teammates want to modify them. More importantly, these environment related files have to be in the repos, because google cloud needs to pull all the code from the source cloud repo before it can deploy a cloud function.

Can anyone tell me what might be a good fix to my cloudbuild.yaml? It has been bugging me for quite a few days.

Thanks!

Answer 1

In general, trying to maintain two separate repositories is likely to be a bunch of needless work and is also likely to cause your repositories to needlessly diverge.

The recommended way to keep configuration that differs is to either store a template in the repository and copy it into place, modifying it with a script, or store multiple configuration files and copy the correct one over, either by hand or with a script that operates differently based on the environment. The destination files (the versions that are actually used) should be placed in .gitignore, since you won't want to check them in directly. I'm pretty sure most platforms can handle this approach, because it's very common and is a best practice.

This means that you'll have one repository with all your files and a similar history. You won't need to worry about things accidentally diverging.