How can I make my dotfiles public with a private fork or repository?

I want to make my dotfiles public. I found {Yadm](https://thelocehiliosan.github.io/yadm/) which looks to be what I want.

Secret data I don't really want in version control at all. I'll store this offline on a USB stick somewhere (ssh keys, gpg keys, etc.). However, there are other files which contain information I want to filter out or change before making public.

Yadm also allows me to have alternate files within the same repository, so I am thinking this design is what I should go with, rather than separate branches.

The reason for having a private repository is because certain files like irc.conf have strings I want to substitute or sanitize. I might want to include this file, but delete all lines that contain SECRET_IRC_NETWORK.

~/.weechat/irc.conf:SECRET_IRC_NETWORK.addresses = "irc.secret.example.com"
~/.weechat/irc.conf:SECRET_IRC_NETWORK.ssl = on
~/.weechat/irc.conf:SECRET_IRC_NETWORK.ssl_cert = "~/.weechat/ssl/SECRET_IRC/SECRET_IRC-SECRET_NAME.pem"
~/.weechat/irc.conf:SECRET_IRC_NETWORK.ssl_priorities = "NORMAL:-VERS-SSL3.0"
~/.weechat/irc.conf:SECRET_IRC_NETWORK.ssl_dhkey_size
~/.weechat/irc.conf:SECRET_IRC_NETWORK.ssl_fingerprint = "SECRET_FINGERPRINT"
~/.weechat/irc.conf:SECRET_IRC_NETWORK.ssl_verify = on
~/.weechat/irc.conf:SECRET_IRC_NETWORK.sasl_username = "SECRET_USERNAME"
~/.weechat/irc.conf:SECRET_IRC_NETWORK.nicks = "SECRET_NAME"
~/.weechat/irc.conf:SECRET_IRC_NETWORK.username = "SECRET_USERNAME"
~/.weechat/irc.conf:SECRET_IRC_NETWORK.realname = "SECRET_NAME"

Another example might be my iptables rules configuration rules6-save:

~/etc/iptables/rules6-save:-A INPUT -s `2001:MY:SECRET:ASSIGNED:RANGE::/64 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT

I would want to substitute 2001:MY:SECRET:ASSIGNED:RANGE with 2001:db8:AAA:AAA:AAA::/64 before it goes public. Is https://git-scm.com/docs/gitattributes "filter driver" how I do this?

Do I need to commit to the public repository (red), or the private fork (green)?

I figure it would work like this:

Comitting to the private repo
Filtering and pushing to public repo (being very careful to check everything is correctly sanitized)

Cherry pick some commit from public
Filtering and merging with private
1. From my research, if you have a public repository on Github/Gitlab you cannot have a private fork and make any pull requests from it. Ideally the private one would be on my own private server.

How can I get around this? I don't want people to know the private fork exists. Therefore it would be important that any commits from the private fork have the author also sanitized.

(Above diagram dot file).

digraph graphname {
    node [shape=rectangle, style="filled"];

    dotfiles [fillcolor="#ff9999", label="master (public) \n Desktop, laptop, workstation, server, vm"];
    friendsworkstation [fillcolor="#ffffbb", label="Friend's Workstation (private)"]
    prFromFriend [fillcolor="#ff9999", label="Friend opens PR (public)"]
    { rank=same dotfilesPrivate friendsworkstation prFromFriend }

    dotfilesPrivate [fillcolor="#99ff99", label="master (private) \n Desktop, laptop, workstation, server, vm"]

    dotfiles -> dotfilesPrivate
    dotfiles -> friendsworkstation
    dotfilesPrivate -> dotfiles
    prFromFriend -> dotfiles
}

Solution

So I ended up solving this. Esentially what I did was used YADM's bootstrap option. I created a bootstrap template bootstrap##yadm.j2

#!/usr/bin/env bash

SUBMODULE_INIT_VIM=`jq '.submodule_init_vim' ~/.yadm/bootstrap_vars.json`

{% if YADM_CLASS == 'Workstation' -%}
    SSH_HOSTS=$(cat ~/template_data/ssh/hosts.json) envtpl --keep-template ~/.ssh/config##Workstation.tpl -o ~/.ssh/config
    SEC=$(cat ~/template_data/weechat/sec.json) PASSPHRASE='tiddles' envtpl --keep-template ~/.weechat/sec.conf##Workstation.tpl -o ~/.weechat/sec.conf
    SERVERS=$(cat ~/template_data/weechat/servers.json) envtpl --keep-template ~/.weechat/irc.conf##Workstation.tpl -o ~/.weechat/irc.conf
    PATHS=$(cat ~/template_data/shell/path.json##Workstation.Linux) envtpl --keep-template ~/.bashrc##Workstation.Linux.tpl -o ~/.bashrc
    if [ $SUBMODULE_INIT_VIM = "true" ]; then
        vim '+PlugUpdate' '+PlugClean!' '+PlugUpdate' '+qall'
    elif [ $SUBMODULE_INIT_VIM = "false" ]; then
        echo "Not initalizing submodules for vim"
    fi
RUN_PACKAGE_MANAGER=`jq '.run_package_manager' ~/.yadm/bootstrap_vars.json`
if [ $RUN_PACKAGE_MANAGER = "true" ]; then
    {% if YADM_DISTRO == 'Arch' -%}
    PACMAN_PACKAGES=(`jq -r 'join(" ")' ~/template_data/packages/pacman.json`)
    echo "Running sudo pacman -Syu" ${PACMAN_PACKAGES[@]}
    sudo pacman -Syu ${PACMAN_PACKAGES[@]}
    if [ -f /usr/bin/yay ]; then
        YAY_PACKAGES=(`jq -r 'join(" ")' ~/template_data/packages/yay.json`)
        echo "Running yay -Syu" ${YAY_PACKAGES[@]}
        yay -Syu ${PACMAN_PACKAGES[@]}
    else
        echo "Yay doesn't exist"
    fi{%
    elif YADM_DISTRO == 'Debian' -%}
    APT_PACKAGES=(`jq -r 'join(" ")' ~/template_data/packages/debian.json`)
    echo "Running sudo apt-get install" ${APT_PACKAGES[@]}
    sudo apt-get install ${APT_PACKAGES[@]}
    {% else -%}echo "Unknown distribution"{% endif %}
elif [ $RUN_PACKAGE_MANAGER = "false" ]; then
echo "Not installing any packages"
fi
{% elif YADM_CLASS == 'Router' -%}
    MY_RANGE='2001:db8:AAA:AAA:AAA' envtpl --keep-template ~/.config/etc/iptables/rules6-save##Router.tpl -o ~/.config/etc/iptables/rules6-save
{% elif YADM_CLASS == 'VirtualMachine' -%}
   echo "NOTE: Some configs for virtual machines"
{% else -%}
   echo "ERROR: Unknown class selected"
{% endif -%}

I created some bootstrapping variables and read them in with jq. This allowed me to skip certain parts of the initialization.

{
  "submodule_init_vim": false,
  "run_package_manager": true
}

I template-atized many of my configuration files and put the template data in ~/template_data.

If you look at my .bashrc config you can see how I read in the path:

export PATH="{% for v in PATHS | from_json %}{{v.path |join(':')}}{% endfor %}"

Which takes in the data from the /template_data/shell/path.json##Linux.

[
    {"path": ["/usr/local/sbin",
              "/usr/local/bin",
              "/usr/sbin",
              "/usr/bin",
              "/sbin",
              "/bin",
              "/usr/libexec",
              "$HOME/.local/bin"]
    }
]

This was a simple example, but I did the same for my SSH hosts too.

{% for v in SSH_HOSTS | from_json %}{%
    if v.Host != '' %}Host {{ v.Host }}{%
    endif %}{%
    if v.Comment != '' %}
    {{ v.Comment }}{% endif %}{%
    if v.Hostname != '' %}
    Hostname {{ v.Hostname }}{%
    endif %}{%
    if v.Port != '' %}
    Port {{ v.Port }}{% endif %}{%
    if v.User != '' %}
    User {{ v.User }}{% endif %}{%
    if v.HostKeyAlgorithms != '' %}
    HostKeyAlgorithms {{ v.HostKeyAlgorithms }}{%
    endif %}{%
    if v.KexAlgorithms != '' %}
    KexAlgorithms {{ v.KexAlgorithms }}{% endif %}{%
    if v.Ciphers != '' %}
    Ciphers {{ v.Ciphers }}{% endif %}{%
    if v.MACs != '' %}
    MACs {{ v.MACs }}{% endif %}{%
    if v.PasswordAuthentication != '' %}
    PasswordAuthentication {{ v.PasswordAuthentication }}{% endif %}{%
    if v.IdentifyFile != '' %}
    IdentityFile {{ v.IdentifyFile }}{% endif %}

{% endfor %}

Where I read in the template data from template_data/ssh/hosts.json

[
    {
        "Host":"NSA",
        "Comment": "# Compute with world's dick pix",
        "Hostname":"203.0.113.1",
        "Port": "",
        "User": "nsa",
        "HostKeyAlgorithms":"",
        "KexAlgorithms": "",
        "Ciphers": "",
        "MACs":"",
        "PasswordAuthentication": "",
        "IdentifyFile":"~/.ssh/id_ed25519_nsa"
    },
    {
        "Host":"CIA",
        "Comment": "",
        "Hostname":"203.0.113.2",
        "Port": "",
        "User": "cia",
        "HostKeyAlgorithms":"",
        "KexAlgorithms": "",
        "Ciphers": "",
        "MACs":"",
        "PasswordAuthentication": "",
        "IdentifyFile":"~/.ssh/id_ed25519_cia"
    }
]