Search code examples
windowsbashgitforeachgit-submodules

How to run multiple commands within git submodule foreach?


What I have tried

git submodule foreach git checkout main
git submodule foreach git add --all
git submodule foreach git diff-index --quiet HEAD || git commit -m "%CommitMessage%"
git submodule foreach git push

This runs command 1 for all submodules, then command 2 for all submodules, etc.

What I would like

I would like to only have one foreach, and do all of the commands for a submodule at once, then move on to the next submodule.

Question

Is there a way to have the git submodule foreach call a method, or in some way call multiple commands at once?

I want to do this within a batch script on Windows.


Solution

  • To complement larsks' helpful Unix solution (which would also work in Unix-like environments on Windows, such as Git Bash and WSL) with a solution for Windows, from a batch file (see the next section for PowerShell):

    • As in other cases where git supports passing shell commands, these are evaluated by Git Bash, i.e the Bash implementation that comes bundled with git on Windows. As such, you must use Bash (POSIX-compatible) shell syntax even on Windows.

    • Note: git submodule foreach defines the following (environment) variables to provide information about the submodule at hand, which you may reference in your shell command:

      • $name, $sm_path, $displaypath, $sha1 and $toplevel; as noted, due to having to use Bash syntax in your shell command, you need to reference these variables as shown (e.g. as $name instead of batch-file style %name%)
      • See git help submodule for details.
    git submodule foreach "git checkout main && git add --all && git diff-index --quiet HEAD || git commit -m \"%CommitMessage%\" && git push"
    
    • Use "..." quoting on a single line (cmd.exe / batch files don't support multiline quoted strings, and programs only expect " , not also ' to have syntactic function on their process command lines).

      • The %CommitMessage% batch-style variable reference is expanded up front, by cmd.exe, \"...\" is used to properly escape the embedded "..." around the expanded result.

      • Caveat: If the value of %CommitMessage% contains cmd.exe metacharacters such as | and &, the command will break, because cmd.exe sees these as unquoted, due to not understanding that the surrounding \" are escaped double quotes; as you report, set "CommitMessage=%info1% | %info2% | %info3%" caused a problem in your case, and there are two solution options:

        • Either: If feasible, manually ^-escape the metacharacters; e.g.:

          set "CommitMessage=%info1% ^| %info2% ^| %info3%"
          
        • Or, as you have done, use delayed variable expansion, which bypasses the problem (but can result in literal ! characters getting eliminated):

          • Use setlocal enableDelayedExpansion at the top of your batch file.
          • Then use !...! instead of %...% to refer to your variable, i.e. !CommitMessage!
    • Chain the commands with &&, so that subsequent commands only execute if the previous ones succeeded.


    The PowerShell perspective (on both Windows and Unix-like platforms):

    • PowerShell has flexible string literals, including support for multiline literals.

    • The here-string variant used below helps readability and obviates the need for escaping embedded quotes.

    # NOTE: In Windows PowerShell and PowerShell (Core) 7.2-,
    #       you must manually \-escape the embedded " chars.
    #       (... -m \"$CommitMessage\")
    #       $CommitMessage is expanded *by PowerShell*, up front.
    git submodule foreach @"
      git checkout main && 
      git add --all && 
      git diff-index --quiet HEAD || git commit -m "$CommitMessage" && 
      git push
    "@
    

    Important:

    • As noted in the code comments, Windows PowerShell and PowerShell (Core) versions up to v7.2.x unfortunately require embedded " chars. to be explicitly \-escaped when passing arguments to external programs such as git, which is fortunately no longer need in PowerShell (Core) 7.3+

    • Because PowerShell too uses sigil $ for variable references, you must `-escape any $ characters you want to preserve as such, as part of the (POSIX-compatible) shell command to be executed by git; e.g., in order to pass verbatim $name through in order to refer to the submodule name, use `$name

      • However, this only necessary if "..." quoting is used, i.e an expandable string, which in turn is only necessary if you need PowerShell's string interpolation (expansion), such as to embed the value of PowerShell variable $CommitMessage as shown above.

      • If string interpolation isn't needed, use '...' quoting, i.e. a verbatim string ('...'), in which case pass-through $ chars. need no escaping.

    • larsks' Unix solution can be used as-is in PowerShell (Core) 7.3+, but only from Unix-like environments (including WSL, if you have PowerShell (Core) installed there (too)), given that the standard Unix shell, /bin/sh, is explicitly called.

      • As an aside: This technique therefore involves an extra sh process per submodule, but is convenient, because the -e option - to abort when a command fails - allows specifying the commands individually, without having to chain them with &&