Search code examples
linuxmacosbashsed

sed command with -i option failing on Mac, but works on Linux

I've successfully used the following sed command to search/replace text in Linux:

sed -i 's/old_link/new_link/g' *

However, when I try it on my Mac OS X, I get:

"command c expects \ followed by text"

I thought my Mac runs a normal BASH shell. What's up?

EDIT:

According to @High Performance, this is due to Mac sed being of a different (BSD) flavor, so my question would therefore be how do I replicate this command in BSD sed?

EDIT:

Here is an actual example that causes this:

sed -i 's/hello/gbye/g' *
Solution

  • The -i option (alternatively, --in-place) means that you want files edited in-place, rather than streaming the change to a new place.

    Modifying a file in-place suggests a need for a backup file - and so a user-specified extension is expected after -i, but the parsing of the extension argument is handled differently under GNU sed & Mac (BSD) sed:

    • GNU : "If no extension is supplied, the original file is overwritten without making a backup." - effectively, you can omit specify a file extension altogether. The extension must be supplied immediately after the -i, with no intervening space.
    • Mac (BSD) : "If a zero-length extension is given, no backup will be saved." - you must supply an extension, but it can be the empty string '' if you want, to disable the backup.

    So GNU & Mac will interpret this differently:

    sed -i 's/hello/bye/g' just_a_file.txt
    • GNU : No extension is supplied immediately after the -i, so create no backup, use s/hello/bye/g as the text-editing command, and act on the file just_a_file.txt in-place.
    • Mac (BSD) : Use s/hello/bye/g is the backup file extension (!), use just_a_file.txt as the text-editing command, but uh-oh!: the command code given there is j (not, eg s, a valid command code for substitution), so error with invalid command code j.
    # This still create a `my_file.txt-e` backup on macOS Sonoma (14.5)
# and a `my_file.txt''` on Linux
sed -i'' -e 's/hello/bye/g' my_file.txt

    Placing the extension immediately after the -i (eg -i'' or -i'.bak', without a space) is what GNU sed expects, but macOS expect a space after -i (eg -i '' or -i '.bak').

    and is now accepted by Mac (BSD) sed too, though it wasn't tolerated by earlier versions (eg with Mac OS X v10.6, a space was required after -i, eg -i '.bak').

    The -e parameter allows us to be explicit about where we're declaring the edit command.

    Until Mac OS was updated in 2013, there wasn't

    Still there isn't any portable command across GNU and Mac (BSD), as these variants all failed (with an error or unexpected backup files):

    • sed -i -e ... - works on Linux but does not work on macOS as it creates -e backups
    • sed -i '' -e ... - works on macOS but fails on Linux
    • sed -i'' -e ... - Create -e backups files on macOS and '' backup on Linux

    Portable solution

    You have few options to achieve the same result on Linux and macOS, e.g.:

    1. Use Perl: perl -i -pe's/old_link/new_link/g' *.

    2. Use gnu-sed on macOS (Install using Homebrew)

    # Install 'gnu-sed' on macOS using Homebrew
brew install gnu-sed
# Use 'gsed' instead of 'sed' on macOS.
gsed -i'' -e 's/hello/bye/g' my_file.txt

    Note: On macOS, you could add the bin path of gnu-sed containing the sed command to the PATH environment variable or create an alias for gsed as sed (replacing macOS sed with GNU sed). It is best not to do this, since there may be scripts that rely on the macOS built-in version.

    If you are using sed in a script, you can try to automate switching to gsed:

    #!/usr/bin/env bash
set -Eeuo pipefail

if [[ "$OSTYPE" == "darwin"* ]]; then
  # Require gnu-sed.
  if ! [ -x "$(command -v gsed)" ]; then
    echo "Error: 'gsed' is not istalled." >&2
    echo "If you are using Homebrew, install with 'brew install gnu-sed'." >&2
    exit 1
  fi
  SED_CMD=gsed
else
  SED_CMD=sed
fi

# Use '${SED_CMD}' instead of 'sed'
${SED_CMD} -i'' -e 's/hello/bye/g' my_file.txt
    1. Use -i '' on macOS and BSD or -i (GNU sed) otherwise
    #!/usr/bin/env bash
set -Eeuo pipefail

case "$OSTYPE" in
  darwin*|bsd*)
    echo "Using BSD sed style"
    sed_no_backup=( -i '' )
    ;; 
  *)
    echo "Using GNU sed style"
    sed_no_backup=( -i )
    ;;
esac

sed "${sed_no_backup[@]}" -e 's/hello/bye/g' my_file.txt