Search code examples
bashvariablessedreplacemultiline

Substituting keyword in string with multi-line variable via sed

Note: The following question is related to this post, but both the focus of the question and the format of the input variable are different (i.e., contents of a multi-line file). Thus, the correct solution is likely different as well.

It appears that replacing a keyword in a string with a multi-line variable via sed works as long as said variable contains explicit newline characters (\n).

$ target="foo \n bar \n baz"
$ echo "qux\nquux" > file
$ solution=$(cat file)
$ echo "$solution"
qux\nquux
$ echo $target | sed -e "s|bar|$solution|"
foo \n qux
quux \n baz

However, when I open the file file in a text editor and substitute the newline character with a linebreak, the replacement with sed fails.

# Newline character was manually replaced with linebreak in texteditor.
$ solution=$(cat file)
$ echo "$solution"
qux
quux
$ echo $target | sed -e "s|bar|$solution|"
sed: -e expression #1, char 9: unterminated `s' command

How do I have to change the sed-command to conduct the sought substitution when the input variable that does not have explicit newline characters?

Solution

  • Introduction/Setup

    sed is not necessarily the appropriate tool for this particular job. Consider the below options -- the samples for each of which expect the following setup:

    # Run this before testing any of the code below
source='bar'
target='This string contains a target: <bar>'
solution='This has
multiple lines
and /slashes/ in the text'

    ...and each of which will emit the following output:

    This string contains a target: <This has
multiple lines
and /slashes/ in the text>

    Note that with sed, you would need to choose a delimiter that isn't used for the expression (thus, with s/foo/bar/, neither foo nor bar can contain a /); the below answers both avoid this limitation.

    Shell-Builtin Parameter Expansion

    The shell can perform the relevant substitution with only built-in string manipulation functionality:

    result=${target//$source/$solution}
echo "$result"

    Perl One-Liners As A sed Alternative

    For longer input strings where the shell's built-in matching is inappropriate, you might also consider a perl one-liner, as described in BashFAQ #21:

    in="$source" out="$solution" perl -pe 's/\Q$ENV{"in"}/$ENV{"out"}/g' <<<"$target"