Passing a Variable to SED Command

Question

What is the syntax to pass a variable to sed command that updates the second column in a CSV file. The variable name is $tag This is the command I have used but I don't know where to put the variable exactly.

   basename "$dec" | sed 's/.*/&,A/' >> home/kelsabry/Downloads/Tests/results.csv 

where $decis variable that returns to me a certain directory.

Output:

Downloads, A
Documents, A
etc.

My command to pass the variable into sed to update the second column was:

   basename "$dec" | sed 's/.*/&,'$tag'/' >> home/kelsabry/Downloads/Tests/results.csv 

but it gave me this output:

Downloads, '$tag'
Documents, '$tag'
etc.

So, where should I write the variable $tag in sed command?

Answer 1

Unfortunately, sed is neither aware of fields nor capable of accepting variables. For that, you'd use shell or awk or shell or some other language. sed is a Stream EDitor and in your example is taking input from stdin, not a variable.

If you do want to embed shell variables inside a sed script, understand that you are basically creating your sed script on-the-fly, and it's important to make sure you do it safely.

For example, if there's the possibility that your $tag variable might contain something that will cause misinterpretation of the sed script (i.e. perhaps it came from user input), you need protection. In POSIX shell, perhaps something like this:

if [ "$tag" != "${tag#*[!A-Z]}" ]; then
  printf 'ERROR: invalid tag\n' >&2
  exit 1
fi

or even:

case "$tag" in
  [A-Z]) : ;;
  *) printf 'ERROR: invalid tag\n' >&2; exit 1 ;;
esac

then

# Note the alternative to `basename`
echo "${dec##*/}" | sed 's/$/,'"$tag"'/' >> path/to/file.csv

Note that sed doesn't know anything about fields or CSV. sed is simply being used to append a string on to the end of the line.

Of course, in csh (which perhaps shouldn't be used for scripted automation), you are missing the more useful parameter expansion tools, but you can still protect yourself in other ways:

if ( $%tag == 1 ) then
  switch ($tag)
    case [A-Z]:
      printf '%s,%s\n' `basename "$dec"` "$tag"
      breaksw
    default:
      printf 'ERROR: invalid tag\n'
      exit 1
      breaksw
  endsw
else
  printf 'ERROR: invalid tag\n'
  exit 1
endif

(Note: this is untested. Mileage varies based on multiple conditions. May contain nuts.)

The issue you listed in your question was a quoting problem. You said: sed 's/.*/&,'$tag'/' >. An alternative might be to use awk:

echo "${dec##*/}" | awk -v tag="$tag" '{print $0 OFS tag}' OFS=, >> path/to/file.csv

Awk is a more complete programming language, and supports named variables, unlike sed. The -v option allows you to pre-load an awk variable with the contents of a shell variable.

CSH is considered harmful by some. I'd recommend doing this in a POSIX shell instead, if only to take advantage of the much larger pool of experts who can help with your scripting questions. :)