How to capture stderr in a variable and pipe stdout through

Question

I'm trying to store the headers (from stderr) of a curl response in a variable and pipe the body (from stdout) to grep.

Here's my current attempt:

{
  HEADERS=$(curl -vs $URL 2>&1 1>&3-)
  echo "$HEADERS"
} 3>&1 | grep "regex" >> filename
echo "$HEADERS"

When I run the script with bash -x script.sh I see + HEADERS='...' with the expected output, but I can't access them with either $HEADERS nor "$HEADERS" inside and outside the inline group.

The body get's piped through as expected.

Answer 1

As anubhava correctly diagnosed, the problem is that you're setting HEADERS in a sub-process, not in the main process of your shell.

You can use Bash's process substitution to avoid the problem, without using temporary files which have to be cleaned up in the event of the shell being interrupted:

HEADERS=""
{ HEADERS=$(curl -vs "$URL" 2>&1 1>&3-); } 3> >(grep "regex" > file)
echo "$HEADERS"

The grep sub-process is hidden by the process substitution. The space between 3> and >(…) is necessary.