Search code examples
bashsedldapbase64

using sed to replace some 64 based encoded strings (calling external echo + base64)

I'm using ldapsearch to query AD servers. Some results are 64 encoded (always with :: before), like:

company: Medidata
company: Milestone
company:: QWRQIFNlcnZpw6dvcw==
company: ROFF

Manually, I can decode those encoded strings with:

echo QWRQIFNlcnZpw6dvcw== | base64 -d

But I'm not able to convert all strings using sed (to replace only the encoded strings).

This works, but does do the replacement. It is used just for debug.

cat output.txt | sed "s/:: \(.\+\)/: `echo \\\1`/"
company: Medidata
company: Milestone
company: QWRQIFNlcnZpw6dvcw==
company: ROFF

What I would like to do is:

cat output.txt | sed "s/:: \(.\+\)/: `echo \\\1 | base64 -d`/"
base64: invalid input
company: Medidata
company: Milestone
company: 
company: ROFF

base64 complains about the input, but it looks good to me.

What am I doing wrong?

Solution

  • With sed "s/:: \(.\+\)/: `echo \\\1`/" and sed "s/:: \(.\+\)/: `echo \\\1 | base64 -d`/" the subshell `...` is expanded before sed even runs. Therefore base64 complains that it cannot decode the literal string \1.

    You have to instruct sed (which has a different syntax than bash) to call an external program. Standard sed cannot call external programs. See How to embed a shell command into a sed expression?. But GNU sed can:

    sed -E "s/(.*:): (.+)/printf %s '\1 '; echo '\2' | base64 -d/e"

    This assumes that the line you are replacing does not contain any '. In your case, this shouldn't be a problem I think.