Search code examples
linuxmacospipestdoutcat

An error occurred while processing STDERR using the pipe


My working directory is as follows:

MyWorkDirectory
└── test

The file "test" contains a line of information:

1:2:3:4:5

When I use the following command:

cat test foo

the output is as follows:

1:2:3:4:5
cat: foo: No such file or directory

I want to ignore STDOUT and only deal with STDERR, and I want to use the cut command to get the third STDERR segment separated by the ":",So I tried the following command:

cat test foo 2>&1 1>/dev/null | cut -d ':' -f 3

I think the output here should be as follows:

No such file or directory

However, there are many different outputs here:

1、

 1
No such file or directory

2、

 No such file or directory
3

3、

2

4、

...

Why are these outputs generated? What commands should I use if I want to achieve my goal?


Solution

  • The issue here is "multiple redirect".

    The reason this can not be reproduced on Ubuntu, is because Ubuntu (like many Debian based Linux distros) uses Ash or Bash shell by default, while recent MacOS version switched to using ZSH.

    ZSH on Ubuntu will give similar result to what you are getting.

    Apparently, using both redirect > and pipe | causes ZSH to split you stderr similar to tee command.

    This post shows a solution to the problem: https://unix.stackexchange.com/questions/265061/how-can-i-pipe-only-stderr-in-zsh

    Specifically, you need to close stdout before redirecting it to /dev/null to prevent the split:

    cat test foo 2>&1 >&- > /dev/null | cut -d ':' -f 3
    

    This works with ZSH on Ubuntu, and should work on Mac OS.
    If it does not, check the linked post for enabling mult_ios option in your shell.