I'm struggling to understand command redirection/reuse...
I understand there's the <(...) <(...)
methodology and the $( ... && ... )
techniques for combining output. But I don't really fully understand what the difference is (I realize that the (...)
dumps you inside a new shell, causing you to potentially jump dirs and lose any vars you haven't exported, but I'm unsure of how it effects the general redirection scheme) and I'm still quite confused as to how to do one-to-many redirection after skimming over the examples and instructions in:
Advanced Bash Scripting Guide 1
Advanced Bash Scripting Guide 2
My own attempts to play around with it have mostly resulted in "ambiguous redirect" errors.
For example, let's say I want to do a one liner given by the pseudocode below
CMD 1 && CMD 2 && CMD 3 --> (1)
CMD 4 (1) --> (2)
CMD 5 (1) --> CMD 6 --> (3)
CMD 7 (2) && CMD 8 (3) --> (4)
CMD 9 (2) --> (5)
CMD 10 (2) (3) -->(6)
VAR= echo (4) && echo (5) && echo (6)
Or as a process diagram
CMD 1 +CMD 2 && CMD 3
|\
| ---> CMD 5 ------> CMD 6-----\
V / V
CMD 4 ----------------u--------> CMD 10
| \ V /
| -------->CMD 7 + CMD 8 /
V | /
CMD 9 | /
\ | /
\ V /
--------> VAR <----------
Where outputs are designated as -->
; storage for reuse in another op is given by -->(#)
; and combination operations are given by &&
.
I currently have no idea how to do this in a single line without redundant code.
I want to truly master command redirection so I can make some powerful one-liners.
Hopefully that's clear enough... I could come up with proof of concept examples for the commands, if you need them, but the pseudocode should give you the idea.
In reply to your comment:
sort <(cd $CURR_DIR && find . -type f -ctime $FTIME) \
<(cd $CURR_DIR && find . -type f -atime $FTIME) \
<(cd $CURR_DIR && find . -type f -mtime $FTIME) | uniq
can be written as (which I believe is clearer)
(find . -type f -ctime $FTIME && find . -type f -atime $FTIME \
&& find . -type f -mtime $FTIME) | sort | uniq
Given three programs one which produces "a" or "z" as output. Produce a string that contains the sorted output and also the unique output as a one liner:
mkfifo named_pipe{1,2,3}; (echo z ; echo a ; echo z) > named_pipe1 & \
tee < named_pipe1 >(sort > named_pipe2) | sort | uniq > named_pipe3 & \
output=$(echo sorted `cat named_pipe2`; echo unique `cat named_pipe3`); \
rm named_pipe{1,2,3}
Produces sorted a z z unique a z
One thing you may notice about this solution is that its been split up so that each grouping of commands has its own line. I suppose what I'm getting at here is that one liners can be cool, but often clarity is better.
The mechanism by how this works is use of a program called tee and named pipes. A named pipe is exactly like an anonymous pipe eg. cat words.txt | gzip
, except that it can be referenced from the file system (but no actual data is written to the file system). Note that writing to a named pipe will block until another process is reading from the named pipe. I've been gratuitous with my use of pipes here, just so you can get a feel of how to use them.
Tee, as noted by others, can replicate input to several outputs.