Lets say I had a series of operations I wanted to apply to some data. The programs implementing the operations are not necessarily written in the same language, but they all work by reading from STDIN and writing to STDOUT.
In a unix environment it can be set it up as a pipeline like:
cat data.txt | prog1.sh | prog2.pl | prog3.py | prog4 > out.txt
and it will execute the 4 operations concurrently on the stream of data.
Does the same happen in windows?
I remember testing this out a few years ago with cygwin on windows xp, but I only saw a single prog running in the task manager.
Has anything changed with cygwin, the new xp service packs, or windows 7/8 that would allow for concurrent pipelining? Or has it always worked and I just made a silly mistake in my tests?
I don't have access to a windows machine right now or I'd test it out myself. If someone knows what's going on, I appreciate any help.
While the Unix-like layer implemented by Cygwin has many flaws compared to a native POSIX system or to native Windows programming (especially where performance is concerned), the pipes it implements are quite "real." The programs in the pipeline will run concurrently and will process the data they receive in parallel.
However, as with any pipeline, the speed of the entire operation will be determined by the speed of the slowest component. So if one of the programs in the pipeline is markedly less efficient than the others, it will dominate the CPU usage in the process list.