Search code examples
linuxbashcurlgreptee

Tee pipe into 3 different processes and grepping the second match


I am trying to create a bash script which shows me the latest stats about corona infection numbers in the countries Germany and Switzerland and also in the whole world.

corona () {
    curl -s https://corona-stats.online\?minimal\=true | tee >(head -n 1) > >(grep "(CH)\|(DE)")
    curl -s https://corona-stats.online\?minimal\=true | tail -n 20 | grep World
}

As you can see, to do this I had to create this very ugly script where curl is called twice. I had to do this because the website looks like this:

Rank World                               Total Cases  New Cases ▲ Total Deaths New Deaths ▲ Recovered  Active    Critical Cases / 1M pop
1    USA (US)                               7,497,256     2,585 ▲      212,694         34 ▲  4,737,369 2,547,193   14,190         22,617
2    India (IN)                             6,397,896     5,936 ▲       99,833         29 ▲  5,352,078   945,985    8,944          4,625
3    Brazil (BR)                            4,849,229                  144,767               4,212,772   491,690    8,318         22,773
4    Russia (RU)                            1,194,643     9,412 ▲       21,077        186 ▲    970,296   203,270    2,300          8,185
...
22   Germany (DE)                             295,943       413 ▲        9,586                 259,500    26,857      362          3,529
...
58   Switzerland (CH)                          54,384       552 ▲        2,075          1 ▲     45,300     7,009       32          6,272
...
     World                                 34,534,040    63,822 ▲    1,028,540      1,395 ▲ 25,482,492 8,023,008   66,092       4,430.85


Code: https://github.com/sagarkarira/coronavirus-tracker-cli
Twitter: https://twitter.com/ekrysis

Last Updated on: 02-Oct-2020 12:10 UTC

US STATES API: https://corona-stats.online/states/us
HELP: https://corona-stats.online/help
SPONSORED BY: ZEIT NOW
Checkout fun new side project I am working on: https://messagink.com/story/5eefb79b77193090dd29d3ce/global-response-to-coronavirus

I only want to display the first line, the last line of the table (World) and the two lines about Germany and Switzerland. I manged to display the first line as well as the two countries by piping the output of curl into head -n 1 and grepping the country codes. I was able to do both things thanks to this answer.

Now I want to get the last line in the table, the one where the current cases of the whole World are displayed. I tried to use tee again to pipe it into a third process tee >(head -n 1) > >(grep "(CH)\|(DE)") > >(tail -n 20 | grep World). But that didn't work. My first question is, how can I pipe an output into 3 different processes using tee?

The second question revolves around the way I try to grep the World line. I tail the last 20 lines and then grep "World". I do this because if I simply grep "World", it only return the title line where "World" can also be found. So my second question is: How can I grep only the last or second occurance?


Solution

  • You can chain several tee commands and throw away only the last output of tee:

    curl -s ... | tee >( cmd1 ) | tee >( cmd2 ) | tee > >( cmd3 )
    

    Actually, we can shorten it to:

    curl -s ... | tee >( cmd1 ) | tee >( cmd2 ) | cmd3
    

    because we do not use the output of the last tee anyway.

    Having multiple commands write to the terminal at the same time might get the output mixed up. A much more elegant solution is to use only one grep, e.g.

    curl -s ... | grep '(DE)\|(CH)\|World.*,'
    

    The expression World.*, will just look for a comma in the same line after World, in order to exclude the head line.