Search code examples
bashprocesslimitsubshell

Limiting the number of subshells spawned


I'm trying to limit the amount of subshells that are spawned in a script that I'm using to sniff our internal network to audit Linux servers in our network. The script works as intended, but due to the way I'm nesting the for loop, it spawns 255 Sub Shells for each network, so therefore it kills the CPU due to the fact that there are over 1000 processes spawned off. I need to be able to limit the amount of processes, and since variables lose their value when in a Sub Shell, I can't figure out a way to make this work. Again, the script works, it just spawns a ton a processes - I need to limit it to, say 10 Processes max:

#!/bin/bash

FILE=/root/ats_net_final

for network in `cat $FILE`;do
                for ip in $network.{1..255};do
                        (
                                SYSNAME=`snmpwalk -v2c -c public -t1 -r1 $ip sysName.0 2>/dev/null | awk '{ print $NF }'`
                                SYSTYPE=`snmpwalk -v2c -c public -t1 -r1 $ip sysDescr.0 2>/dev/null | grep -o Linux`
                                if [ $? -eq 0 ];then
                                        echo "$SYSNAME"
                                        exit 0;
                                else
                                        echo "Processed $ip"
                                        exit 0
                                fi
                        ) &
                done
done

This solution I found that works, but not in my case, because no matter what, it still will spawn the processes before the logic of limiting the processes. I think that maybe I've just been looking at the code too long and it's just a simple logic issue and I'm placing things in the wrong area, or in the wrong order.


Answer Accepted:

I've accepted the answer from huitseeker. He was able to provide me with direction of how the logic works, allowing me to get it to work. Final script:

#!/bin/bash

FILE=/root/ats_net_final

for network in `cat $FILE`;do
        #for ip in $network.{1..255};do
        for ip in {1..255};do
            (
                ip=$network.$ip
                SYSNAME=`snmpwalk -v2c -c public -t1 -r1 $ip sysName.0 2>/dev/null | awk '{ print $NF }'`
                SYSTYPE=`snmpwalk -v2c -c public -t1 -r1 $ip sysDescr.0 2>/dev/null | grep -o Linux`
                if [ $? -eq 0 ];then
                    echo "$SYSNAME"
                    exit 0;
                else
                    echo "Processed $ip"
                    exit 0
                fi
            ) &
            if (( $ip % 10 == 0 )); then wait; fi
        done
        wait
done

Solution

  • An easy way to limit the number of concurrent subshells to 10 is:

    for ip in $(seq 1 255);do
      (<whatever you did with $ip in the subshell here, do with $network.$ip instead >) &
      if (( $ip % 10 == 0 )); then wait; fi
    done
    wait
    

    With the last wait being useful not to let the subshells of the last round of the inner loop overlap with those created in first round of the next outer run.