Search code examples
linuxbashsedparallel-processingcpu

Is there a way to process this code fast and what is the error of this code?


I have a code to character replacing and I can't figure out what is the error of this code? Is there a way to process this code faster with something like dedicating several cpu cores for this script?

#!/bin/bash

startdirectory="/data/test/cpy"
searchterm="""
replaceterm=""


i=0; 

for file in $(grep -l -R $searchterm $startdirectory)
    do
      cp $file $file.bak
      sed -e s/$searchterm/$replaceterm/ig $file > tempfile.tmp
      mv tempfile.tmp $file
      let i++;

    done

Solution

  • Since this executes sequentially it took a lot of time. I tried to run parallelly as chunks and it worked. Thanks.

    Unless the processing you are doing is crazily intense, and from what you post it isn't, then dedicating several CPU cores to the task (not an easy thing to do) will just overload the I/O channels from disk to RAM - unless you have parallel to the metal file access. This answer was really helpful