Search code examples
linuxawkjarfind

Understanding a linux script with combination of find + grep + awk

I have a script with this:

find . -name '*.nar' | while read NARFILE; do 
  if jar tvf $NARFILE | grep -q jackson-databind; then
    OUTPUT=$(jar tvf $NARFILE | grep jackson-databind)
    echo $NARFILE':'$OUTPUT
  fi
done | 
grep '\-2.9.10.jar' | 
awk -F':' '{print $1}' | while read NARFILE; do
  jar uvf $NARFILE META-INF/bundled-dependencies/jackson-databind-2.9.10.8.jar
  jar xvf $NARFILE META-INF/DEPENDENCIES
  sed -i '/jackson-databind/s/2.9.10$/2.9.10.8/' META-INF/DEPENDENCIES
  jar uvf $NARFILE META-INF/DEPENDENCIES
  zip -d $NARFILE  META-INF/bundled-dependencies/jackson-databind-2.9.10.jar 
done

I am not able to comprehend what's happening here.

My vague understanding so far:

  1. find all files in the folder with .nar extension
  2. then we check whether contents of file has 'jackson-databind'
  3. if yes, we create some output(I am not sure what is the value for it)
  4. then we do grep to take entries with '2.9.10.jar'
  5. then we use awk command to update the file with jackson-databind-2.9.10.8.jar

And then I am lost.

Please help.

Solution

  • This comes without any guarantees and with the warning, that if you don't trust the source of a script, you probably shouldn't execute it, even if you think you fully understand what it does. Sometimes there are fine, seemingly innocuous things, that can be harmful in a specific context.

    That being said, from what you provide, it would seem (double exclamation mark) that it deals with NAR files as in Apache NiFi archives, that are used to provide the NiFi processor its own set of dependencies and libraries isolated from the rest of whatever is running. The two relevant files in such archives, that are being touched by the script are META-INF/DEPENDENCIES - see this answer - and Jackson Databind jar files in the META-INF/bundled-dependencies directory, that contains the jar files that will actually be loaded and provided to the NiFi processor.

    Note, that neither of the Jackson versions mentioned are current, and, 29.10.8 has at least three known vulnerabilities. The (micro) Milestone 2.9.10.8 patched a couple of vulnerabilities. Explore their github for current versions and more information.

    Steps:

    find . -name '*.nar' | while read NARFILE; do

    Find and go over all files with the .nar file ending.

    if jar tvf $NARFILE | grep -q jackson-databind; then

    Try to list contents of the current .nar archive file and check if there is mention of "jackson-databind" aka if it contains a file. grep -q means quiet and in combination with the if then the following will only be executed if something was found. In other words, if there is no match for jackson-databind, then it would essentially return nothing and it go to the next .nar file in the loop.

        OUTPUT=$(jar tvf $NARFILE | grep jackson-databind)
    echo $NARFILE':'$OUTPUT
  fi
done |

    Store the match in OUTPUT, then echo the current .nar file together with this line and pipe it futher. Essentially this means up to this point it goes over all .nar files, looks inside, if it finds jackson-databind then it "bundles" the .nar filename together with the line for jackson-databind and gives this to the next step in a kind of key-value pair if you want.

    grep '\-2.9.10.jar' | 
awk -F':' '{print $1}' | while read NARFILE; do

    First it quasi filters for 2.9.10.jar so that matched line containing jackson-databind of another version, nothing will be done. awk -F':' then takes apart what I called "key-value pair" of .nar file and line again and gives the .nar filename to the next step.

    jar uvf $NARFILE META-INF/bundled-dependencies/jackson-databind-2.9.10.8.jar

    This updates the .nar archive with the relative file path META-INF/bundled-dependencies/jackson-databind-2.9.10.8.jar. So this is a file that is assumed to be in your filesystem, relative to where you execute this script from.

    jar xvf $NARFILE META-INF/DEPENDENCIES
sed -i '/jackson-databind/s/2.9.10$/2.9.10.8/' META-INF/DEPENDENCIES
jar uvf $NARFILE META-INF/DEPENDENCIES

    Extracts, then edits, then updates the archive using those edits the META-INF/DEPENDENCIES file in the *.nar file. The edit: In the lines that contain 'jackson-databind' exchange the string '2.9.10' for '2.9.10.8', is what sed -i (in-place editing) does.

    zip -d $NARFILE  META-INF/bundled-dependencies/jackson-databind-2.9.10.jar

    Removes the old jackson-databind-2.9.10.jar from the archive.

    In short, again what it - seems - to do is the update the version of Jackson Databind inside the NAR archives from an old version to a newer old version.