Search code examples
shellcode-coverage

code coverage tools for validating the scripts


Is there any way to validate the coverage of the shell scripts? I have project having lots of shell scripting, and need to ensure static analysis can be performed on the coverage for the shell scripts. Is there any tool available?


Solution

  • I seriously doubt that there could be any static code analysis performed on shell scripts - especially due to the fact that shell scripts are supposed to call external programs and based on what these external programs return - and there are myriads of external programs and external environment states. It's similar to the problem of static analysis of code heavily relying on eval-like mechanism, but shell scripting is all about eval-style programming.

    However, there are some general pointers that could prove useful for "proper" validation, code coverage and documenting of shell scripts as major languages do:

    • You can always run a script with -x (AKA xtrace) option - it will output trace looking like that to stderr:

      + log_end_msg 0
      + [ -z 0 ]
      + retval=0
      + log_end_msg_pre 0
      + :
      + log_use_fancy_output
      + TPUT=/usr/bin/tput
      + EXPR=/usr/bin/expr
      + [ -t 1 ]
      + [ xxterm != x ]
      + [ xxterm != xdumb ]
      
    • Bash makes it possible to redirect this stream to external FD (using BASH_XTRACEFD variable) - that's much easier to parse in practice.

    • It's not trivial, but it's possible to write a program that will find relevant pieces of code being executed using xtrace output and make you a fancy "code coverage" report - like what was called how many times, which pieces of code weren't run at all and thus lack test coverage.

    • In fact, there's a wonderful tool named shcov already written that uses this process - albeit it's somewhat simplistic and doesn't handle all possible cases very well (especially when we're talking about long and complex lines)

    • Last, but not least - there's minimalistic shelldoc project (akin to javadoc) that helps generating documentation based on comments in shell scripts. Yep, that's a shameless plug :)