How to cut a range of lines defined by variables

Question

I have this python crawler output

[+] Site to crawl: http://www.example.com
[+] Start time: 2020-05-24 07:21:27.169033
[+] Output file: www.example.com.crawler

[+] Crawling
   [-] http://www.example.com
   [-] http://www.example.com/
   [-] http://www.example.com/icons/ubuntu-logo.png
   [-] http://www.example.com/manual
    [i] 404 Not Found
[+] Total urls crawled: 4

[+] Directories found:
   [-] http://www.example.com/icons/
[+] Total directories: 1

[+] Directory with indexing

I want to cut the lines between "Crawling" & "Total urls crawled" using awk or any other tool, so basically i wanna use variables to assign the NR to the first keyword "Crawling", and a second variable assigned to it the NR value of the second limiter "Total urls crawled", and then cut the range between the two, i tried something like this:

awk 'NR>$(Crawling) && NR<$(urls)' file.txt

but nothing really worked, the best i got is a cut from the Crawling+1 line to the end of the file which isn't helpfull really, so how to do it and how to cut a range of lines with awk with variables!

awk

Answer 1

If I got your requirement correctly you want to put shell variables to awk code and search strings then try following.

awk -v crawl="Crawling" -v url="Total urls crawled" '
$0 ~ url{
  found=""
  next
}
$0 ~ crawl{
  found=1
  next
}
found
'  Input_file

Explanation: Adding detailed explanation for above.

awk -v crawl="Crawling" -v url="Total urls crawled" '   ##Starting awk program and setting crawl and url values of variables here.
$0 ~ url{                      ##Checking if line is matched to url variable then do following.
  found=""                     ##Nullify the variable found here.
  next                         ##next will skip further statements from here.
}
$0 ~ crawl{                    ##Checking if line is matched to crawl variable then do following.
  found=1                      ##Setting found value to 1 here.
  next                         ##next will skip further statements from here.
}
found                          ##Checking condition if found is SET(NOT NULL) then print current line.
'  Input_file                  ##Mentioning Input_file name here.