Search code examples
batch-filefor-loopbatch-processingfindstrlogfile

Batch script to grab lines with findstr without filepath

I've got a log file that monitors a large system including requests and acknowledgements. My objective is to be able to:

1. Loop through the script and get the lines where requests & their acknowledgements happen

2. Pull the entire lines of importance as strings and store them as variables for string modifying to output somewhere else.

Here's what I have so far:

@ECHO off
setlocal
setlocal enabledelayedexpansion
setlocal enableextensions

:: Lets get today's date, formatted the way the ABCD File is named
for /f "tokens=1-5 delims=/ " %%d in ("%date%") do set targetDate=%%f-%%d-%%e
:: Now we set the targetFile name
SET ABCDLogsFile=C:\Users\me\Documents\monitoring_file_for_jim\ABCDFIX*%targetDate%.log
::****Scrapped original approach*****
set "ackFoundCount=0"
set "reqFoundCount=0"
::Get lines with acks
for /f delims^=^ eol^= %%a in ('findstr /c:"\<ACK\>" "%ABCDLogsFile%"') do (
    set /a "ackFoundCount+=1"
    setlocal enabledelayedexpansion
    for %%N in (!ackFoundCount!) do (
        endlocal
        set "ackFound%%N=%%a"
    )
)
::Get lines with requests
for /f delims^=^ eol^= %%b in ('findstr /c:"ReqSingle" "%ABCDLogsFile%"') do (
    set /a "reqFoundCount+=1"
    setlocal enabledelayedexpansion
    for %%N in (!reqFoundCount!) do (
        endlocal
        set "reqFound%%N=%%b"
    )
)

setlocal enabledelayedexpansion
for /l %%N in (1,1,2 %reqFoundCount%) do echo REQ %%N FOUND= !reqFound%%N!
pause
for /l %%N in (1,1,2 %ackFoundCount%) do echo ACK %%N FOUND= !ackfound%%N!
endlocal

EDIT 2 dbenham

The roundabout way I was trying to accomplish this before was totally unnecessary. Thanks to the questions and answer here:

'findstr' with multiple search results (BATCH)

I've got my script working similarly. However, I'm curious if its possible to get findstr output without the filepath at the beginning. I only need to substring out the timestamp in the log, which would always be the first 12 characters of each line (without the filepath). My output currently is prefixed with the path, and while I could get the path where the log would eventually be in production, it would be safer to try and do it another way. At the time that this script would eventually be run, there would only be 1 or 2 reqs and acks each, that is why I store all which are found. It's not necessary but I think it would be reassuring to see two if there are two. Here is what the output looks like for acks and reqs alike:

C:\Users\me\Documents\monitoring_file_for_jim\ABCDFIX 2015-04-01.log:2015-03-26 07:00:11,028 INFO  etc...

I'm thinking that if I could strip the filepath off the start, then all I'd need to do to get just the timestamps of the events would be 

for /l %%N in (1,1,1 %reqFoundCount%) do echo Req %%N occurred at: !reqFound%%N:~0,12! >> MorningAckChecks.txt
for /l %%N in (1,1,1 %ackFoundCount%) do echo ACK %%N occurred at: !ackfound%%N:~0,12! >> MorningAckChecks.txt
Solution

  • I suspect you could not get SKIP to work because you you were iterating the delimited list of line numbers with a FOR statement, which means the number is in a FOR variable. Problem is, you cannot include FOR variables or (delayed expansion) when specifying a SKIP value, or any other FOR option. The batch parser evaluates the FOR options before FOR variables are expanded, so it couldn't possibly work. Only normal expansion can be used when including a variable as part of FOR options.

    But I don't understand why you think you need the line numbers at all. FINDSTR is already able to parse out the lines you want. Simply use FOR /F to iterate each matching line. For each line, define a variable containing the line content, and then use substring operations to parse out your desired values.

    But I can offer an alternative that I think could make your life much easier. JREPL.BAT is a sophisticated regular expression text processor that could identify the lines and parse out and transform your desired values, all in one pass. JREPL.BAT is a hybrid JScript/batch script that runs natively on any Windows machine from XP onward.

    If I knew what your input looked like, and what your desired output is, then I could probably knock up a simple solution using JREPL.BAT. Or you could read the extensive built in documentation and figure it out for yourself.

    Documentation is accessed from the command line via jrepl /?. You might want to pipe the output through MORE so you get one screen of help at a time. But I never do because my command line console is configured with a large output buffer, so I can simply scroll up to see past output.

    EDIT - In response to comment and updated question

    Here are the relevant snippets of your code that are causing the problem.

    SET ABCDLogsFile=C:\Users\me\Documents\monitoring_file_for_jim\ABCDFIX*%targetDate%.log
findstr /c:"\<ACK\>" "%ABCDLogsFile%"
findstr /c:"ReqSingle" "%ABCDLogsFile%

    The issue is your ABCDLogsFile definition includes a wildcard, which causes FINDSTR to prefix each matching line with the full path to the file name where the match occurred.

    I have a simple solution for you - Just change the definition of ABCDLogsFile as follows:

    SET "ABCDLogsFile=C:\Users\me\Documents\monitoring_file_for_jim\ABCDFIX<%targetDate%.log"

    Explanation

    My solution relies on two undocumented features

    1) Undocumented file mask wildcards.

    • < - Very similar to *
    • > - Very similar to ?

    These symbols are normally used for redirection, so they must be either quoted or escaped if you want to use them as file mask wildcards.

    We discuss the undocumented feature at DosTips - Dir undocumented wildcards. Sprinkled throughout the thread (and a link) are some example use cases.

    I document my understanding of exactly how the non-standard wildcards work at http://www.dostips.com/forum/viewtopic.php?p=39420#p39420

    2) FINDSTR works with the non-standard wildcards

    FINDSTR will prefix each matching line with the file name (and possibly path) if any of the following conditions occur

    • The /M option is used
    • The /F option is used
    • Multiple input files are explicitly listed on the command line
    • Multiple input files are implied via a file mask with at least one * or ? wildcard on the command line

    Your are getting the file path prefix because of the last trigger - the * in your file mask.

    But you can use < instead to get the same result, except the non-standard wildcards do not trigger the file prefix in the output.

    Problem solved :-)

    I talk about this FINDSTR feature at http://www.dostips.com/forum/viewtopic.php?p=39464#p39464.

    Some day I hope to update my What are the undocumented features and limitations of the Windows FINDSTR command? post with this tasty little tidbit.