How to create a script that will access all folders, subfolders and unzip the files to read the log

Question

I am trying to create a script that it will access a directory and check all the folders, but inside those folders are zip files that need to be unzipped, after that I will read the log file searching for a key word.

$WST = "S:\Docker\Logs\JP43001WST01"
$Dir_List = "S:\File.txt"
Get-ChildItem -Directory S:\Docker\Logs\JP43001WST01 | ForEach-Object { $_.Name } > $Dir_List
$Folders = Get-Content $Dir_List

foreach($folder in $Folders)
{
    $files = Get-ChildItem $WST\$folder -Recurse -Filter *.zip | ForEach-Object { $_.Name }
    #$files
    foreach ($file in $files)
    {
        Expand-Archive $WST\$folder\$file -DestinationPath "S:\Unzip"
        $logs = Get-ChildItem "S:\Unzip" | ForEach-Object { $_.Name }

        foreach ($log in $logs)
        {
            Get-Content -Path "S:\Unzip\$log" | findstr "Error processing BC 'onAfterSWDayOpen'"
        }

    }

}

I am stuck on the foreach loop. The folder structure would be something like:

WST01→20230621→WST0120230621_DEBUG-001.zip→WST0120230621_DEBUG-001.log

But inside WST01 are many subfolders with different names, and inside the subfolder are many different zip files. And each zip file contains one log file.

I could fix the issue using foreach the issue that I am facing now is that the log files are too big, and when I use the findstr is not working.

Answer 1

I think it would be easier if you were to use a helper function to unzip the log file from each zip.

Start your code with this:

function Extract-LogFile {
    [CmdletBinding()]
    param (
        [Parameter(Mandatory = $true, ValueFromPipeline = $true, ValueFromPipelineByPropertyName = $true, Position = 0)]
        [string]$SourceArchive,

        [Parameter(Mandatory=$true, Position = 1)]
        [ValidateNotNullOrEmpty()]
        [string]$DestinationPath,

        [string]$Extension = '.log'
    )

    Add-Type -Assembly System.IO.Compression.FileSystem

    # create the destination folder if this does not yet exist
    $null = New-Item $DestinationPath -ItemType Directory -Force

    # make sure we are using absolute paths
    $zipSource = Convert-Path -LiteralPath $SourceArchive
    $unzipDest = Convert-Path -LiteralPath $DestinationPath


    try {
        $archive  = [IO.Compression.ZipFile]::OpenRead($zipSource)
        $logFiles = $archive.Entries | Where-Object {$_.Name -like "*$Extension"}
        if (@($logFiles).Count) {
            $logFiles | ForEach-Object {
                # create the full path and filename for the unzipped file
                $fileOut = Join-Path -Path $unzipDest -ChildPath $_.Name
                # $true to overwrite an existing file that has the same name as the destination file
                [IO.Compression.ZipFileExtensions]::ExtractToFile($_, $fileOut, $true)

                # output an object with both the path to the zip archive and the path of the unzipped log file
                [PsCustomObject]@{Archive = $zipSource; LogFile = $fileOut}
            }
        }
        else {
            Write-Warning "No '$Extension' file found in $zipSource.."
        }
    }
    finally {
        # clean-up
        if ($archive) {$archive.Dispose()}
    }
}

Then add your main code below:

$WST    = "S:\Docker\Logs\JP43001WST01"
$result = Get-ChildItem -Path $WST -Filter '*.zip' -File -Recurse | ForEach-Object {
    $logs = @(Extract-LogFile -SourceArchive $_.FullName -DestinationPath 'S:\Unzip' -Extension '.log')
    foreach ($item in $logs) {
        # if as you say the log files are huge, use switch to read line-by-line
        $found = $false
        switch -Wildcard -File $item.LogFile {
            "*Error processing BC 'onAfterSWDayOpen'*" { $found = $true; break }
        }
        if ($found) {
            # output the object where this string was found
            $item
        }
    }
}

# output on screen
$result | Format-Table -AutoSize

# send output to CSV file
$result | Export-Csv -Path 'X:\Somewhere\errorlogs.csv' -NoTypeInformation