I am trying to join multiple log files into a single archive file, then move that archive file to another location, in an effort to both clean up old log files, and save hard drive space. We have a bunch of tools that all log to the same root, with a per-tool folder for their logs. (E.g.,
each of which will have log files inside, like
I want to go into each of these subfolders, archive up all the files older than 5 days, then move the archive to another (long-term storage) drive and delete the now-zipped files. The tools I'm using are PowerShell and 7zip. The below code is using test locations.
I have cobbled together two scripts from various sources online, over the course of two full shifts, but neither one works right. Here's the first:
# Alias for 7-zip
if (-not (test-path "$env:ProgramFiles\7-Zip\7z.exe")) {throw "$env:ProgramFiles\7-Zip\7z.exe needed"}
set-alias 7zip "$env:ProgramFiles\7-Zip\7z.exe"
$Days = 5 #minimum age of files to archive; in other words, newer than this many days ago are ignored
$SourcePath = C:\WorkingFolder\FolderSource\
$DestinationPath = C:\Temp\
$LogsToArchive = Get-ChildItem -Recurse -Path $SourcePath | Where-Object {$_.lastwritetime -le (get-date).addDays(-$Days)}
$archive = $DestinationPath + $now + ".7z"
#endregion
foreach ($log in $LogsToArchive) {
#define Args
$Args = a -mx9 $archive $log
$Command = 7zip
#write-verbose $command
#invoke the command
invoke-expression -command $Command $Args
The problem with this one is that I get errors trying to invoke the expression. I've tried restructuring it, but then I get errors because my $Args have an "a"
So I abandoned this method (despite it being my preferred), and tried this set.
#region Params
param(
[Parameter(Position=0, Mandatory=$true)]
[ValidateScript({Test-Path -Path $_ -PathType 'container'})]
[System.String]
$SourceDirectory,
[Parameter(Position=1, Mandatory=$true)]
[ValidateNotNullOrEmpty()]
[System.String]
$DestinationDirectory
)
#endregion
function Compress-File{
#region Params
param(
[Parameter(Position=0, Mandatory=$true)]
[ValidateScript({Test-Path -Path $_ -PathType 'leaf'})]
[System.String]
$InputFile,
[Parameter(Position=1, Mandatory=$true)]
[ValidateNotNullOrEmpty()]
[System.String]
$OutputFile
)
#endregion
try{
#Creating buffer with size 50MB
$bytesGZipFileBuffer = New-Object -TypeName byte[](52428800)
$streamGZipFileInput = New-Object -TypeName System.IO.FileStream($InputFile,[System.IO.FileMode]::Open,[System.IO.FileAccess]::Read)
$streamGZipFileOutput = New-Object -TypeName System.IO.FileStream($OutputFile,[System.IO.FileMode]::Create,[System.IO.FileAccess]::Write)
$streamGZipFileArchive = New-Object -TypeName System.IO.Compression.GZipStream($streamGZipFileOutput,[System.IO.Compression.CompressionMode]::Compress)
for($iBytes = $streamGZipFileInput.Read($bytesGZipFileBuffer, 0,$bytesGZipFileBuffer.Count);
$iBytes -gt 0;
$iBytes = $streamGZipFileInput.Read($bytesGZipFileBuffer, 0,$bytesGZipFileBuffer.Count)){
$streamGZipFileArchive.Write($bytesGZipFileBuffer,0,$iBytes)
}
$streamGZipFileArchive.Dispose()
$streamGZipFileInput.Close()
$streamGZipFileOutput.Close()
Get-Item $OutputFile
}
catch { throw $_ }
}
Get-ChildItem -Path $SourceDirectory -Recurse -Exclude "*.7z"|ForEach-Object{
if($($_.Attributes -band [System.IO.FileAttributes]::Directory) -ne [System.IO.FileAttributes]::Directory){
#Current file
$curFile = $_
#Check the file wasn't modified recently
if($curFile.LastWriteTime.Date -le (get-date).adddays(-5)){
$containedDir=$curFile.Directory.FullName.Replace($SourceDirectory,$DestinationDirectory)
#if target directory doesn't exist - create
if($(Test-Path -Path "$containedDir") -eq $false){
New-Item -Path "$containedDir" -ItemType directory
}
Write-Host $("Archiving " + $curFile.FullName)
Compress-File -InputFile $curFile.FullName -OutputFile $("$containedDir\" + $curFile.Name + ".7z")
Remove-Item -Path $curFile.FullName
}
}
}
This actually seems to work, insofar as it creates individual archives for each eligible log, but I need to "bundle" up the logs into one mega-archive, and I can't seem to figure out how to recurse (to get sub-level items) and do a foreach (to confirm age) without having that foreach produce individual archives.
I haven't even gotten into the Move and Delete phase, because I can't seem to get the archiving stage to work properly, but I certainly don't mind grinding away at that once this gets figured out (I've already spent two full days trying to figure this one!).
I greatly appreciate any and all suggestions! If I've not explained something, or been a bit unclear, please let me know!
EDIT1: Part of the requirement, which I completely forgot to mention, is that I need to keep the structure in the new location. So the new location will have
C:\ServerLogs\2ndApp --> C:\Archive\2ndApp
C:\Archive
EDIT2: For this latter part, I used Robocopy - It maintains the folder structure, and if you feed it ".zip" as an argument, it'll only do the .zip files.
this line $Args = a -mx9 $archive $log likely needs to have the right side value wrapped in double quotes OR each non-variable wrapped in quotes with a comma between each so that you get an array of args.
another method would be to declare an array of args explicitly. something like this ...
$ArgList = @(
'a'
'-mx9'
$archive
$log
)
i also recommend you NOT use an automatic $Var name. take a look at Get-Help about_Automatic_Variables and you will see that $Args is one of those. you are strongly recommended NOT to use any of them for anything other than reading. writing to them is iffy. [grin]