check A LOT of folders with Powershell

Question

my problem is with Powershell. I have a very big Folder. Insider are about 1 600 000 Subfolders. My task is to erase all empty folders or files beneath them which are older than 6 months. I wrote a loop with foreach but it takes ages before powershell starts with it ->

...

foreach ($item in Get-ChildItem -Path $rootPath -recurse -force | Where-Object -FilterScript { $_.LastWriteTime -lt $date })
{
# here comes a script which will erase the file when its older than 6 months
# here comes a script which will erase the folder if it's a folder AND does not have child items of its own

...

The Problem: my internal memory gets full(4GB) and i cant properly work anymore. My guess: powershell loads all 1 600 000 folders, and only after that it begins to filter them.

Is there a possibility to prevent this?

Answer 1

You are correct, all 1.6M folders, or at least references to them, are being loaded at once. Best practice is to filter left & format right; IOW, remove those folders before you hit Where-Object if at all possible (unfortunately, gci doesn't support a date filter AFAICT). Also, if you keep things in the pipeline, you'll use less memory.

The following will restrict $items to only those folders which match your criteria, then perform your loop over those objects.

$items = Get-ChildItem -path $rootpath -recurse -force | ?{ $_.LastWriteTime -lt $date }
foreach ($item in $items) {
# here comes a script which will erase the file when its older than 6 months
# here comes a script which will erase the folder if it's a folder AND does not have child items of its own
}

Or streamlining further:

function runScripts {
    # here comes a script which will erase the file when its older than 6 months. Pass $input into that script. $input will be a folder.
    # here comes a script which will erase the folder if it's a folder AND does not have child items of its own Pass $input into that script. $input will be a folder.
}
Get-ChildItem -path $rootpath -recurse -force | ?{ $_.LastWriteTime -lt $date }|runScripts

In this last case, you're using runScripts as a function which uses the pipelined object as a parameter which can be operated on ($input), so you can send everything through the pipeline instead of using those intermediate objects (which will consume more memory).