Getting only a repeating files from directory and subdirectories

Question

I'm trying to do script for finding non-unique files.

The script should take one .csv file with data: name of files, LastWriteTime and Length. Then I try to make another .csv based on that one, which will contain only those objects whose combination of Name+Length+LastWriteTime is NON-unique.

I tried following script which uses $csvfile containing files list:

$csvdata = Import-Csv -Path $csvfile -Delimiter '|'
$csvdata |
    Group-Object -Property Name, LastWriteTime, Length |
    Where-Object -FilterScript { $_.Count -gt 1 } |
    Select-Object -ExpandProperty Group -Unique |
    Export-Csv $csvfile2 -Delimiter '|' -NoTypeInformation -Encoding Unicode

$csvfile was created by:

{
    Get-ChildItem -Path $mainFolderPath -Recurse  -File |
        Sort-Object $sortMode |
        Select-Object Name, LastWriteTime, Length, Directory |
        Export-Csv $csvfile -Delimiter '|' -NoTypeInformation -Encoding Unicode
}

(Get-Content $csvfile) |
    ForEach-Object { $_ -replace '"' } |
    Out-File $csvfile -Encoding Unicode

But somehow in another $csvfile2 there is only the one (first) non-unique record. Does anyone have an idea how to improve it so it can list all non-unique records?

Answer 1

You need to use -Property * -Unique to get a list of unique objects. However, you cannot use -Property and -ExpandProperty at the same time here, because you want the latter parameter to apply to the input objects ($_) and the former parameter to apply to an already expanded property of those input objects ($_.Group).

Expand the property Group first, then select the unique objects:

... |
Select-Object -ExpandProperty Group |
Select-Object -Property * -Unique |
...