Powershell recursively move 20,000 files at a time

I'm a novice with Powershell. I'm trying to move data to be processed elsewhere that has limitations on the number of files at a time it can handle. My source folder has hundreds of subfolders that each in turn may also have up to 100 subfolders, totaling up to millions of small files. File names are assumed to be unique 25-digit output from an export so checking existing file names at each move will not be necessary if excluding the check speeds up the overall process.

I'd like to automate moving (not copy) these with Powershell from source path to an incremental named destination folder of 20,000 files each. The destination output folder names are not important as long as they each only have no more than 20,000 files in each destination and don't overlap existing folder names.

Example:

Source : C:\MyFiles has 100 subfolders that each also have 100 subfolders.

C:\Myfiles\Folder000\Folder000A\file1.txt C:\Myfiles\Folder000\Folder000A\file2.txt C:\Myfiles\Folder000\Folder000A\file3.txt C:\Myfiles\Folder000\Folder000B\file1.txt C:\Myfiles\Folder000\Folder000B\file1.txt C:\Myfiles\Folder001\Folder000A\file1.txt ...etc

Destination D:\My20Kfiles

D:\My20kFiles\000001 [20,000 messages] D:\My20kFiles\000002 [next 20,000 messages] D:\My20kFiles\000003 [next 20,000 messages] ...etc

Solution

try this

 $yourdir="C:\MyFiles"
 $yourdestdir="D:\My20Kfiles"
 $loop=20000
 $i=$loop;
 gci $yourdir -Recurse -File -Filter *.txt  | %{$newdir="$yourdestdir\{0:d6}\" -f [math]::DivRem($i++, $loop, [ref]$null)   ;new-item -Path $newdir -Force -ItemType Directory; Move-Item -Path $_.FullName -Dest $newdir -Force }