Split a folder into multiple subfolders in terminal/bash script

Question

I have several folders, each with between 15,000 and 40,000 photos. I want each of these to be split into sub folders - each with 2,000 files in them.

What is a quick way to do this that will create each folder I need on the go and move all the files?

Currently I can only find how to move the first x items in a folder into a pre-existing directory. In order to use this on a folder with 20,000 items... I would need to create 10 folders manually, and run the command 10 times.

ls -1  |  sort -n | head -2000| xargs -i mv "{}" /folder/

I tried putting it in a for-loop, but am having trouble getting it to make folders properly with mkdir. Even after I get around that, I need the program to only create folders for every 20th file (start of a new group). It wants to make a new folder for each file.

So... how can I easily move a large number of files into folders of an arbitrary number of files in each one?

Any help would be very... well... helpful!

Answer 1

This solution can handle names with whitespace and wildcards and can be easily extended to support less straightforward tree structures. It will look for files in all direct subdirectories of the working directory and sort them into new subdirectories of those. New directories will be named 0, 1, etc.:

#!/bin/bash

maxfilesperdir=20

# loop through all top level directories:
while IFS= read -r -d $'\0' topleveldir
do
        # enter top level subdirectory:
        cd "$topleveldir"

        declare -i filecount=0 # number of moved files per dir
        declare -i dircount=0  # number of subdirs created per top level dir

        # loop through all files in that directory and below
        while IFS= read -r -d $'\0' filename
        do
                # whenever file counter is 0, make a new dir:
                if [ "$filecount" -eq 0 ]
                then
                        mkdir "$dircount"
                fi

                # move the file into the current dir:
                mv "$filename" "${dircount}/"
                filecount+=1

                # whenever our file counter reaches its maximum, reset it, and
                # increase dir counter:
                if [ "$filecount" -ge "$maxfilesperdir" ]
                then
                        dircount+=1
                        filecount=0
                fi
        done < <(find -type f -print0)

        # go back to top level:
        cd ..
done < <(find -mindepth 1 -maxdepth 1 -type d -print0)

The find -print0/read combination with process substitution has been stolen from another question.

It should be noted that simple globbing can handle all kinds of strange directory and file names as well. It is however not easily extensible for multiple levels of directories.