I am looking to sort a list of RGB colors by their least distance, and hopefully this will give a better sorting than Luminosity and HSV. I am using the distance formula to calculate the color's distance, but How can I sort the list based on the least distance?
Forget that you've got colours for a second, and simply consider points in 3D (or 2D) space.
You can't simply "sort" points by distance. You can sort them by distance to some specified point X, but that is probably not what you're after.
Instead what you're probably looking for is an ordering of the points that minimizes the total length of the path through them. This is essentially the traveling salesman problem, and its a lot more computationally intensive than just sorting the list.
Ideally you want to perform this distance calculation using a perceptually correct distance function - which usually means converting your points to Lab space and them using one of the standards.