Clear Color Swatches on Xcode Color Window?

Question

You know how under Storyboard / Interface Builder, you can assign a custom color and save it to the color palette below? How would you clear those and start over with all of them blank? I'm not sure how to delete them rather than overriding them with another color.

Answer 1

As of Xcode 9 / macOS High Sierra, the accepted answer doesn't work any more. The colors in the palette are now selectable/draggable - once a blue border appears around the selected color, press delete key to remove it.

If you don't see a blue border, when clicking on a color, make sure that the "Colors" window is active.