Worked with CRDs a bit so far. Recently a colleague of mine claimed that custom resources only ever manage pods in the end. Is this true? Can you do CRDs for services or config maps as well?
A custom resource definition doesn't manage anything. It is just a definition for some specific API group, version, and kind (as you see at the top of a Kubernetes manifest) that is not part of the standard Kubernetes API. If you've installed a CRD then you can kubectl get customtypename and run similar commands, but nothing actually happens as a result of creating one of these objects.
The other part of the typical operator pattern is a Kubernetes controller. This is a program, that usually runs inside the cluster, that uses the Kubernetes API to take some action.
A controller can read and write any Kubernetes resource; it is not limited to Pods. You could, for example, create a ConfigMap in response to a custom resource changing, or reference a ConfigMap in your custom resource and combine those two data to invoke some external API.