Search code examples
spring-cloud-dataflow

Setting node selector for spring cloud dataflow task and stream deployments on kubernetes

We want to fix all our spring cloud dataflow task and stream deployments to a particular set of nodes.

I have this working manually for a sample task eg 

task launch test-timestamp --properties "deployer.*.kubernetes.deployment.nodeSelector=env:development"

(this wasn't obvious as the documentation here https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/#configuration-kubernetes-deployer seems to imply the key is just nodeSelector not deployment.nodeSelector)

This correctly adds the node selector into the pod yaml for kubernetes.

But I want this to be set automatically ie using task.platform.kubernetes.accounts.default properties in the SCDF server config.

Ive tried:

task.platform.kubernetes.accounts.default.deployment.nodeSelector: env:development

and

task.platform.kubernetes.accounts.default.nodeSelector: env:development

but neither seem to work. What is the correct way to configure this?

Same question for stream deployments via skipper.

Also how do I set this up for scheduled tasks?

Solution

  • Sorry that you've had to try a few options to get to the bottom of finding the right deployer property that actually works.

    In general, from SCDF's Shell/UI, the deployer token is a short-form for spring.cloud.deployer.kubernetes property. It's a repetitive thing to supply when you have more deployer properties to configure in a stream/task, so we have a short-form for that reason.

    However, the nodeSelector is not a deployer-level property with a default, though. It is only available as a deployment level property, so that means, it is only available as an option for a per deployment basis.

    To put it differently, it is not available as an option for "global" configuration, so that's why task.platform.kubernetes.accounts.default.deployment.nodeSelector: env:development is not taking into account. Same also is true for Streams through Skipper, as well.

    It can be improved, though. I created spring-cloud/spring-cloud-deployer-kubernetes#300 for tracking - feel free to subscribe to the notifications. Both Streams and Tasks should then be able to take advantage of it as a global configuration. Once the PR is merged, you should be able to try it with SCDF's 2.2.0.BUILD-SNAPSHOT image.

    As for the K8s-scheduler implementation, we do not have support for nodeSelectors yet. I created spring-cloud/spring-cloud-scheduler-kubernetes#25 - we could collaborate on a PR if you want to port the functionality from K8s-deployer.