I have a sample SCDF dataflow stream application which am able to deploy and get it running in kubernetes.But i rely on the dataflow dashboard to create application which points to the docker images and then define stream and add properties and deploy it.Now i am trying to automate the manual steps of creating the apps in the dataflow server console , create stream definition and deploy it .Is there any sample project or reference for the same.
There are a few options here.
1) We rely on SCDF's Java DSL to automate and acceptance-test the application registration, creation and deployment of streams and tasks on all the supported platforms. You will take spring-cloud-dataflow-rest-client as a dependency in your custom Java/Boot applications; with that, you could gain access to DataFlowOperations, which will allow you to perform all the actions that you can via Shell or the UI. Have a look at TickTockTests.java to get an idea. In fact, we use this pattern internally for our integration-tests, too (see: DockerComposeIT.java).
2) REST APIs: You could use the stream/task's create, deploy/launch, or stream update APIs as part of your CI tooling and the workflows. You have the flexibility to either invoke them via scripts or by other means. This concept was presented at a webinar — it might give you some ideas.