In my cloud stream application I've used the spring-cloud-starter-stream-processor-tasklaunchrequest-transform starter app as task-processor.
The starter is locally built and installed in local Maven repository.
Commands used to start my local Dataflow Server and DF Shell, and then register my apps and create-cum-deploy the stream are as below:-
mvn clean installmvn clean installInstall the task app in local maven - mvn clean install
Build Dataflow Server Local - mvn clean package (with changed remoteRepo location to our enterprise maven repo)
Start DF Server Local - mvn spring-boot:run
Build Dataflow Shell - mvn clean package
Start DF Shell - mvn spring-boot:run
Import out of box starter apps with Rabbit binding stream-applications-rabbit-maven
Register apps -
app register --name task-processor --type processor --uri maven://org.springframework.cloud.stream.app:spring-cloud-starter-stream-processor-tasklaunchrequest-transform:jar:1.1.0.BUILD-SNAPSHOTapp register --name task-launcher-local --type sink --uri maven://org.springframework.cloud.stream.app:task-launcher-local-sink-rabbit:jar:1.0.4.RELEASECreate Stream -
stream create foo --definition "rabbit --rabbit.queues=cloud-stream-source --rabbit.requeue=true --spring.rabbitmq.host=localhost --spring.rabbitmq.port=5672 --spring.cloud.stream.bindings.output.contentType='text/plain' | task-processor --uri=maven://com.example:thumbnail-generator:0.0.1-SNAPSHOT | task-launcher-local" --deploy
However the deploy of the stream is failing due to startup failure of the task-processor. The error logged in the processor logs is:
no main manifest attribute, in /Users/abc/.m2/repository/org/springframework/cloud/stream/app/spring-cloud-starter-stream-processor-tasklaunchrequest-transform/1.1.0.BUILD-SNAPSHOT/spring-cloud-starter-stream-processor-tasklaunchrequest-transform-1.1.0.BUILD-SNAPSHOT.jar
Now there are 2 points to raise:
spring-cloud-starter-stream-processor-tasklaunchrequest-transform. And that confuses me if any customisation is required to make it runnable and usable for my stream app.I've tried changing the starter app to a runnable spring boot app, and installed that version in my local repo but unless the application looks at correct local repo (and not at the default one), I can't use it.
I believe it's an issue with the app and it lacks the visibility of the local maven settings, hence going by default values.
Any inputs will be of help.
Register apps - app register --name task-processor --type processor --uri maven://io.spring.cloud:taskprocessor:jar:1.1.0.BUILD-SNAPSHOT
According to this, you're registering the sample taskprocessor application. Though there's nothing wrong in trying it out, we would recommend using the standardized application and if there any missing features in it, please file a GH issue for it.
However the deploy of the stream is failing due to startup failure of the task-processor.
If you're in fact trying to use the standardized application, you're application registration command should be something like:
app register --name task-processor --type processor --uri maven://org.springframework.cloud.stream.app:tasklaunchrequest-transform-processor-rabbit:1.1.0.BUILD-SNAPSHOT --force
The maven repo location mentioned in the logs is default location for MacOS and not the one setup in my local maven configurations (/Users/abc/myrepo/.m2/repository)
SCDF uses Aether library to resolve and download maven artifacts. By default, the requested application is checked in local maven repository first and if it isn't present, it is then downloaded from the remote repository. This is no different than any other maven workflow.
The error is valid for the built jar file of starter app - spring-cloud-starter-stream-processor-tasklaunchrequest-transform. And that confuses me if any customisation is required to make it runnable and usable for my stream app
The OOTB applications are utility applications. To the most part, they should satisfy common requirements. If there's a need to adjust the generic behavior, you could then extend them.
With that said, perhaps you could try running the locally built application(s) via java -jar to confirm whether the application bootstraps successfully and then register it to SCDF. You could also enable DEBUG level logs at each application for troubleshooting.