I have an existing process that I am trying to convert into SCDF implementation. The current process is,
HTTP Receiver (receive HTTP POST data) -> RabbitMQ -> MQ Receiver Service -> Process/Transform -> DB Sink
How can I achieve such flow, if possible? (1st issue)
So far, I figured that SCDF supports producer -> process -> consumer flow only.
I also tried to create multiple flows (HTTP Source -> RabbitMQ Sink & RabbitMQ Source -> Process -> DB) but deployment of RabbitMQ (as source and sink) does not work on Cloud Foundry (2nd issue). Aperently, I can deploy a stream with either Rabbit source or sink but not both togather (even as different stream).
Appreciate your help.
What you have described is the fundamental streaming pipeline using named destinations and it is expected to work including Cloud Foundry environment.
You can check out this sample for more information.
If you still see the issue, please share the exact error/exception while you try deploy the streams.