My Stream Definitions are as Follows:-
IngestToBroker = custom-producer --spring.cloud.stream.bindings.output.content-type=application/json --spring.cloud.stream.bindings.output.producer.headerMode=raw > :Kafkatopic1
DataProcessor = :Kafkatopic1> custom-processor--spring.cloud.stream.bindings.output.content-type=application/json --spring.cloud.stream.bindings.input.consumer.headerMode=raw --spring.cloud.stream.bindings.output.producer.headerMode=raw > :kafkatopic2
myCountsOne = :DataProcessor.custom-processor > field-value-counter --field-name=messageStatusOne
myCountsTwo = :DataProcessor.custom-processor > field-value-counter --field-name=messageStatusOne
MyFileSink = :kafkatopic2> file --directory=C:\usr\sdflogs --name=My-File-Sink2
Current Scenario
1) IngestToBroker and DataProcessor works good. I get the Required JSON output as follows, which is sent to KafkaTopic1 and then through processor and to kafkaTopic2 and finally to file.
{"messageStatusOne":"RECEIVED","DateTimeOne":"2017.09.26.22.03.34","messageStatusTwo":"PROCESSED","DateTimeTwo":"2017.09.26.22.03.34"} {"messageStatusOne":"ERROR","DateTimeOne":"2017.09.26.21.06.45","messageStatusTwo":"NOT AVAILABLE","DateTimeTwo":"2017.09.26.21.06.45"} {"messageStatusOne":"RECEIVED","DateTimeOne":"2017.09.26.21.06.52","messageStatusTwo":"PROCESSED","DateTimeTwo":"2017.09.26.21.06.52"} {"messageStatusOne":"REVIEW","DateTimeOne":"2017.09.26.21.06.59","messageStatusTwo":"HOLD","DateTimeTwo":"2017.09.26.21.06.59"}
Custom-Producer and Custom-Processor are standalone apps that I have deployed which basically changes the Data and the former pushed new JSON record every few seconds.
2) Same is the case with all the Stream Definitions . They all work good, I get the required JSON. EXCEPT for the field-value-counter definitions myCountsOne and myCountsTwo
Issue 1) Dashboard PIC for Analytics on my Local SCDF server
it seems to NOT be active. There are no drop downs for Metrics and Visualizations. I really need this to work and I am pretty new to SCDF. What am I missing ? how do I get to this to work ?
2) Also on a sideNote I am trying to configure MarkLogic with SCDF but will SCDF accept with ODBC drivers ? Please forgive me if I sound stupid in asking all this. Thank you.
Update I just noticed this error in my Spring Data Flow Startup console
"cannot get jedis connection nested exception is redis.clients.jedis.exceptions.jedisConnectionException"
Do we need to add any redis dependencies to my Custom-Processor which is deployed as a standalone application ? this is my Spring Boot Version -1.5.6.RELEASE. Thank you.
To further simplify the problem in hand, let's try the following stream in your environment. Once you have success with it, you can then review the other streams on what could go wrong.
Stream:
dataflow:>stream create foo --definition "http --port=9000 | field-value-counter --fieldName=messageStatusOne --name=bar" --deploy
Data:
dataflow:>http post --target http://localhost:9000 --data {"messageStatusOne":"RECEIVED","DateTimeOne":"2017.09.26.22.03.34","messageStatusTwo":"PROCESSED","DateTimeTwo":"2017.09.26.22.03.34"} POST (text/plain) http://localhost:9000 {"messageStatusOne":"RECEIVED","DateTimeOne":"2017.09.26.22.03.34","messageStatusTwo":"PROCESSED","DateTimeTwo":"2017.09.26.22.03.34"} 202 ACCEPTED
List:
dataflow:>field-value-counter display --name bar
Displaying values for field value counter 'bar'
╔════════╤═════╗
║ Value │Count║
╠════════╪═════╣
║RECEIVED│ 1║
╚════════╧═════╝
