Search code examples
curlspring-cloud-dataflow

Spring Cloud Data Flow does not work how is expected for file sink and curl client

In Mac, working with Spring XD

  • 1.3.2.RELEASE

The following works fine:

stream create --name orderFilter --definition "http --port=9090 | filter --expression=#jsonPath(payload,'$.countryCode').equals('US') | file --name=manolito.txt --dir=/Users/manueljordan/Desktop --mode=APPEND" --deploy

Through its own shell the following works:

http post --target http://localhost:9090 --data "{\"id\":\"2773\",\"countryCode\":\"US\",\"orderAmt\":\"100\"}"
http post --target http://localhost:9090 --data "{\"id\":\"2774\",\"countryCode\":\"CA\",\"orderAmt\":\"125\"}"
http post --target http://localhost:9090 --data "{\"id\":\"2775\",\"countryCode\":\"US\",\"orderAmt\":\"110\"}"

The data is filtered and the output is located in /Users/manueljordan/Desktop how is expected.

Until here all is ok.

Now migrating to Spring Cloud Data Flow working with

  • spring-cloud-dataflow-server-local-1.2.2.RELEASE.jar
  • spring-cloud-dataflow-shell-1.2.2.RELEASE.jar

Through the GUI (http://localhost:9393/dashboard/index.html#/streams/definitions) I can create the same stream.

Again through the shell (from spring-cloud-dataflow-shell) the following works:

  http post --target http://localhost:9090 --data "{\"id\":\"2773\",\"countryCode\":\"US\",\"orderAmt\":\"100\"}"
  http post --target http://localhost:9090 --data "{\"id\":\"2774\",\"countryCode\":\"CA\",\"orderAmt\":\"125\"}"
  http post --target http://localhost:9090 --data "{\"id\":\"2775\",\"countryCode\":\"US\",\"orderAmt\":\"110\"}"

The same http post used for Spring XD

Situation 01 even when the three http post from above work about the filter behaviour, the output file is not created in /Users/manueljordan/Desktop, it is created in /private/var/folders/nf/ql69b5zd7w54jw6nb5zmvck80000gn/T/dataflow/output instead

Observe the dataflow directory within ql69b5zd7w54jw6nb5zmvck80000gn

Why this behaviour? How fix this?, some special configuration missing?

Situation 02 about cURL, according with the reference documentation, in the 10.1. Endpoints section, just practically below the Figure 3. The Spring Cloud Data Flow Server shows that cURL can be used too.

Through iterm 2 if I use the following commands: (it according with How to POST JSON data with Curl from Terminal/Commandline to Test Spring REST?) 

curl -H "Content-Type: application/json" -X POST -d '"{\"id\":\"2773\",\"countryCode\":\"US\",\"orderAmt\":\"100\"}"' http://localhost:9090
curl -H "Content-Type: application/json" -X POST -d '"{\"id\":\"2774\",\"countryCode\":\"CA\",\"orderAmt\":\"125\"}"' http://localhost:9090
curl -H "Content-Type: application/json" -X POST -d '"{\"id\":\"2775\",\"countryCode\":\"US\",\"orderAmt\":\"110\"}"' http://localhost:9090

Nothing happens (I press enter to execute the command and just pass to the new line prompt, that's all). Why? what is missing?

Solution

  • See answer here: https://github.com/spring-cloud/spring-cloud-dataflow/issues/1587#issuecomment-313266803

    We monitor both SO and GitHub issues so no need to cross-post.

    -Thomas