Search code examples
spring-batchspring-cloud-taskpcf

Running a spring batch with partitions in cloud foundry

I have created an app with spring batch(with partition) application taking example of this https://github.com/mminella/S3JDBC. My app is reading some files from object store and doing some processing and writing back to object the store. My app with local partition works fine in my machine.

I changed the maven, to run in cloud foundry , did change for deployer partition handler and step execution listener and deploying on pcf.

But while trying to push and run the app on pcf , I am getting an issue : Failing URI /v2/info. I tried to log the error found that there is one call to my app e.g https://mypcf.com:443/v2/info and after that it gives the error. I cant provide full logs because of some restrictions. So I want to know :

  1. To deploy a spring batch in pcf(is there any extra configuration needed except the maven dependency and code changes for deployerpartitionhandler and stepexecutionlistener and @cloudtask): org.springframework.cloud spring-cloud-deployer-cloudfoundry 1.1.0.M1
  2. Is it mandatory to have a separate data base service like my-sql for the partition job. Cant I use H2(the default one, if I don't configure anything)?
  3. Do I need to do any configuration in pcf to support running multiple partitions ?
  4. As I am running remote partitioning , can I run that app on local STS or Intellij(not on PCF-DEV)so that it will run my app in pcf(remote) and launch the workers.(Sorry for the stupid question , I am new to PCF).
Solution

  • Thanks for checking out my example. To answer your questions:

    1. You should be able to use the latest deployer release (instead of that rather old version).
    2. Yes. Partitioned steps need to all be able to share the same job repository data store so an in memory database like H2 will not work for that use case.
    3. Besides defining your datasource, that's all that is required to live in PCF. That being said, there are other things that need to be configured, but you can use other mechanisms to do so (Spring Cloud Config Server, application.properties/yml, etc).
    4. Yes, you should be able to run the master locally and have it deploy the workers onto PCF if you're using the CF deployer.