Search code examples
javabatch-filejava-ee-6jboss7.x

Are there mature projects for Java Batch Job Farms?


We have Java EE servers that provide us load balanced, clustered web farms. I want to use the same EE technology, built on Java EE servers, but for running batch jobs.

I dont want to reinvent the wheel and dont imagine that I am the first to think of this.

This would be similar to WebShere's Feature Pack for Modern Batch, but I am looking for an Open Source solution, preferably supporting or capable of supporting JBOSS 7 and using a liberal license (Apache / MIT etc).

The ideal solution allows for:

  • is clustered - load balancing and monitoring jobs across as many machines within a cluster
  • runs everything within the single EE server jvm
  • has jobs as loadable modules pulled from a central repository, does not need to deploy new jobs to the server
  • support transactional (chunked) or compute intensive (single step) applications
  • provides an isolated container to run any number of job
  • allows for jobs to stopped, paused and restarted
  • load balances by running chunks across as many machines are the job is configured for
  • prioritizes jobs such that high priority jobs run first
  • allows jobs to be scheduled or launched on demand
  • has a simple web interface that allows monitoring of jobs
  • supports JMX
  • rich API

:D While we are at it...

  • has great documentation
  • optional commercial support is available but not necessarily from a 'for-profit' company that runs the project development

Dear Santa,

I have been good this year and helped old Mrs Wiggins crossing the street that one time and...


Solution

  • I think Spring Batch meets all your requirement.

    This is developed by Spring Comunity and good support of:

    1. Chunking
    2. Transaction management
    3. Stop, Start
    4. Failover
    5. Retry
    6. Job monitoring (Spring Batch Admin Web)
    7. Messaging
    8. Integration with job Schedulers
    9. Can run in load balanced environment
    10. Good documentation and Rich API
    11. Good books available with enough details such as "Spring Batch in Action" by Arnaud Cogoluègnes Thierry Templier Gary Gregory Olivier Bazoud