Search code examples
aws-sdkamazon-swf

Amazon SWF ActivityWorker - workerForCommonTaskList vs workerForHostSpecificTaskList


In Amazon SWF what is the difference between the following kinds of ActivityWorker - workerForCommonTaskList vs workerForHostSpecificTaskList. Does it mean the workerForHostSpecificTaskList picks up tasks on a list meant to be executed on the host where the ActivityWorker is running? If so how does one add tasks to such a list?


Solution

  • A task list is essentially a queue. SWF supports an unlimited number of task lists and they are created on demand without an explicit registration. It is up to an application to decide how many workers consume from a task list. The common design pattern is to have a common task list that worker on every host listens to and a host specific task list per host (or mesos or cubernetis task or even process instance). Then one of the activity tasks that was dispatched to a common task list returns the host specific task list name and then activities can be scheduled to a host specific task list to route activity executions on a specific host.

    How activity is scheduled to a specific task list is client side library specific. In Java AWS Flow framework it is done by passing it to an ActivitySchedulingOptions structure that can be passed as an additional parameter to an activity invocation. See fileprocessing sample that demonstrates such routing.

    BTW, have you looked at the Cadence, which is open source alternative to SWF which is actively developed and much more feature reach than SWF?