I have problem with write code for that problem. My application need VPN connection to download some information from another systems.
So, I create multiple jobs to download some informations from another system and each of this jobs should be establish VPN connection, next download data and finally close the connection. I would like to wrap all of these jobs even one that will handle the VPN connection and only one establish ane close connection.
I think that I should be use one queue - 'VPN Jobs' and put there other jobs that will be performed in other queues.
To establish VPN connection i must use jnc (Juniper network connect). I would like to run jobs with ActiveJob and resque.
I hope I explained it come some understandable :)
Update
Image for better for a better understanding.
- VPN - Wrapper queue to VPN connection
- Q1, Q2, Q3 - Queues with some jobs needed vpn connection
I think you should use resque-batched-job gem for organize your jobs to batches. This gem provide after_enqueue and after_batch hooks. You can use them for establish and close VPN connection.
I am not a sure exactly but probably this gem has ability to organize nested batches. In this case you can create a main batch job which establish connection to VPN and run nested batch job for uploading images in different queues.
FYI: Sidekiq has the same functionality (Batch section) out of the box but in Pro version.
