General Direction of this Question
For this question, think of the job management that a web browser needs to do for downloading websites, images, and possibly other content for displaying to the user. Technically, this is a very similar scenario to what I'm facing. I'm asking for the choices I have (AsyncTask, Service, 3rd party libraries, ...) for implementing such a job management.
Details
I'm developing an app for android which needs to download stuff in the background and update the UI whenever more data is available from the running download.
When the user chooses to, he navigates to another Activity of his desire. The previously downloaded data then is unimportant to the new Activity. Hence the first download needs to be stopped (to spare bandwidth for a new download) and the new download for the new Activity needs to be started. Typical download times will vary between few seconds and half a minute to complete.
When the user navigates back to the first Activity, the first download shall not start from scratch. Instead, it shall be resumed.
When a download for the current Activity finishes, but there were other recent downloads which did not complete, they shall be continued so that in case the user navigates to those other activities which need that data, it can be presented immediately. Downloads shall be continued even if the app goes to background, or even regularly, say, once a day, so the app is ready to present reasonably up to date content even when offline.
What choices do I have for such a job management at API level 15? Keep in mind that it's about the job management / scheduling, not the downloading. Downloading stuff is rather easy to program, but scheduling is the difficult part.
Summary of requirements:
Choices I thought about
AsyncTask
I think it's not suitable, because it is bound to the Activity, so it's a bug to have this run in the background while the app is not in the foreground.
IntentService
According to Asking an IntentService for information about its queue it is impossible to alter an IntentService's queue.
Service
So far, I think I need to rewrite a scheduling mechanism from scratch, or base it on the source code of IntentService like suggested by the above SO answer.
Downloads shall be continued even if the app goes to background, or even regularly, say, once a day, so the app is ready to present reasonably up to date content even when offline.
This is a non-trivial point. This means that you need some sort of out-of-process scheduling component, which on API Level 15 means AlarmManager. That causes problems on newer versions of Android, so I would recommend either:
Using a wrapper library like android-job across the board, or
Using a ThreadPoolExecutor in a Service for handling the actual downloading, and using android-job to trigger that service to download things that need to be downloaded periodically
cancel/suspend jobs (hence cancel downloads)
Both of my options support cancelling jobs. You will have to invent your own mechanism for "suspend".
observer gets notified on job progress
IMHO, this has nothing to do with job management/scheduling. You will wind up using some sort of event bus for this (greenrobot's EventBus, LocalBroadcastManager, an RxJava-based bus, a MutableLiveData-based bus, etc.).
data must be accessible in parts
IMHO, this has nothing to do with job management/scheduling. I am not certain that it is even practical, depending on what you are using for your networking.
resumable jobs (read: resumable downloads; creating a new job for resuming a download is ok, of course)
You can create jobs using both of my options. You would have to create your own mechanism for "resumable".
jobs can run while app is not in foreground
This depends on what you mean by "foreground". If by "foreground" you mean "has an activity that is visible to the user", then both of my options cover this. However, if you go with the ThreadPoolExecutor approach, you will want that to be a foreground service, as on Android 8.0+, an unbound background service (e.g., one kicked off by AlarmManager) can only run for ~1 minute, and your downloads in total might exceed that. If you go with android-job, it should use JobScheduler on Android 5.0+, which gives you more like ~10 minutes before you start having problems.