Paging library - populate from cache while requesting from network

Question

I have to load cached version of data from database and simultaneously I want to make a request to server for fresh data and I want to do this on per page basis. So, for example for first page I want to show a cached version of first page data from database while requesting fresh data only for first page.
I want to achieve this using Paging Library.
I tried creating custom data source which helped me intercept page load request which then I used to make a network call with required page number and limit and meanwhile I returned a cached version from db, the problem is after getting fresh data from network I update the database but those updates are not reflected.
(I believe the whole table is being observed for any modifications using Invalidation Tracker and data source is invalidated whenever tables are invalidated, I added that tracker in my data source too but still it ain't working; I was able to make out that Invalidation Tracker thing by temporarily creating: LivePagedListProvider getJobs() in JobDao and checking generated implementation)

Code:

public class JobListDataSource<T> extends TiledDataSource<T> {

private final JobsRepository mJobsRepository;
private final InvalidationTracker.Observer mObserver;


String query = "";

public JobListDataSource(JobsRepository jobsRepository) {
    mJobsRepository = jobsRepository;

    mObserver = new InvalidationTracker.Observer(JobEntity.TABLE_NAME) {
        @Override
        public void onInvalidated(@NonNull Set<String> tables) {
            invalidate();
        }
    };

    jobsRepository.addInvalidationTracker(mObserver);
}


@Override
public int countItems() {
    return DataSource.COUNT_UNDEFINED;
}

@Override
public List<T> loadRange(int startPosition, int count) {
    return (List<T>) mJobsRepository.getJobs(query, startPosition, count);
}


public void setQuery(String query) {
    this.query = query;
}
}

Jobs repository functions:

public List<JobEntity> getJobs(String query, int startPosition, int count) {
    if (!isJobListInit) {
        JobList jobList = mApiService.getOpenJobList(
                mRequestJobList.setPageNo(startPosition/count + 1)
                .setMaxResults(count)
                .setSearchKeyword(query)
        ).blockingSingle();
        mJobDao.insert(jobList.getJobsData());
    }
    return mJobDao.getJobs(startPosition, count);
}

public void addInvalidationTracker(InvalidationTracker.Observer observer) {
    mAppDatabase.getInvalidationTracker().addObserver(observer);
}

Answer 1

So I understood why it wasn't working, there was a mistake at my end, I was passing wrong parameters to getJobs method of JobDao in JobsRepository.

The getJobs method of JobDao goes as follows:

@Query("SELECT * FROM jobs ORDER BY jobID ASC LIMIT :limit OFFSET :offset")
List<JobEntity> getJobs(int limit, int offset);

And the call getJobs() in JobsRepository goes as follows:

return mJobDao.getJobs(startPosition, count);

So the first parameter was the limit and the next one was the offset but I was passing other way around.
Now it works like a charm!

Furthermore, I made a change to getJobs() in JobsRepository: First get data from db, if available return and make an async request to network if required. If no data is available in db, make a synchronous call to network, get data from network, parse it and save it db and now access latest data from db and return it. So the function goes like this:

//you can even refactor this code so that all the network related stuff is in one class and just call that method
public List<JobListItemEntity> getJobs(String query, int startPosition, int count) {
    Observable<JobList> jobListObservable = mApiService.getOpenJobList(
            mRequestJobList.setPageNo(startPosition / count + 1)
                    .setMaxResults(count)
                    .setSearchKeyword(query));

    List<JobListItemEntity> jobs = mJobDao.getJobsLimitOffset(count, startPosition);

    //no data in db, make a synchronous call to network to get the data
    if (jobs.size() == 0) {
        JobList jobList = jobListObservable.blockingSingle();
        updateJobList(jobList, startPosition, false);
    } else if (shouldFetchJobList(jobs)) {
        //data available in db, so show a cached version and make async network call to update data as this data is no longer fresh
        jobListObservable.subscribe(new Observer<JobList>() {
            @Override
            public void onSubscribe(Disposable d) {

            }

            @Override
            public void onNext(JobList jobList) {
                updateJobList(jobList, startPosition, true);
            }

            @Override
            public void onError(Throwable e) {
                Timber.e(e);
            }

            @Override
            public void onComplete() {

            }
        });
    }
    return mJobDao.getJobsLimitOffset(count, startPosition);
}

updateJobList() code:

    private void updateJobList(JobList jobList, int startPosition, boolean performInvalidation) {
    JobListItemEntity[] jobs = jobList.getJobsData();

    Date currentDate = Calendar.getInstance().getTime();

    //tracks when this item was inserted in db, used in calculating whether data is stale
    for (int i = 0; i < jobs.length; i++) {
        jobs[i].insertedAt = currentDate;
    }

    mJobDao.insert(jobs);

    if (performInvalidation) {
        mJobListDataSource.invalidate();
    }
}

(I also renamed the getJobs() in JobDao to getJobsLimitOffset() as it makes it more readable and that is also the way methods are generated by paging library)