Search code examples
spring-dataaerospike

Pageable not working with spring data aerospike

I am using aerospike spring data

compile 'com.aerospike:spring-data-aerospike:1.0.2.RELEASE'
compile 'com.aerospike:aerospike-client:3.3.4'
compile 'com.aerospike:aerospike-helper-java:1.2'

When am trying to work with pageable am getting all the records instead of just one page with the specific set of records

Page<Account> findByGender(String gender, Pageable pageable);

This method is listed under interface

public interface AccountRepo extends AerospikeRepository<Account, UUID>

And accessing it by passing Gender and pageable

Pageable pageable = new PageRequest(page, pageSize);
Solution

  • Aerospike is a distributed database, which means that scans and queries are sent to all the nodes in the cluster. Each of the nodes is shared-nothing with the others, so they don't coordinate the order or records returned. Similarly, within the node scans and queries are multi-threaded operations spread across the cores. That's another reason why the order of the results is different each time.

    The workaround for pagination is to cache the full set of results on the client-side, then move through them. In an RDBMS, memory is allocated on the server-side for sorting and caching the results. In a distributed RDBMS, a specific node needs to be designated, all the results moved from the other nodes to it, and there it must be buffered, sorted, and cached. The design philosophy of Aerospike is to not waste server resources in such a way, but to move the responsibility of that operation to the application side instead, leaving the server cluster to handle many requests at very low latency.

    In your application you should buffer, sort, and cache the result, and paginate through this cache. You can actually cache these results back into Aerospike (for example into an in-memory namespace), with a limited TTL.

    Piyush posted in Medium on this topic: I wrote some Medium posts on how to sort results from a scan or query: