Search code examples
javaxodus

How to get subset result from `txn.getAll("Entity")` method?


This is an example code to get list of Entities, example Users:

final EntityIterable allUsers = txn.getAll("User); // returns million records for example
for (Entity user: allUsers) {
    System.out.println(user.getProperty("fullName"));
}

My question, if the User store have millons of records, how can we get subset of results only? Like having a skip and limit is this possible through the Xodus API or we really need to program it to iterate on all entries and do the partitioning manually? What is the most efficient and fastest way to get subset of Entities in Xodus?

Here is a complete example implementation we're trying to solve:

@Override
public List<User> listUsers(String instance, final String storeName, long skip, long limit) {
    final List<User> users = new LinkedList<>();
    final PersistentEntityStore entityStore = PersistentEntityStores.newInstance(xodusRoot + instance);
    try {
        entityStore.executeInTransaction(new StoreTransactionalExecutable() {
            @Override
            public void execute(@NotNull final StoreTransaction txn) {
                final EntityIterable allUsers = txn.getAll(storeName);
                // TODO: How to skip/limit with 100k to millions of records, efficiently
                for (Entity userEntity : allUsers) {
                    User user = new User();
                    user.setEntityId((String) userEntity.getId().toString());
                    user.setUsername((String) userEntity.getProperty("username"));
                    users.add(user);
                }
            }
        });
    } finally {
        entityStore.close();
    }
    return users;
}

Solution

  • It's enough to just evaluate txn.getAll(storeName).skip(skip).take(limit). You would get an EntityIterable (Iterable<Entity) instance which you can transform then to a collection of users.