Xpages how to sort preselected large amount data from view

Question

I have a domino database with the following view:

Project_no    Realization_date    Author
1/2005        2015-01-02          Alex/Acme
3/2015        2015-02-20          John/Acme
33/2015       2016-06-20          Henry/Acme
44/2015       2015-02-13          John/Acme
...

Now I want to get all projects from this view that starts i.e with "3" (partial match), sort them by Realization_date descending and display first 1000 of them on Xpage.

View is large - some selection can give me 500.000 documents.

The FT search view option is not acceptable because it returns 5.000 docs only.

Creation of ArrayList or ListMap resulted with java out of memory exception (java Domino objects are recycled). Exceeding the memory may help of course but we have 30k users so it may be insufficient.

Do you have any ideas how can I achive this?

Answer 1

I would optimize data structure for your view. For example make a ArrayList<view entry>, that will represent the minimum information from your view. It mimics the index. The "view entry" is NOT Notes object (Document, ViewEntry), but a simplified POJO that will hold just enough information to sort it (via comparator) and show or lookup real data - for example Subject column to be shown and UNID to make a link to open that document. This structure should fit into few hundred bytes per document. Troublesome part is to populate that structure - even with ViewNavigator it may take minutes to build such list.