Search code examples
sparqlstardog

Pagination in SPARQL query with Stardog: is using LIMIT and OFFSET without ORDER BY OK?


For some scalability reasons, I would like to paginate a SPARQL query to a Stardog endpoint. Currently, I generate multiple queries with various LIMIT and OFFSET parameters. I don't want to use ORDER BY since it is very slow. However, I can't figure out if the implementation of Stardog guaranties the same results are returned for each LIMIT and OFFSET configuration.

ps: the only discussion I could find on the Web about this problem is here.


Solution

  • A Stardog engineer here. You're right that ORDER BY has a noticeable overhead. Without ORDER BY Stardog cannot guarantee that the order of results will be consistent across multiple executions with different OFFSET values. However, esp. for simpler queries and in the absence of data changes, it should generally work that way.

    Stardog generally tries to evaluate queries in a lazy, streaming fashion so it's often possible to consume the result set incrementally without external pagination. If you share the query plan (or, even better, use the query profiler), we might be able to help with performance.