Search code examples
pythonbigtablegoogle-cloud-bigtable

BigTable - Drop/Skip the first N Rows


Is it possible to drop/skip the first N Rows in BigTable?

I'm trying to perform pagination for an API endpoint with BigTable outputs.

The closest filter is: CellsRowOffsetFilter (BigTable Documentation)

However, this filter only skips the N versions of the cell, and not the output row.

It's possible to drop the first N rows using iterators, but this will consume BigTable's read throughput, which doesn't seem like a good idea.


Solution

  • One approach as discussed in the comments to your question, would be to read some number of rows using a row limit, then pass the row key of the last row in the response as the start_key for the next request.

    See also Bigtable node client: How to set read offset for pagination?.