Deep pagination with jump to random page using search_after?


I am working on a website, where I need to search through an index with millions of documents, and be able to display them as pages of 500 documents. With pagination we need to support the jump to a random page.

I looked at Elastic Search documentation and it says we should use search_after for deep pagination. Documentation till 7.2 also says

search_after is not a solution to jump freely to a random page but rather to scroll many queries in parallel. It is very similar to the scroll API but unlike it, the search_after parameter is stateless, it is always resolved against the latest version of the searcher. For this reason the sort order may change during a walk depending on the updates and deletes of your index.

Reference: Search After | Elasticsearch Guide [7.2] | Elastic

Does that mean Elastic Search from version 7.3 onwards support jump to random page with search_after ?

Latest version’s documentation also says that scroll is not recommended.

Then what options do I have for deep pagination along with jump to random page?

Thanks and regards,

Deep, random pagination is a pretty classic problem. scroll is compounded by having to keep the state around and to manage that. And search_after isn’t a silver bullet, I don’t think that after 7.2 search_after changed that much, what did change is that 7.10 added Point In Time (PIT) in X-Pack (not Open Distro!) that made search_after more sane (search_after suffers badly in data sets with moving data).

What is your pagination sort context? Can you integrate that into your query? Have you considered aggregations to paginate?