FAQ: How do I review results of more than 1,000,000 documents?

Question:

If I run a search that returns a large set of results, how do I handle reviewing documents that are after the first 1,000,000? In other words, how do I handle getting to those documents in the pages that aren’t displayed?

Response:

Nextpoint upgraded our search functionality to ElasticSearch 7.6 in March 2020.  With the upgrade came increased indexing/searching speeds (over 240%!), more stability, and the opportunity to continue to scale our search functionality as we move forward.

With this upgrade, we also applied a limitation on how high we can page in a set of results to ensure optimal performance is achieved.  These are things like, letting you click “page 30 out of 40,000 pages”, using the next <-> previous page arrows, and how many documents you can move through in your search results.

Before the upgrade, the limit was 10,000 documents, but now, by default, that limit is set at 1,000,000 results in a set.

We do not expect this to be a common problem our users run into, but we do want to make sure we provide you solutions for navigating results when the returned amount exceeds 1,000,000 documents.

Options for handling 1,000,000+ results

1. Apply a more restrictive search to reduce the overall set.  Nextpoint search is extremely flexible in allowing you to quickly add/remove criteria to refine or expand your search.  If you are looking at more than 1,000,000 search results, we highly suggest doing this to not only allow you to see all results but also to make your review more manageable.
   
   
2. Utilize the Filter as opposed to the Search functionality, and you will be able to page beyond 1,000,000 results.  This is due to the differences in how our search and filter functionality index documents.  
   
   If you do not yet have the documents sequestered in a folder, it is suggested you create a folder, run your particular search and then apply your folder to your results.  You will then be able to filter for that folder and paginate beyond the first 1,000,000 resulting documents and/or you can take advantage of the "Not Reviewed for Relevancy" and "Not Reviewed for Privilege" columns (if in Review) in order to review every document in the folder until all have been reviewed.  See screenshot below.
   
   Additionally, if you opt to folder results exceeding 1,000,000, you then have the option to split the folder into more manageable sets, thus allowing for a closer review of all documents stemming from the original folder and/or batch assignments.
   
   FILTER:
   
   
   
   UTILIZE REVIEW METRICS AT FOLDER LEVEL:
   
   
   
   
   
3. Utilize the column sorting functionality to reverse your result display.  When you run a search, the columns in your grid view can be sorted in ascending or descending order.  If you start the review of your results in ascending order (alphabetically, numerically or by the order in which a folder/issue has been applied), and need to review the end of the results, you do have the option to resort in descending order and review from that point.
   
   It is important to note, this option is limited to results of up to 20,000 documents, but can be used in conjunction with options 1 and 2 above.