Deduplication Settings

*This functionality is available for Advanced users only.

Why use Deduplication?

Many collection efforts result in multiple versions of the same files (documents, mailboxes, etc). Uploading and storing a file multiple times wastes not only storage space, but also causes unnecessary complexity, confusion, and cost increases.

Understanding Deduplication settings

When you open a new Nextpoint case instance, Deduplication is turned on by default. To view/edit your Deduplication settings, go to SETTINGS > Import and click "Edit". 

Disable or enable Deduplication using the pill-shaped toggle at the top right of the Deduplication Settings screen. You can also change the definition of a duplicate by editing the criteria below.

File Match Criteria

When Deduplication is enabled, documents with the same content hash value are always considered to be an exact match. (That’s why the setting says “Always On.”)

In addition to content hash, emails may contain another unique identifier—the email-message-ID—which is generated by the client program (Outlook, Gmail, etc.) or the first email server at the moment the email is sent.

If you choose Include Email-Message-ID in your file match criteria, documents/emails with the same Email Message-ID will also be considered to be an exact match—even if their content hashes and headers do not necessarily match. This is considered to be a more aggressive form of deduplication.

Context Criteria

A file's context is determined by its load file values or its location within a folder structure.

Sometimes, two or more files may have the same content hash and/or email-message-ID, but appear in different contexts. (For example, copies of the same loose email file loaded at different times from different custodians’ folders.)

Choosing Include Context ensures that these files are not subject to deduplication, and will be preserved. This is the default setting.

Choosing Ignore Context means that any files meeting your File Match Criteria will be deduplicated, even if they appear in different contexts. It’s considered to be a more aggressive choice.

Note: An email's location within a container file (MBOX, PST) is not considered as Context Criteria. This is because the email is technically part of the container, rather than an independent document.

Once you’re satisfied with your Deduplication settings, click "Back".

Return to Review Workflow