CSV Overlays
What is an overlay? An overlay is an index file formatted as a CSV that, when applied to your Discovery or Litigation database via the Overlay workflow, can change the metadata or coding of a document which has been previously imported.
Using the load file mapper, users will be provided the opportunity to easily match fields from your csv overlay file to the fields displayed in your Nextpoint database, from within the application. We’ve also included several instances of load file and field validation to ensure your overlay is successful.
Some notable highlights of this release:
- Ability to perform CSV overlays via the new CSV Overlay Import Type.
- Document mapping key, providing Nextpoint with instructions for locating documents in your database which will be updated with information during the overlay.
- Effortless field matching with 'exact match' recommendations and the ability to search alternative existing fields or create new fields 'on the fly'.
- Backup Overlay File makes reverting data prior to the overlay stress-free.
This functionality is available for Advanced users only.
Formatting Requirements for a CSV Overlay
Your Overlay file should be formatted so that the first row of the CSV contains column headers (field names) and any subsequent row reflects a specific document.
Coding overlay column headers fall into one of three categories:
- Keys: Used to locate and target a specific document to which data will be applied
- Nextpoint System Fields: always available in all databases, not configurable
- Nextpoint Custom Fields: user-created fields which can be configured
We suggest supplying column headers as lowercase, underscore separated values. The system will do its best to coerce values like "Document Date" to "document_date", but we recommend formatting as much as possible before uploading to the File Room, as described in the steps below.
Each CSV row must contain at least one document key value. The Document Key provides Nextpoint with instructions for locating documents in your database which will be updated with information during the overlay. Available keys include:
CSV Header | Description |
---|---|
npid | The Nextpoint ID of a document. This can be exported in bulk from Nextpoint using the "Nextpoint ID" export field. For individual documents, this is listed under "System Data" as DocID. Tip: This is the most accurate and preferred key for an overlay; it is guaranteed to be unique and will result in the fastest execution speed. |
bates_start | The Bates value that corresponds with the first page of a document in Nextpoint. This can be exported in bulk from Nextpoint using the "Bates Range Start" export field. |
After your Document Key is defined in the first column of your overlay, all other fields which you intend to overlay in your database will fill in to the right in the first row. Subsequently, each row defines the data which will be overlaid in each particular field. See illustration above.
Outlined below is a list of Default System Fields to which you can overlay data to in your Nextpoint database. The only other fields which can be overlaid in your database are Custom (user-created) fields as described in the tab to the right.
CSV Header | Description |
---|---|
app_name | Appears as "App Name" under coding fields. |
author | Does not appear under coding fields, but is available for mapping. Often a merged field of document_author and email_author. |
bcc | Appears as "BCC" under coding fields. |
cc | Appears as "CC" under coding fields. |
created_date_time | This field must be a date and will be parsed as such. Note: American date formatting is always utilized, it is recommended that dates are provided in the format MM/DD/YYYY. |
document_author | Appears as "Document Author" under coding fields. |
document_date | Appears as "Document Date" under coding fields. This field must be a date and will be parsed as such. Note: American date formatting is always utilized, it is recommended that dates are provided in the format MM/DD/YYYY. |
document_last_author | Appears as "Document Last Author" under coding fields. |
document_subject | Appears as "Document Subject" under coding fields. |
document_title | Appears as "Document Title" under coding fields. |
document_type | Appears as "Document Type" under coding fields. |
email_author | Appears as "Email Author" under coding fields. |
email_received | This field must be a date and will be parsed as such. Note: American date formatting is always utilized, it is recommended that dates are provided in the format MM/DD/YYYY. |
email_sent | This field must be a date and will be parsed as such. Note: American date formatting is always utilized, it is recommended that dates are provided in the format MM/DD/YYYY. |
email_subject | Appears as "Email Subject" under coding fields. |
encrypted | Appears as "Encrypted" under coding fields. |
file_name | Appears as "File Name" under coding fields. |
file_path | Appears as "File Path" under coding fields. |
last_print_date | This field must be a date and will be parsed as such. Note: American date formatting is always utilized, it is recommended that dates are provided in the format MM/DD/YYYY. |
mailbox_file | Appears as "Mailbox File" under coding fields. |
mailbox_path | Appears as "Mailbox Path" under coding fields. |
modified_date_time | This field must be a date and will be parsed as such. Note: American date formatting is always utilized, it is recommended that dates are provided in the format MM/DD/YYYY. |
np_folder_prefix | Does not appear under coding fields, but is available for mapping. Provides user with ability to add documents to a folder via an Overlay file. See coordinating np_folder_position field. Applicable value(s) which can be mapped via Overlay are any existing folder abbreviation/prefix. This article outlines how to use it. |
np_folder_position | Does not appear under coding fields, but is available for mapping. Provides user with ability to add documents to a folder in a particular order via an Overlay file. See coordinating np_folder_prefix field. Applicable value(s) which can be mapped via Overlay are digits outlining which position you would like the document to take in a particular folder (e.g. a cell value of 6 would be the 6th document in the folder). This article outlines how to use it. |
recipients | Appears as "Recipients" under coding fields. |
root_folder | Appears as "Root Folder" under coding fields. |
shortcut | Appears as "Shortcut" under coding fields. |
title | Does not appear under coding fields, but is available for mapping. Often a merged field of document_title and email_subject. |
In addition to Nextpoint's default system field described in the previous tab, you can also overlay information to any custom fields which you create in your database.
All column headers should be lowercase with any spaces in the custom field names replaced by underscores. For example: “City Name” is represented in the csv header row as “city_name”
The way to format the CSV cells containing your document-level data will depend on the type of field being overlaid.
CSV Header | Description |
---|---|
Freeform & Paragraph | Any value can be provided. |
Date |
The coding overlay tool will attempt to parse data provided for this field into YYYY-MM-DD HH:MM:SS 24-hour format, which is how we require users to enter dates into our custom date fields, as it is sortable. If the provided value cannot be parsed into that format, the originally provided value will be overlaid. Note: American date formatting is always utilized, it is recommended that dates are provided in the format indicated above. |
List & Locked List | The exact list option you want to be applied. If this list option does not exist it will be created. |
Picklist |
These should be provided as a semi-colon delimited list of picklist options that should be checked. For example: "Option 1;option2; option 3" Trailing and preceding whitespace will be trimmed from the delimited values. This will go into Nextpoint as:
Keep in mind, all original picklist coding (the selections, not the options themselves) will be removed and replaced with what is provided in this field. |
Single Checkbox | If any data is provided in this field, the checkbox will be checked. If an empty cell is provided, the checkbox will not be checked. |
Overlay Formatting Notes
Bates, Custodians, Native_File and Text_File cannot be overlaid. Please contact the Nextpoint support team for assistance with this action.
Additionally, please ensure your load file is saved as a CSV with UTF-8 Encoding.
Importing a CSV Overlay with the Load File Mapper
Before initiating any Overlay, it’s important to note that the documents to which you would like to overlay data must already be imported into your Nextpoint database.
1 | Upload CSV Overlay load file to File Room
- Navigate to the File Room: In Discovery databases, via DATA File Room. In Litigation via MORE Data File Room.
- Upload your CSV Overlay load file(s) to the File Room via one of the four following options:
Upload a folder of files via Drag & DropSelect the folder(s) on your desktop, thumb drive, or other location, and Drag & Drop into your File Room. All contents and subdirectory information will be maintained.
Upload loose file(s) via Drag & DropWithin the File Room, click the green Create Folder name your folder and select OK.
Maintaining an organizational system for your data Overlay files will help ensure you can best track your various overlays as time progresses.
Click into your newly created folder, and drag and drop your Overlay file into the folder location.
Upload an individual file via Upload Files buttonWithin the File Room, click the blue Upload Files button and select your Overlay file from the directory prompt.
Note: This option only allows upload of one CSV Overlay file at a time.
Request file upload from third partyIf you are requesting a CSV Overlay file from a third party (e.g. counsel, etc.), you can securely Request Files from your Nextpoint File Room.
It is a simple process in which you ("requestor") request files from a specified third-party, the "recipient" of that request receives a secure link to upload their respective file(s), and then you can access the uploaded files right away.
Read more on requesting from third parties here >>
Best Practices for Uploading
- Overlay files must be in CSV format (no excel or .dat)
- We recommend organizing your CSV Overlay files with the data set you are overlaying or in a specific Overlay Files folder in the File Room. The better organized you are here, the easier it will be to cross reference later, as needed.
2 | Initiate Overlay Workflow
Once your Overlay file has been uploaded to the File Room, navigate to the Overlays tab, also under the top-level DATA tab.
Click the blue Start an Overlay button to initiate the overlay workflow.
3 | Confirm Overlay Settings
Once you initiate the overlay workflow, you will be navigated to the next step, Overlay Settings. Here, you will verify and/or outline settings applicable to your current overlay.
The Overlay Settings you will need to set are as follows:
- Type of Import: Verify Overlay is selected. Click the pencil icon
to edit, as needed.
- Overlay File Selection: Click the folder icon
to access the File Room contents and select your overlay file.
- Overlay Name: Recommended for most efficient tracking once the overlay is complete.
- Backup restored data: On by default, this setting will ensure existing field values are preserved in a separate CSV file before any information is overlaid. The resulting CSV file will be helpful if you ever need to revert an overlay to the previous state of coding/metadata.
4 | Review & Set Overlay Mappings
Load File Mapper at a Glance
When the Overlay Import Type is selected, the third step in the guided workflow is where you review and set your overlay file mapping specifications in the load file mapper.
The key objective of the load file mapper is to ensure the fields from your overlay file are mapping to the appropriate Nextpoint database fields. If the mapping is not made, Nextpoint will not know where to send the load file information during processing.
Once in the load file mapper, you will be presented with four core columns which provide you with data from your overlay file, a list of database fields to which load file data can be mapped, and a status indicator of your field mapping. These four columns are described in detail below.
A | The first column, Field from your load file, will be populated with a list of the fields contained in the header row of your selected overlay file.
B | The second column, Destination field, contains a list of available fields in your Nextpoint database to which corresponding overlay file field data can be mapped. Nextpoint will attempt to auto-map exact matches. You can also edit the auto-mapped recommendation, create a new destination field 'on the fly', or choose to skip the field within the field drop-down.
C | The third column, Load file preview, contains a three row preview of the data within the load file for each field. This information can provide assurance the appropriate values correspond with a particular field.
D | The fourth column, Field status, provides confirmation if a load file field has been successfully mapped to a destination field or if additional user action is required prior to import.
A Closer Look at the Destination Field Column
Now that you have an overview of four columns within the mapper, let’s take a closer look at the Destination field column and the different opportunities for mapping information from your Overlay load file to your Nextpoint database.
The first row in your Destination field column will be populated with document mapping key. The Document mapping key provides Nextpoint with instructions for locating documents in your database which will be updated with information during the overlay. See Formatting Requirements for a CSV Overlay above for further information on document mapping keys.
Click on the gear icon to validate the load file field which Nextpoint should use to locate the documents for your overlay.
Note: In the above visualization, you will note the duplicative ProdBeg (Bates_start) is skipped by the user. Bates cannot be overlaid, so there is no need to map that field beyond the document mapping key selection.
Auto-Mapping
For each load file field in column 1, Nextpoint will first look for an exact match with an existing Destination field. If an exact match is found, the field match will be auto-mapped.
For example, in the below image, BegAttach and BCC were an exact match so they are auto-mapped.
User-Mapping
For each load file filed in column 1, if an exact match can’t be made, fields can be user-mapped to existing or new fields.
For example, Tags were not auto-mapped in the below image because no exact match was present (and tags cannot be mapped). Therefore, we ask you as the user to select where that information should be mapped to in your Nextpoint database.
If a field needs to be user-mapped, each Destination field input contains a drop-down with a comprehensive list of all the fields which are available for mapping in your current database.
Varying text styles within the drop-down help to distinguish different types of fields.
- Normal (unbolded) field names are default fields which exist in every Nextpoint database. When possible, we recommend mapping to these default fields for streamlined searching later on.
- Grayed out fields are already in use within the current mappings and cannot be used twice. Remove/change the other field mapping in order to utilize a grayed out field.
- Bolded fields are user-created (or custom) fields which can be used in the mapper.
Create a New Field for Mapping
If the field you would like to use for mapping isn't available in the Destination field list drop-down, you are provided with the option to create a new database field on the fly. To create a new field while in the mapper, click the icon to the right of the applicable Destination field list.
Upon clicking the icon, confirm the auto-populated name or set your preferred field title, then click Create.
As soon as you click Create, that field will now be in your database and data can be mapped to that field moving forward.
You can skip a field mapping by selecting Skip Field in the Destination field drop-down, but skipping a field during mapping means skipping data going into your database. Therefore, we do require users explicitly skip a field for mapping via the Skip Field selection.
You can also skip all unmapped fields as a last selection before moving to the next step, processing.
5 | Initiate Your Overlay
Once all load file mappings are complete, click the blue Overlay button at the bottom right and processing will begin. Nextpoint will take your original overlay file, make a copy with any mapping requirements, and make any necessary conversions on the back-end during processing.
Warning!
All fields provided will be entirely overwritten by what is in the provided CSV. This means that if a field has an empty value in the CSV, that field will have its existing data removed in Nextpoint.
Before initiating your Overlay, we recommend ensuring Backup restored data is Enabled in when confirming your overlay settings. Doing so will ensure you are able to revert your overlay changes, if needed.
6 | Review Overlay Results
After your overlay is complete, we recommend reviewing your overlay results to ensure your overlay mapped as expected. You can review any overlay batch details via DATA > Overlays > click on the name of the overlay in the overlay batch list.
Several reports/files are generate during processing which will be helpful in your review:
- The overlay file used for processing will be added to your original folder selected from the File Room for import. Any selections made in the load file mapper will be reflected in this resulting file.
- Field Mapping Report: Fields used for mapping will be stored as a report in the individual overlay details page.
- Processing Results Report: Outlines the document mapping key utilized, which Nextpoint DocIDs the key matched, and the associated coding/metadata applied during overlay.
- Backup Overlay Load File: An CSV export of documents in your overlay batch and the coding/metadata in place prior to processing the overlay. Helpful in the event you need to revert your overlay for any reason.
7 | Family Link
Note: This step is only applicable if you were overlaying email family relationships via the combination of Bates_start document mapping key and BegAttach field.
The very last step once processing is complete is to Family Link your overlay batch. Family linking is the process which visually links imported (produced) emails and attachments using a set of two fields: "Unique ID" and "Family ID".
This takes place after import processing is complete and via DATA Overlays click on the overlay name to view Batch Details. At the bottom right of your Batch Summary page, click . A sliding menu will appear with checkboxes & headers for Batch, User, Uploaded, Docs (batch number, importing user, upload date, batch document count), and a selection drop-down for your Linking Options.
- Select the overlay batch(es) you would like to Family Link.
- Select the fields Nextpoint should use to link parent emails to their attachments via the Linking Options drop-down.
- Click Family Link Documents
- Confirm your selection by clicking Run Family Linking.
Read more on family linking here >>
After processing is complete, you will receive a green Complete, yellow Complete With Errors, or red Error status. Read more on Common Overlay Errors and Solutions here >>
Comments
Please sign in to leave a comment.