How to Import Produced Data for Manual Import Type

Open your production folder and review the subdirectories for the following specifications:

• DATA:  Check for a .CSV and/or .DAT load files.  If you have a .DAT, you must convert this file type to a more user-friendly .CSV format in Step 2.  Additional file types such as .lfp, .log, .opt, may be discarded for purposes of these instructions.
• IMAGES: Check for single-page tiffs/jpgs named by their Bates number.  In some cases, the image files are provided as PDFs named by their Bates start and that is acceptable.
• TEXT:  Check for per-document text files.  These are named by the Bates start of each document.
• NATIVES:  Check for any native file types which have been provided.  These are named by the Bates start of the document.  Oftentimes Excel, Powerpoint, and other file types which do not image well (or at all). 

A reminder of what you should anticipate seeing in a ranged image import:

For successful import to Nextpoint, load files must be saved in a CSV (comma-separated value) format.

If you located a .DAT in your DATA folder in Step 1, you must first convert the file to a .CSV before proceeding.  Not only is it required for the Manual Import Type in Nextpoint, but making the conversion from .DAT    .CSV allows you to more easily view your load file and make necessary modifications (in Excel).

  VS  

Steps to convert a .DAT to .CSV:

1. Open your .DAT file in a text editor such as TextPad or Sublime Text (this is the application preferred by Nextpoint Data Analysts). 
   
   To open, right click on the .DAT file    Select "Open With"     click on the text editor program you would like to use.
   
   
2. Once opened in your text editor, the values in your .DAT file will likely be separated by the symbols þ□þ (thorn, □ (ASCII 20), and another thorn). So, the text displayed resembles this:
   
   þProdBegþ□þProdEndþ□þBegAttachþ□þENDAttachþ□þCustodiansþ
   
   If using Sublime text, your beginning state may look like this:
   
   þProdBegþ<0x14>þProdEndþ<0x14>þBegAttachþ<0x14>þENDAttachþ
   
   
3. Using the functions in TextPad or Sublime, you must complete a find and replace on the aforementioned characters to convert the load file format. Use the following sequence:
   
   1. Find-replace all " (quote) with "" (double quote) - This will ensure your line breaks remain consistent.
      
      
   2. Find-replace all þ (thorn) with " (quote)
      
      
   3. Find-replace all □ (ASCII 20) with , (comma) - □ may read as <0x14> if working in Sublime Text.
      
      
      
      
4. The find and replace actions you take above in Step 3 will result in all values being now separated by "," (quote comma quote, or in other words, comma separated values / CSV).  The text displayed should resemble this:
   
   
   
   
5. Though it is often not necessary, at times DAT files are not provided in UTF-8 format. Therefore, it is best practice to save the file with UTF-8 encoding. Select "File" > "Save with Encoding" > "UTF-8".
6. Save your file as: nextpoint_load_file.csv.  
   
   
7. Locate your nextpoint_load_file.csv in your production folder and open the file in Excel. Your values will be separated into columns, and much easier to work with moving forward.
   
   

Prefer a video tutorial on converting your load file from .DAT to .CSV?  Watch here >> 

Now, you will make modifications to your .CSV load file AND Nextpoint database fields to ensure all information in your load file has somewhere to go when imported into your Nextpoint database.

Once your load file is converted in Step 2, open your resulting .CSV in Excel, if you have not already.  Each row in your load file (spreadsheet) contains the information for one document. 

DOCUMENT BOUNDARIES

1. In your load file, review the first two columns to confirm Bates_start and Bates_end or a general control number is present (e.g. Bates_begin/Bates_end OR prod_beg/prod_end)
   • These beginning and ending numbers will identify which ranges of single page tiff/jpgs should be pulled from the IMAGES folder and combined to create individual documents.
     
     
     
     
2. If these two columns are named something other than Bates_start and Bates_end, you must rename these columns in your load file to Bates_start and Bates_end in order for your import to be successful.  *Capitalization and underscores (vs. a space) do not matter.
   
   
   

FAMILY RELATIONSHIPS

1. In your load file, review your column headers to confirm Begattach and Endattach column headers are present.  Sometimes these are titled production_begin and production_end, etc... 
   • These beginning and ending attachment numbers are vital to establishing email family relationships after your import is complete. 
   • Establishing these visual relationships after import is referred to as Family Linking and is further covered below in step 7.
2. In your load file, if named anything other than begattach and endattach (e.g. prod_begin/end, production_beg_attach/production_end_attach, etc..), you must rename these columns to Begattach and Endattach in order for your import to be successful. *Capitalization and underscores (vs. a space) do not matter.
   
   
   
3. In your Nextpoint Discovery database, if begattach and endattach fields are not present under Settings    Coding    Fields, then you must add begattach and endattach as Freeform fields.
   
   
   
   
   • Importing to a Litigation database?  You can add these fields under MORE     Settings    Coding    Fields.

METADATA FIELDS

Once your document boundaries and family relationships are verified in the steps above, the next step is to verify the metadata fields in your load file.  It is recommended you address the remaining load file fields in the following order:

1. DEFAULT FIELDS
   1. First, see this list of Default Fields and Document Attributes, Exhibit A ("Exhibit A"). Fields on this list already exist in each Nextpoint database.
   2. If a column header in your load file matches, or is similar to, a field listed in Exhibit A, you must rename your load file header to match the field name from Exhibit A.
      
      IMPORTANT: Your load file column header must exactly match the field name in Default Fields and Document Attributes, Exhibit A in order for your import to be successful.
      
      

   FAQ re: Default Fields

   Q: I'm concerned if I use a header in my load file that is similar to, but not exactly the same, as a Default Field, I will mess up my import. 

   A: Don't worry, you will not "break" the import!  Sometimes, it may be necessary to create similar fields to what is existing, and that is OK.  For example, Nextpoint has a default Email_Sent field which incorporates the date AND time an email was sent.  If you have Email_Sent_Date and Email_Sent_Time in your load file, it is perfectly fine to keep your load file fields named as such.  However, if you do this, please make sure to set up the corresponding field in your Nextpoint Discovery database by navigating to Settings    Coding    Fields    Create New    Select Freeform Field    enter the field name    click "Create".  This gives your load file data somewhere to go in Nextpoint once imported.

2. PROTECTED SYSTEM FIELDS
   1. Next, see this list of Protected System Fields, Exhibit B ("Exhibit B").  Data cannot be imported into any of these fields because they are generated by the Nextpoint application.
   2. If a column header in your load file matches a field listed in Exhibit B, you must rename the fields in your load file to something different AND ALSO set up a corresponding field in your database.
      • Renaming the field in your load file eliminates the use of a protected system field, and setting up the corresponding field in your database gives the information from your load file "somewhere to go" when your load file is ultimately imported into Nextpoint.
      • Suggestions for what to rename the field in your load file (and subsequently add to your database) are provided in parentheses next to each field in Exhibit B.
        
        RENAME FIELD IN YOUR LOAD FILE TO SOMETHING OTHER THAN PROTECTED SYSTEM FIELD:
        
        
        
   3. If you change a column header in your load file (in 2.2 above), you must also set up the corresponding field in your Nextpoint Discovery database by navigating to Settings    Coding    Fields    Create New    Select Freeform Field    enter the field name    click "Create".
      
      
      
      
      • Importing to a Litigation database?  You can add these fields under MORE     Settings    Coding    Fields.
3. TEXT & NATIVE PATHS
   
   Next up?  Text and Native Paths. 
   
   During import, Nextpoint needs to know which text and native files to pull from their respective TEXT and NATIVE folders, and line up with their corresponding document image(s) from the IMAGES folder.  This is accomplished by using the text_file and/or native_file column headers in your load file, which contain the path to, and name of, the text and native files, respectively.
   
   Oftentimes, you will not be provided with ALL natives, but rather a select set of file types which do not image well (or at all).  If you notice your native_file column lacking information in many of the cells, there is likely nothing wrong with your import, and rather only certain file types were provided natively (e.g. Excels, Powerpoints, Autocad drawings, etc).
   
   
   1. In your load file, make sure these two column headers are named text_file and native_file, respectively.  They must be named as such in order for the import to work correctly.
      
      
      
      
   2. Then, in your load file, check to make sure the paths are relative to where the load file will be saved.  It is critical to your import's success that the nextpoint_load_file.csv includes native and text path information relative to where the load file resides in your production set's base folder.
      • The relative path for any native file starts with NATIVES. For example, the first value of the native_file column in the image above is NATIVES\00001\NXP000149.xlsx
      • The relative path for any text file starts with TEXT.  For example, the first value of the text_file column in the image above is TEXT\00001\NXP_0000127.txt
      • See more information on relative paths here.
4. EVERYTHING ELSE
   1. After addressing the Default Fields, Protected System Fields, and Text/Native Paths above, check for any remaining headers in your load file without a corresponding field in your Nextpoint database.
   2. If remaining fields are found, create a new Field in your database named exactly the same as your load file header(s).
      • Working in a Discovery database? Navigate to Settings   Coding   Fields   Create New   Select Freeform Field and enter the field name.
      • Working in a Litigation database?  Navigate to MORE first, then Settings    Coding    Fields    Create New    Select Freeform Field and enter the field name.

After your load file is configured in Step 3, it is critical to the import’s success to save your load file as nextpoint_load_file.csv.  This is what indicates to Nextpoint during import "this is a load file and you should use it for this import".

1. First, move your load file to the root of your production folder. 
   
   
2. Then, upload your production folder to your Nextpoint File Room by dragging and dropping the unzipped folder.
   
    
   

Initiate the Import Workflow via DATA    Import    Import Files    Select Manual Import.

Once in the guided import workflow, take the following steps:

1. Select the Manual Import type in Step 1 of the Import Workflow    click Next.
   
   
   
   
2. You will be navigated to Import Data Settings.  Here, select your files for import by first clicking on the folder icon next to the Files text.
   
   
   
   Then, select your parent production folder    click orange Add selected to import list button     click blue Import button.
   
   
   
   After you've selected your file for import, complete the remainder of the Import Data Settings. This includes naming your import batch, selecting a folder to which you would like your documents added after processing is complete, deduplication settings(we recommend deduplication be off for produced data), and Child Processing settings. 
   
   What is Child Processing?  When enabled, children of parent emails or contents of zip files will be extracted and process in their standard format as stand-alone documents after import.

   When disabled, children will not be extracted from their parents. Disabling this feature may be leveraged in use cases such as native production imports with a load file, or in the event that children have been processed separately from the import at hand.
   
   Once the aforementioned settings are determined, click the blue Next button to initiate your import.

   

Once your Import is has completed processing, you will receive an email notification, and subsequently run “Family Linking" from your Batch Summary page. This is vital for visually establishing parent-email relationships in your import.

1. You will receive an email notification stating your import is Complete. Click on the link contained within the email, or navigate back to Data    Imports   and click on the title of the batch you just imported to view the Batch Summary.   Working in a Litigation database?  Navigate here via More    Data    Imports.
2. At the bottom right of your Batch Summary page, click  . A sliding menu will appear with checkboxes & headers for Batch, User, Uploaded, Docs (batch number, importing user, upload date, batch document count), and a selection drop-down for your Linking Options.
   1. Select the batch(es) you would like to Family Link.   You can link more than one batch at the same time.
   2. Select the fields Nextpoint should use to link parent emails to their attachments via the "Linking Options" drop-down.  If you utilized Begattach/Endattach in Step 3/Family Relationships, select that option from the drop-down.
      
      
   3. Click "Family Link Documents"
   4. Confirm your selection by clicking "Run Family Linking"
      
      

      

3. Once linking is complete you will receive an email with a link back to the Batch Summary page where you can download and review a report to ensure the emails and attachments were linked without issue.