Summary

The purpose of this guide is to help users that will be performing lab data imports into eClinical read the import logs so that they can do reconciliation or determine where errors might be in the imported data.

Table of Contents

Assumptions

It is assumed that if you are doing lab imports and you need to read the error log, that your import is already designed and that you have either a sample file or a live set of data that can be imported using the configuration within your application.

There are several messages or sections of the log file that appear in every log file that you should be aware of when reading the files. These are simple status indicators and provide basic information about who ran the import, when it started, which import it is (as defined in the dataArchitect), what file was imported, when it finished, and the results of the import. Those messages typically look like the messages listed below.

Initialization of Import Process

This message reports the ID of the import, the person running the import, when it started (in UTC time) and the file that was used for the import.

Import Summary: import2 Process Initiated By: LastName, FirstName Process Started On: Thu Sep 17 15:59:20 UTC 2009 

*** Process: Lab Results 15-Sep-09.txt ***


Finalizing the Import Process

This message reports that the import has finished, again reports the name of the imported file, then gives you the duration of the process, and how many files were processed, loaded, and failed. In the case where a single ascii file is used, the counts will never be greater than one. In the case where a zip file is uploaded containing multiple files, the numbers will match the number of files in the archive that was submitted.

*** Finish Process: Lab Results 15-Sep-09.txt *** 
Process Finished In: 1279613ms 
Processed: 1 Loaded: 1 Failed: 0


Error Messages

Error messages regarding imports can be found in 3 specific places. The first is an error that will appear within the Import screen if you cause or trigger an error that is in regards to the user interface and managing the files. The second is the basic error log which gives very high level information about the error and what caused it. The third is a detailed error log that is found in the archives screen of the Import interface. Error messages are currently broken down into two categories for imports:

  • Uploading Errors
  • Importing Errors
  • Exception Errors

Uploading Errors

When uploading a file to be imported, there are a variety of messages that may appear if something is performed incorrectly or if the file does not meet the correct specifications. Those errors are detailed as follows.

If the “Import” button is pressed without specifying a file from the local computer:

You must specify a file for: (import name)


If the import file specified is corrupted:

Error when writing the uploaded file to file server for: (import name)


If the file specified is not in the proper format or has an incorrect extension:

Please upload a file with proper file format


Importing Errors

When running the import process, there are a variety of standard errors that may appear in the log indicating missing data or data that will not be imported. These errors are as follows.

If the import tries to update a value on a question that is locked:

SYSTEM MESSAGE: Attempted to change value for locked question vitalsV1 on form Physical Exam for Patient 123456(A-C)


If the pass condition is not met, meaning the criteria to import that record has not been met:

Row (number) did not meet the pass condition


If the record (patient) cannot be found and the import does not have permission to create new records:

Record not found for tool (tool name)


If the import has permission to update existing records but the record is locked and the import is not allowed to override (ignore) the lock:

Record is locked and overrideLock parameter is false


If the import does not have permission to update existing records and updates to that record’s data are found:

Record is found but updateExisting attribute is false


If the import cannot find a form and does not have the permission to create a new form:

Form not found


If the import has permission to update existing forms but the form is locked and the import is not allowed to override (ignore) the lock:

Form is locked and overrideLock parameter is false


If the import does not have permission to update existing forms and updates to that form’s data are found:

Form is found but updateExisting attribute is false


Exception Errors

Exceptions are errors that are not captured and reported cleanly like the previously mentioned errors. They will still report an error as with all errors but the error may appear with additional information and list some debugging information.

One example of an exception is when the number of columns in a particular flat file does not match the number of columns that the import is expecting and mapped to. That will produce an error similar to the following.

Row: 100 ERROR: Number of cells does not match 
Processing Failed: The list of cells is not the same size as the column list 
The list of cells is not the same size as the column list com.clickfind.utility.data.DataRow.addAllCells(DataRow.java:111) com.clickfind.dataCollector.genericImport.DCTextReader.parseToRow(DCTextReader.java:279) com.clickfind.dataCollector.genericImport.DCTextReader.next(DCTextReader.java:194) com.clickfind.services.medicalResearch.sponsor.sponsorFormatter.processFile(sponsorFormatter.java:69) com.clickfind.dataCollector.genericImport.DCRecordFormatter.runImport(DCRecordFormatter.java:290) com.clickfind.dataCollector.genericImport.DCImport.run(DCImport.java:359) com.clickfind.dataCollector.genericImport.DCImportRunner.run(DCImportRunner.java:47)


Other exceptions include errors that indicate terms similar to the following:

  • Null pointer – this may indicate that a reference was found but the form or record was unable to be located
  • Optimistic Lock – this indicates that someone else was editing the form while the import was trying to update the form
  • Concurrent Modification – this indicates that a question that the import was trying to update was also attempted to be updated by a user in the system at the same time

XML import error/warning messages

Text imports can easily locate a problem with the import data by a row number. For XML documents there is a different way of describing the location where the problem had occurred. XML documents consist of elements, and attributes in a tree like structure with exactly one root element.

XML error or warning messages first specify a path to the node to which the message relates, followed by the text of the message. This path usually starts at the root of the XML info set. Example:

Path: GTP[1]->Study[1]->Site[2]->Investigator[1]->Subject[3] WARNING: Could not find the tool by custom method: some method name here


The path expression starts with node “GTP” followed by a “1″ in parentheses. The “1″ means that this is the first node under the previous node. Due to the XML structure the root node needs to have a “1″ here. The next item in the path expression describes the node under the previous one, here the first “Study” node, followed by the second “Site” node, the first “Investigator” node and the third “Subject” node. If you look at the imported XML file you will find the node structure starting from the root element of the document.

However, if the problem occurs inside of the patient processing, the path will not start at the root. This is due to the performance of the import. When the path starts at the patient node, then the message will list the tool name and patient name. This way it is ensured that problems can be found.

Path: Subject[1]->Visit[1] WARNING: Tool 1234-Site - Record 1234(0001) - Form 'labscreening' not found: Accession/BaseSpecimen[1]/SpecimenCollection/@ActualCollectionDateTime||15-Oct-2010


The XML error or warning messages usually contain xpath expressions that are used for the import configuration. This makes the reading of messages a bit more complicated for users that have not written the import configuration themselves. However, there is usually only a small amount of used xpath expressions. This sample message above means that there is no “labscreening” form that holds the date “15-Oct-2010″ for the lab data collection date.

A typical warning message for XML imports is that not all possible data is available in the import file. Example: Only the screening visit has been done, but no visit2. Lab files usually contain only the data that had been processed so far. However, the configuration of the import was done for complete XML data that does not exist by now. Here is an example of such a warning that can usually be ignored:

Path: Subject[1] WARNING: Tool 1234-Sitel - Record 1234(0001) - Could not find any node in the current context node with xPath: Visit[@ID="2"]


Need more help?

Please visit the Fountayn Contact Information page.