Summary

This article defines attributes that affect how the import functions. 


Table of Contents

The Imports worksheet defines attributes that affect how the import functions.

Basic Setup

Import Id

Enter a unique variable identifier for the import; i.e., import1, import2, etc.

Import Name

Enter the name of the import as it will appear to the end user in the Import menu; i.e., Text Import, XML Import, etc.

Import Description

Enter the optional internal description for the import, which will only appear in the XML file.

Logger Info

In the Logger Info section, you can set a parameter related to the formatting of the log file that is generated during the execution of the import.

Param Name

Enter ‘positionSeparator’

Param Value

Enter ‘false’ if each message should appear in line with its row identifier or enter ‘true’ if each message should appear on a new line under the row identifier. The default value is ‘true’.
The following example shows how the log file will change depending on the positionSeparator parameter.



Log Message

When errors occur during the import, configurable log messages are recorded in the import log file. While the Logger Info section is used to configure the formatting of the log file, this section is used to specify what information is displayed in the log file. Note that by default, all log messages will be displayed.

Message Id

Enter a Message Id based on the list of preexisting messages.

Suppress

Enter ‘true’ if the message should not be included in the log file. Enter ‘false’ if the message should be included. By default, log messages are not suppressed.

Prefix

Prefixes can be used to assign a classification to the different error messages displayed in the import log. These classifications are used as prefixes to the error messages. The following prefixes are currently supported.

  • ERROR – Prepends the error message with the text, ERROR
  • WARNING – Prepends the error message with the text, WARNING
  • IGNORED – Prepends the error message with the text, IGNORED
  • SKIP – Prepends the error message with nothing
  • SYSTEMMESSAGE – Prepends the error message with the text, SYSTEMMESSAGE
  • Custom Prefix – any prefix entered will be added to the front of the error message

In the following example, each prefix is applied to the Log_FormFrozen error message.



Custom Message

The Custom Message field is used to change the message to something other than the default message. When creating custom messages, it is important to note the parameters marked by curly brackets. Though these parameters do not have to be used in the custom message, they will not be included in the error log if they are not included in the custom message. In the following example, the Log_FormFrozen message has been customized.



Reader Information

Reader Type

Enter the document type. Available output types: Text, Excel, or XML.

Param Name

Enter the unique strings defining the properties for the reader type; e.g., delimiter (pipe delimited on text), qualifier, etc.

Param Value

The value to be passed to the Param Name in the configurable part of the reader; data to be stored for the properties needed for the reader type.

Reader Parameters for Text Imports

delimiter

This parameter lets you set the delimiter used to separate the input columns of input file. The default is ‘,’.

qualifier

This parameter lets you set the string qualifier for this reader. All occurances of this value are escaped in the text.

hasHeaders

This parameter lets you set whether the input file has headers or not. The value can be true or false. The default is true.

passCondition

This parameter lets you specify which rows will be imported from the file based on values provided in a column in the file. The value of this parameter can be an expression using the expression language. An example would be similar to the following:

passCondition(37 == ‘A’) OR (37 == ‘HP’) OR (37 == ‘H’) OR (37 == ‘L’) OR (37 == ‘LP’) OR (37 == ‘AA’) OR (37 == ‘AP’) OR (37 == ‘LA’) OR (37 == ‘HA’)

This expression says that if column 37 in the file has any of the above values, the row should be imported.

Note: If you are comparing a column value to a string and that string contains a space, the space will be removed before performing the comparison so something like: 31 == 'NO GROWTH' will be processed as: 31 == 'NOGROWTH'. To avoid this problem, simply use 31 EQUALS_RX NO\sGROWTH (no quotation marks around the string). Also, if one of the strings you are comparing to contains a capital "OR" like: 31 == 'HOOKWORM', you will need to find a way around using the term containing the capital 'OR'. One way to do this is again with regular expressions: 31 HAS_RX HOOK.


mimeType-[extension]

This parameter lets you specify the type of data being processed in the file. The format of this parameter is slightly different from the others. Examples are as follows:

  • Param Type: mimeType-csv, Param Value: text/csv
  • Param Type: mimeType-txt, Param Value: text/plain
  • Param Type: mimeType-xls, Param Value: excel
  • Param Type: mimeType-xlsx, Param Value: excel
  • Param Type: mimeType-xml, Param Value: text/xml


Note: The portion after the hyphen in the Type is the extension of the file. So if you were importing a .dat file, the Parameter Type would be 'mimeType-dat'.


Note: If the import will be executed via the Listener, it is recommended to specify the mimeType. Failure to specify it may result in the Listener not processing the file due to an unknown data type.

Reader Parameters for Excel Imports


Note: It is not recommended to use Excel files for importing purposes in the following cases:
  • If the file is large, Excel imports will consume a large amount of memory
  • If the data to be imported contains dates since Excel may modify the date formats
  • If the Excel workbook contains advanced functions which may interfere with reading the data
We recommend using ASCII files for imports or CSV files if the data must come from an Excel file since CSV files will be processed like an ASCII file. 


hasHeaders

This parameter lets you set whether the input file has headers or not. The value can be true or false. The default is true.

mimeType-[extension]

This is available for Excel and is configured the same as the text imports definition in the previous section.

Formatter Information

Formatter Type

This is the class name of the formatter that will be used in the import. For a standard import, this setting will be ‘record’. For a custom work, the Java file will be placed here – this is the path to the custom formatter.

Param Name

The unique internal parameter name (string) that defines the properties of the configurable part of the formatter; e.g., defaultChangeReason, defaultOverrideReason, and overrideLock.

Param Value

The value to be passed to the aforementioned parameter in the configurable part of the formatter.

Formatter Parameters for Text, XML and Excel Imports

defaultChangeReason

This parameter lets you set the default change reason for a question.

defaultOverrideReason

This parameter lets you set the default override reason for a question.

skipFormNotFoundWarning

This parameter lets you set whether the import log should indicate that a Form was found or not. The value can be true or false. The default is false. If you typically have a log file that is not accurate or if your import is setup in a way that causes forms to be evaluated that do not exist, it may be useful to use this parameter.

ignoreCase

This parameter allows you to use case insensitivity either completely or selectively for data entered into the CRF during an import. If the case of data entered should be ignored when evaluating all question paths, the value of the formatter parameter can be set to the asterisk (*) as shown in the following example.



To specify that only certain question path values should be evaluated in a case-insensitive manner, the parameter value should be a comma-separated list of the question paths used in the import configuration. In the following example, the case will be ignored when evaluating the visit date and date of birth questions only.

Note: In order for the case to be ignored, the question paths used for this parameter must match the question paths specified elsewhere in the import configuration.



overrideLock

This parameter lets you set whether you want to override the lock if the question or container is locked. The value can be true or false. The default is false.


Note: overrideLock is no longer a valid parameter. It should be replaced with overrideFreeze.


overrideFreeze

This parameter lets you set whether you want to override the freeze if the question or container is frozen. If set to true, values will be imported regardless of whether the question or form is frozen or not. The value can be true or false. The default is false.

Tool Property Information

The following four properties can not all be empty. One of them has to be set. If all of them are set, the system evaluates them in the following order: Method Name, Script Id, Tool Input Path, and Current Tool.

Tool Input Path

The path to get the value from the import file. For XML import, put xPath here; for Text and Excel Imports, put the column name here. The value is used to map the tool name in order to find the proper tool for import.

Script Id

The id of the script. The script is used to pre-process the value from import file.

Method Name

The customized method name in the extension file. The method is used to pre-process the value form import file.

Current Tool

Import data to the current tool. The value can be true or false. The default is true.

Record Property Information

Record Input Path

This only applies to XML import. The value is a xPath which is used to identify the context node in XML file.

Create New

This attribute lets you specify whether to create a new record or not if there is no existing matching record found.

Updating Existing

This attribute lets you specify whether to update the existing record or not if there is an existing matching record found.

Record/Form Key Map Information

Record/Form Key Input Path

The path to get the value from the import file. For XML import, put xPath here; for Text and Excel Imports, put the column name here. The value is used to map question value found by Question Path.

Question Path

The question path to get the question value. The value is used to map the value found by Record/Form Key Input Path.

Script Id

The id of the script. The script is used to pre-process the value from import file.

Method Name

The customized method name in the extension file. The method is used to pre-process the value form import file.

Form Property Information

Form Property Id

An unique identifier for each form property.

Form Input Path

This only applies to XML import. The value is a xPath which is used to identify the context node in XML file.

Form Path

The path to get a form from a container.

Create New

This attribute lets you specify whether to create a new form or not if there is no existing matching form found.

Updating Existing

This attribute lets you specify whether to update the existing form or not if there is an existing matching form found.

Allowed Form Properties

A form property can contain other form properties. Use the comma delimited form property id.

Allowed Question Property Sets

If data is to be imported to the current matching form, and if those questions are defined in a Question Property Set, specify the Question Property Set Id.

Question Property Set Information

Question Property Sets allow you to specify a group of questions to be imported to on any referenced matching form. For example, if Form Properties are defined for each visit’s Hematology form, a single Question Property Set can be defined for the values to be imported. The Set Id must be referenced in the Form Property’s ‘Allowed Question Property Sets’ field, and the specific question mapping (input path and question path) should be specified in the Question Property section below the Question Property set. This allows updates to any question mapping to be made in a single location, rather than in each form property’s individually-defined question properties.

Note: Use of Question Property Sets is optional. If question mappings will be individually defined for related form properties, remove the Question Property Set section.


Set ID

Specify a unique identifier for each question property set, for example ‘hemQuestions’.

Question Property Information

The following four properties can not all be empty. One of them has to be set. If all of them are set, the system evaluates them in the following order: Method Name, Script Id, Question Input Path.

Question Input Path

The path to get the value from the import file. For XML import, put xPath here; for Text and Excel Imports, put the column name here. The value is used to map the value found by Question Path.

Question Path

The question path to get the question value. The value is used to map the value found by Record/Form Key Input Path.

Script Id

The id of the script. The script is used to pre-process the value from import file.

Method Name

The customized method name in the extension file. The method is used to pre-process the value form import file.

Log Messages

Log messages can be broken down into types. Basic log messages can be used to provide broad information about an import and its success or failure. Error log messages can be used to gain more specific information about failed imports. This section describes each method and provides an example message. The blue text designates the portions that are specific to the import and its runtime.

Basic Log Messages

Log_Summary

The name of the import.

Import Summary: {0}
{0}: Name of the import

Import Summary: import1

Log_Username

Name of the user who initiated the import process.

Process Initiated By: {0}
{0}: Name of the user who initiated the import process

Process Initiated By: Rachel Workman

Log_ServerInfo

The name of the server that ran the import.

Process Started at: {0}
{0}: Name of the server that ran the import

Process Started At: server01

Log_StartTime

Time at which the import process began.

Process Started on: {0}
{0}: Date and time at which the import process began

Process Started On: Tue Mar 22 14:57:03 UTC 2016

Log_TotalTime

Total time that it took to run the import.

Process Finished in: {0} ms
{0}: Total time that it took to run the import in milliseconds

Process Finished in: 1607 ms

Log_FileProcessed

When a single file is being uploaded, this log message displays a ‘1’ if the file is processed successfully and a ‘0’ otherwise. When multiple files are being uploaded at once through a zip file, this log message displays X where X is the number of files in the zip file.

Processed {0}
{0}: A number based on the success or failure of the file processing as described above

Processed: 1

Log_FileLoaded

When a single file is being uploaded, this log message displays a ‘1’ if the file is loaded successfully and a ‘0’ otherwise. When multiple files are being uploaded at once through a zip file, this log message displays X where X is the number of files that were successfully processed.

Loaded: {0}
{0}: A number based on the success or failure of the file loading as described above

Loaded: 1

Log_FileFailed

When a single file is being uploaded, this log message displays a ‘1’ if the file failed and a ‘0’ otherwise. When multiple files are being uploaded at once through a zip file, this log message displays X where X is the number of files that failed to process.

Failed: {0}
{0}: A number based on the success or failure of the file loading as described above

Failed: 0

Log_StartFinishProcess

Displays the start and end point for processing.

*** Start Process: {0}
***Finish Process: {0}
{0}: Name of the file being processed

*** Start Process: test.xls
*** Finish Process: test.xls

Error Log Messages

Log_FileFormat

Displays an error message when the file is of the wrong format.

Please upload a file with the proper file format

Log_FormFrozen

Displays a message when a specific form is frozen and has the ‘overrideFreeze’ parameter set to false.

Form ‘{0}’ is frozen and ‘overrideFreeze’ parameter is false
{0}: Name of the form

Form ‘Demo‘ is frozen and ‘overrideFreeze’ parameter is false

Log_FormLocked

Displays a message when a specific form is locked.

Form {0} is locked
{0}: Name of the form

Form ‘Demo‘ is locked

Log_InputPathInvalid

Displays a message when the provided input path in the import is invalid.

Input path ‘{0}’ is not valid
{0}: Path provided in the import

Input path ‘demo/demo01‘ is not valid

Log_InputPathInvalidForScript

Displays a message when the input path for a script is invalid.

Input path ‘{0}’ is not valid for script ‘{1}’
{0}: Path provided in the script or DA file
{1}: Id of script

Input path ‘demo/demo01‘ is not valid for script ‘demoScript‘

Log_NoColumnMatch

Displays a message when the cells in the import worksheet are not the same size as the column list.

The list of cells is not the same size as the column list

Log_NoForm

Displays a message when a form could not be found for a parent form or record.

Could not locate the form ‘{0}’ for container ‘{1}’
{0}: The form path
{1}: The name of the container, which can be a form or record

Could not locate the form ‘vis1/screen’ for container ‘M0-111’

Log_NoFormFound

Displays a message when the form cannot be found. The message also includes key/value pairs for key maps if they can be determined. These key maps are separated by ‘||’

Form ‘{0}’ not found: {1}
{0}: The form path
{1}: The key value pairs for key maps if they can be determined

Form ‘vis1/screen’ not found: 2||1::04/05/2013||04May2013

Log_NoFormUpdate

Displays a message when the specified form is found but is not configured to be updated.

Form ‘{0}’ is found but ‘Update Existing’ attribute is false
{0}: Name of the form

Form ‘Demo01′ is found, but ‘Update Existing’ attribute is false.

Log_NoQuestion

Displays a message when a question could not be found on a form or record.

Could not locate the question ‘{0}’ for container ‘{1}’
{0}: Question alias
{1}: Name of the container

Could not locate the question ‘age’ for container ‘M0-111’

Log_NoQuestionFound

Displays a message when the specified question is not found on the form.

Question ‘{0}’ not found for form ‘{1}’
{0}: Question alias
{1}: Name of the form

Question ‘DemoQ1‘ not found for form ‘DemoF1‘

Log_NoRecordFound

Displays a message when a specific record or subject cannot be found.

Record not found: {0}
{0}: A key/value pair for the key maps in the format key1||value1::key2||value2 if the values are resolvable

Record not found: reg.sub||111:dem.age||18

Log_NoRecordUpdate

Displays a message when the specified record is found but is not configured to be updated.

Record ‘{0}’ is found but ‘Update Existing’ attribute is false
{0}: Name of record

Record ‘DemoR1‘ is found, but ‘Update Existing’ attribute is false

Log_NoScriptID

Displays a message when the provided script ID does not exist.

Could not find the script by id: {0}
{0}: Id of the script

Could not find the script by id: DemoScriptID

Log_NoToolByCustom

Displays a message when the specified custom method fails to locate the tool.

Could not find the tool by custom method: {0}
{0}: Name of method

Could not find the tool by custom method: DemoMethod

Log_NoToolByScript

Displays a message when the specified script fails to locate the tool.

Could not find the tool by script: {0}
{0}: Id of the script

Could not find the tool by script: DemoScript

Log_NoToolByInputPath

Displays message when the specified input path does not lead to a tool.

Could not find the tool by input path: {0}
{0}: Path to a tool that cannot be determined

Could not find the tool by input path: Demo/InputPath

Log_NoXpathFound

Displays message when the specified Xpath cannot be found in the current node of an xml file.

Xpath ‘{0}’ not found within the current context node
{0}: Path to a node of an xml file that cannot be determined

Xpath ‘Demo/XmlPath‘ not found within the current context node

Log_NoXpathMapping

Displays a message when the xpath specified for XML exports is not found.

Has to specify the {0} Xpath mapping in the configuration file
{0}: ‘form’ or ‘record’ depending on the import configuration

Has to specify the ‘DemoForm‘ Xpath mapping in the configuration file

Log_ParentFrozen

Displays a message when the parent form of a form that needs to be created is frozen and the ‘overrideFreeze’ parameter is false.

‘Create new’ is true for ‘{0}’, but its parent ‘{1}’ is frozen and ‘overrideFreeze’ parameter is false
{0}: Form path
{1}: Name of the parent form

‘Create New’ is true for ‘lab’, but its parent ‘Lab Summary’ is frozen and ‘overrideFreeze’ parameter is false.

Log_ParentLocked

Displays a message when the parent of a specific form is locked

‘Create New’ is true for ‘{0}’, but its parent ‘{1}’ is locked
{0}: Form path
{1}: Name of the parent form

‘Create New’ is true for ‘lab’, but its parent ‘Lab Summary’ is locked

Log_PassCondition

Displays a method when the specified column does not meet the requirements of the pass condition.

Did not meet the pass condition {0}
{0}: The expression used for the pass condition configured on the Import worksheet

Did not meet the pass condition DemoCondition

Log_RecordLocked

Displays a message when a specific record or subject is locked.

Record ‘{0}’ is locked
{0}: Name of the record

Record ‘M0-111‘ is locked

Log_RecordFrozen

Displays a message when a specific form is frozen and has the ‘overrideFreeze’ parameter set to false.

Record ‘{0}’ is frozen and ‘overrideFreeze’ parameter is false
{0}: Name of the record

Record ‘M0-111’ is frozen and ‘overrideFreeze’ parameter is false

Need more help?

Please visit the Fountayn Contact Information page.