Overview

This guide covers issue surrounding initial setup of the system, specifically as it relates to the customization of patient and document data sources.

Before Getting Started

Before getting started, it is important to note a few things. There are three kinds of data: patient data, document data, and EMERSE application data. Patient data includes their name, birth date, demographics, and medical record number. Document data describes medical records, patient encounters, and reports. Finally, application data is the data generated through usage of EMERSE, such as patient lists, audit logs, and user roles and privileges.

EMERSE stores application data in the database, document data in Solr’s indexes, and patient data both the indexes, and in the database. Solr’s indexes are separate from the database, stored under the $SOLR_HOME directory, and can easily be terabytes in size. The database is often less than 50 gigabytes.

It is also important to note that as you are making changes to the database tables described here, some changes might be reflected immediately within the user interface, whereas others might not. If you run into trouble when making modifications, a good first step would be to restart everything and then see if the changes have taken place. For example, if you make modifications to the tables and then add clinical documents to the document index, those changes may not be reflected until Solr is restarted. Similarly, if patients are added to the database table, they are typically only copied to the corresponding Solr index once each day through a scheduled job (details in the Configuration Guide). It is possible to force this to happen more frequently when needed (potentially useful during initial setup and testing), and if that is desired such issues are detailed in the Troubleshooting Guide. Forcing a refresh can be done through the Admin app that comes with EMERSE.

Fields in Solr

Each patient and document is a "document" in Solr. When talking about Solr’s abstract notion of a document, we will call it a "Solr document" to differentiate it from a "medical document". A Solr document is just an assignment of values to fields. Patients and medical documents use distinct sets of fields. You can pick the names and number of fields used to described medical documents, so long as you have a minimal set of fields called "special fields" which play a special role in EMERSE and so much be present. Fields that describe patients cannot be changed in name or number.

Fields are described in the FIELD table:

Column name

Description

ID

A surrogate key

INDEX

This tells whether the field is in the DOCUMENT or PATIENT index, and must have one of those string values.

SOURCE

If this field is for the DOCUMENT index, then this fields says whether the field is for documents only of a certain source. A null value means the field is valid for all sources. (Null is also used for when the field describes a PATIENT field, but this column is also ignored in that case.)

NAME

This is the name of the field in Solr.

LABEL

This is the name of the field as shown in EMERSE.

DISPLAY_IN_REPORT

A value of true means show the field in the report view in EMERSE.

DISPLAY_IN_SUMMARY

A value of true means show the field as a column in the source-specific document table for a patient.

DATA_TYPE

This is either TEXT or DATE and tells EMERSE what kind of values are stored in the field.

Rows from this table can be edited in the Admin interface, in the "Fields" tab, so you do not need to directly modify the database. The fields describing the patient index (with INDEX = 'PATIENT') should not be removed from this table.

Field Values

Since each Solr document assigns values to fields, for each field, there is a set of values that actually appear in that field documents in Solr. These are called the "values" of the field. Sometimes the set of values is known upfront (such as departments) and sometimes not. Moreover, these values sometimes are not human-readable.

Patient Demographics

PATIENT table

Table: PATIENT
Population: From external source (such as EHR, or a staging document repository)
Population Frequency: Can be variable, but once per day is reasonable

The EMERSE schema includes a patient table with medical record number (MRN), name, date of birth, and other demographic information which is displayed in the search results. Data in this table are used to display the patient name, validate user-entered or uploaded MRNs and to calculate current ages of the patients. Other demographic data are used to summarize the populations found in a search.

Although the coded demographics information is not required, some features such as the demographics breakdowns within the All Patients search feature will not work if sex, race, ethnicity are not populated. Further, EMERSE can filter the results based on these demographics data.
For all documents indexed, there must be a corresponding patient in the patient table with a medical record number (MRN) that matches the document. This should be taken into consideration when determining the frequency for updating this table.
Column name Description Required or Optional

id

Primary Key

Required

external_id

Medical Record Number

Required

first_name

First Name

Required

middle_name

Middle Name

Optional

last_name

Last Name

Required

birth_date

Birth Date — used to calculate current age

Required

sex_cd

Sex

Optional (but highly recommended)

language_cd

Language (Currently not used. Can be populated in the system, and used for some advanced filtering, but will not be displayed to users.)

Optional

race_cd

Race

Optional (but highly recommended)

marital_status_cd

Marital Status (Currently not used. Can be populated in the system, and used for some advanced filtering, but will not be displayed to users.)

Optional

religion_cd

Religion (Currently not used. Can be populated in the system, and used for some advanced filtering, but will not be displayed to users.)

Optional

zip_cd

ZIP code

Optional

create_date

Date the row was created. Can be used to track changes to the table.

Optional

update_date

Date the row was updated. Can be used to track changes to the table.

Optional

ethnicity_cd

Ethnicity

Optional (but highly recommended)

deleted_flag

Logical delete flag. Useful for merged patient MRNs that result in obsolete MRNs. (see: Merges/Splits) Valid values are 1 = yes, deleted; 0 = no, not deleted

Required

deceased_flag

Currently not used. Valid values are: 1 = yes, deceased; or 0 = no, not deceased

Required

updated_by

To keep track of 'updated by' details

Optional

created_by

To keep track of 'created by' details

Optional
Obsolete/outdated MRNs should never be deleted from the table. Rather, they must remain and be flagged with the deleted_flag. Otherwise the functionality to warn users about obsolete MRNs will not work. Further, if the MRN is simply updated in the table rather than marking the older MRN as deleted and adding a new row, users will not be warned that the MRN has changed. This could cause issues if the MRNs are also stored in other data capture systems since the user would not know that the MRN in the external system is no longer valid.

Many of the columns in the patient table use codes for their values: sex, race, ethnicity, etc. Although these values are not constrained by the database, the UI can display descriptions for them in the patient demographics areas, and are used in the bar charts that breakdown sex, age, race, and ethnicity in the Find Patients feature, as well as in the Filters. The lookup tables for these codes are

  • LKP_PATIENT_SEX

  • LKP_PATIENT_LANGUAGE

  • LKP_PATIENT_RACE

  • LKP_PATIENT_MARITAL_STATUS

  • LKP_PATIENT_RELIGION

  • LKP_PATIENT_ETHNICITY

These tables all have the same structure:

Column Description

CODE

The coded value in the patient table

DESCRIPTION

The description that is shown in the User Interface

The EMERSE distribution has default codes already in the tables, but it is important to make sure that these codes match what your own local institution uses. Otherwise it will be necessary to update these tables with your local codes and text descriptions.

An example of how these codes are used within EMERSE can be found in our Virtual Machine Guide.

The data in the Patient table are automatically copied to a Solr index, by default once per day. This is detailed in the Configuration Guide.

Research Studies and Attestation

Immediately after each login, every user is required to ‘attest’ to their use of EMERSE for that session by specifying their reason for using the system. This is called the Attestation page, and the results are stored in the SESSION_ATTESTATION table. EMERSE provides three basic options (configurable by a system administrator; these options can be disabled) for this attestation: (1) a Free Text field, (2) pre-populated Common Use Cases (for example, “Quality Improvement”, “Patient Care”, “Infection Control”, etc), and (3) a list of research studies to which a user is associated. Additional tables RESEARCH_STUDY_ATTESTATION, and ATTESTION_OTHER may contain additional information depending on whether the user specifies a research study, or other reasons. The free text option would be used by users when no other attestation choices are reasonable. Additionally, previously used entries from the free text box will appear in the table, along with any IRB-approved studies, for the user’s convenience.

For our implementation at Michigan Medicine, we pull data on all studies in the IRB system, even if the study/person is not currently a part of EMERSE. This is because the dataset is generally small, and it makes it easier for users to validate their studies if the data are already populated, once the user is given an EMERSE account
ER diagram - Attestation
Figure 1. Entity relationship diagram of some tables related to capturing attestion data which occurs immediately after a user logs in.

RESEARCH_STUDY table

Table: RESEARCH_STUDY
Population: Populated from external source such as an electronic IRB system
Population Frequency: Can be variable, but once per day is reasonable

If a user is required to select his/her study from the table, then delays in moving IRB data to EMERSE after IRB approval can result in delays access for that user.

This table contains information about research studies. Using this table and RESEARCH_STUDY_MEMBER allows EMERSE to show a list of studies the end user is associated with.

Column name Description Required or Optional

id

Primary Key

Required

external_id

IRB study number — used to link specific studies to usage, and is very helpful for tracking research usage

Required

study_name

Name of the study

Required

principal_investigator_name

Name of the principal investigator

Required

prin_invest_org_id

id of principal investigator. Not currently used by EMERSE. This could be a user id, or email, but it is a good idea to ensure it is unique.

Optional

expiration_date

Expiration date of study. Used to determine if a user should be allowed to proceed. If the expiration date is older than the current date, the user will not be able to select it in the attestation GUI.

Required

project_status

Current project status. This is used to track where a study is in the review and approval process. Only certain study statuses allow access to EMERSE for research. The statuses that allow a study to be selected during attestation are defined in the VALID_RES_STUDY_STATUS table.

Required

last_updated

A last updated date is not used by EMERSE, but can be useful for troubleshooting and tracking changes to the table.

Optional

begin_date

This originally referred to the date the study began or should be allowed to begin. This field that can be used for tracking and troubleshooting.

Optional

VALID_RES_STUDY_STATUS table

Table: VALID_RES_STUDY_STATUS
Population: By System Admin.
Population Frequency: May only need to be done once, at the time of system setup. May need periodic updates if the source data (such as from IRB system) defining study status is changed.

EMERSE contains a simple table defining study statuses. The statuses that are initially populated in the system (loaded up in the build script) are unique to Michigan Medicine—​that is, they were developed locally and are implemented in our separate electronic IRB tracking system—​and other implementations would have to have their own set of valid statuses if these were to be used to validate and approve usage for research. If the status of a research study is not in this table, EMERSE will not allow the study to be used for attestation; in other words, the study would not even be displayed to the user to select.

Column name Description Required or Optional

status

A list of study statuses that EMERSE considers valid in terms of allowing a user to proceed. These statuses are generally defined by the IRB and are universal across studies.

Required

VALID_RES_STUDY_STATUS Table Example:

Status

Exempt Approved - Inital

Approved

Not Regulated

Exempt Approved - Tranistional

RESEARCH_STUDY_MEMBER table

Table: RESEARCH_STUDY_MEMBER
Population: Populated from external source such as an electronic IRB system
Population Frequency: Can be variable, but once per day is reasonable

This table contains information about study team members, and is related to the RESEARCH_STUDY table, described above. Each study can have one or many study team members.

This table at Michigan Medicine contains information on all study team members for all studies, whether they have an EMERSE account or not.
Column name Description Required or Optional

RESEARCH_STUDY_ID

Foreign key reference to row id in RESEARCH_STUDY table

Required

USER_ID

Foreign key reference to row in LOGIN_ACCOUNT table

Required

ROLE_NAME

A string describing a person’s role on the study team. EG. “PI”, “Staff”, “Study Coordinator”. This can be useful when generating usage reports.

Optional

FIRST_NAME

First name of the user who is on the study. It would likely be populated from the source IRB system, but it is not used at all by EMERSE. Nevertheless, it may be useful when generating reports.

Optional

LAST_NAME

Last name of the user who is on the study. It would likely be populated from the source IRB system, but it is not used at all by EMERSE. Nevertheless, it may be useful when generating reports.

Optional

BEGIN_DATE

This is not currently used by EMERSE.

Optional

LAST_UPDATED

Date row was last updated

Optional

DELETED

Flag to indicate if the record has been logically deleted.
0 = false, not deleted;
1 = true, deleted

Required

SESSION_ATTESTATION table

Table: SESSION_ATTESTATION
Population: Used internally by EMERSE
Population Frequency: In real time by EMERSE

Each time a user attests to why they are using EMERSE, a row is inserted into this table, which is one of the audit tables. Attestations related to research can be joined to the RESEARCH_ATTESTION table. Non-research uses can be joined to ATTESTION_OTHER.

Column name Description Required or Optional

id

Primary Key

N/A (populated internally by EMERSE)

type

A string indicating the top level category of attestation. RSA indicates session is used for research. OTH means other usage. Research attestations will have an associated row in RESEARCH_ATTESTATION. If the type is OTH, a row will exist in OTHER_ATTESTATION_REASON when the system populates that table at the time of a user attestation after login.

N/A (populated internally by EMERSE)

User_session_id

A foreign key reference to the USER_SESSION table

N/A (populated internally by EMERSE)

OTHER_ATTESTATION_REASON table

Table: OTHER_ATTESTATION_REASON
Population: By System Admin. Used if Common Use Cases for using EMERSE are desired to ensure consistency of attestation selections.
Population Frequency: May only need to be done once, at the time of system setup.

For non-research attestations, there is a lookup table called OTHER_ATTESTATION_REASON that lists available options. These can be configured by each institution, and may include commonly used access reasons that don’t involve research (such as quality improvement, patient care, etc). These options provide a simple way for a user to click on one of the pre-populated common reasons for use.

Column name Description Required or Optional

USER_KEY

Text based primary key of this table. The column name might better be thought of as as 'reason key'.

Required

DESCRIPTION

The text description that will be displayed in the table of options on the Attestation page.

Required

DELETED FLAG

Has this reason been deleted? (0 = no; 1= yes)

Required

DISPLAY_ORDER

Order of display in the UI. Can be any integer, but should be unique per row. The buttons are ordered by this column via sql sort. Generally start with 0,1,2, etc.

This is no longer used as of version 6.0

Optional.

This is no longer used as of version 6.0

OTHER_ATTESTATION_REASON Table Example:

USER_KEY DESCRIPTION DELETED_FLAG DISPLAY_ORDER *

QI

Quality Improvement

0

0

RVPREPRES

Review Preparatory to Research

0

1

STDYDESC

Study involving only decedents (deceased patients)

0

2

* This column is no longer used as of version 6.0.

ATTESTATION_OTHER table

Table: ATTESTATION_OTHER
Population: Used internally by EMERSE
Population Frequency: Application dependent

The free text reasons that users enter are stored in a table called ATTESTATION_OTHER. This is populated by EMERSE and is not customizable by users.

Column name Description Required or Optional

SESSION_ATTESTATION_ID

A unique ID for the session attestation. Used for audit logging.

Required

FREE_TEXT_REASON

The free text reason that a user entered.

Required

OTHER_ATTEST_REASON_KEY

This will currently only be populated by the system with FRETXT.

Required

ATTESTATION_OTHER Table Example:

SESSION_ATTESTATION_ID FREE_TEXT_REASON OTHER_ATTEST_REASON_KEY

50208

Testing out the system

FRETXT

52060

Testing out the system

FRETXT

46051

Looking up a patient in clinic

FRETXT

71052

infection control monitoring

FRETXT

74107

cancer registry operational work

FRETXT

Clinical Documents

EMERSE search is enabled by the indexing of clinical text documents by Apache Solr. Documents in a clinical environment can come from a myriad of sources like transcription, radiology, pathology, or from an electronic health record. Normally the structure, data, and metadata related to these documents from different sources varies considerably.

To simplify things, we configure Solr with a single document schema containing all fields from all sources. The document schema defines things like:

  1. what fields may appear in documents, and

  2. how to index those fields for search.

The latter is called the analysis settings for the field. Because there is only one schema that is used all sources, every source must use the same analysis settings for the same field. But, there is no requirement that different sources use the same fields, with the exception of "required fields."

Required fields must be present in every document so they can serve certain special functions. For instance, when the text of documents is to be searched, there must be only one field that needs to be searched, and it must search the text of documents from every source. Similarly, if search is to be constrained by date, there must be one field across all sources that holds the relevant date to filter on. (There can be other date fields, but they are not used during filtering, only for display in the user interface.)

Though they are required, the name of the field is customizable, which is achieved by looking up the name of the Solr field in the DOC_FIELD_EMR_INTENT table. This table has one row for each required field, which is identified to the EMERSE software by the NAME column. The values in this column must not be changed otherwise EMERSE will not find the row it’s looking for. The DESCRIPTION column describes how this field is used by EMERSE so you can set the proper solr field in the SOLR_FIELD_NAME column.

The DOC_FIELD_EMR_INTENT table covers all required fields except for the SOURCE field, whose name is not customizable (hence why it’s not in the table). This field tells EMERSE what documents belong to what sources. Its value should be one of the values of the SOURCE_KEY column in the DOCUMENT_SOURCE where sources are listed.

Beyond these required fields, a source may have any number of other fields that describes other information about the document, such as who wrote the document, when it was last modified, the department it is from, etc. These extra fields can be configured to be shown in EMERSE or can remain completely hidden (if say, they are used only to aid in debugging the document loading process).

The visibility of fields in EMERSE along with other field-level display settings are described in the DOCUMENT_FIELDS table. Each row of that table describes the settings for a single field of a single source. There must be a row for each required field as well, since required fields may still be displayed or hidden in certain views, and still need display information such as a label.

Sources themselves are described in the DOCUMENT_SOURCE table.

ER diagram - Documents
Figure 2. Entity relationship diagram of some tables related to clinical documents.

DOC_FIELD_EMR_INTENT table

Table:

DOC_FIELD_EMR_INTENT
Population:

No changes necessary if you use the defaults, otherwise only once at setup.

This table maps required fields to Solr field names. The required fields must be present in every source, and they serve special functions for EMERSE as described in the DESCRIPTION field.

SOURCE is also a required Solr field, but it is not listed here since its name is not customizable. SOURCE must store the value of the SOURCE_KEY column of the source the document is for. The values in the NAME column must not be changed, but SOLR_FIELD_NAME can be changed as desired.

Name Description SOLR_FIELD_NAME Required or Optional

MRN

Patient medical record number, which is a unique patient identifier

MRN

Required

RPT_ID

Unique document identifier. This must be unique across all documents and sources

ID

Required

CLINICAL_DATE

Date when the clinical event occurred. Often this would be considered the "note date" or "document date". When displayed for users within EMERSE in the Summaries section, this is the default sort column with the most recent date shown at the top.

ENCOUNTER_DATE

Required

LAST_UPDATED

Date when the document was last updated, since changes are sometimes made to documents

LAST_UPDATED

Required

RPT_TEXT

The actual text of the clinical document. This field is used by Lucene for lower-case indexing (case-insensitive searching).

RPT_TEXT

Required

RPT_TEXT_NOIC

A copy of the document text to be indexed using a case-sensitive Lucene filter (NOIC = NO Ignore Case)

RPT_TEXT_NOIC

Required

DOCUMENT_SOURCE table

Table:

DOCUMENT_SOURCE
Population:

Must be customized at setup. Must be changed when sources are added or removed.

Each source of documents (e.g., pathology, radiology, commercial EHR, legacy EHR, etc.) is listed as a row in the document_source table. The EMERSE application searches and displays the results based on document source. Additionally, the Filters in EMERSE can leverage these source data to limit queries to a specific source (e.g., searching only pathology reports). Document sources normally differ in their format and metadata depending on the source of origin. Each row in this table corresponds to a column in the Overview display within EMERSE, and as a subset of documents when a patient is selected.

Column name Description

SOURCE_KEY

A short name or abbreviation for the document source. This field needs to be unique as it is the primary key of the table. It should not contain periods.

USER_DESCRIPTION

A Description for the source of documents. This field is used only internally and can be useful for system admins who set up EMERSE to provide a a description of the source_key for easier recognition. This is not displayed to users in the UI.

HTML_FLAG

If set to false (0), the body of the note is wrapped in a <pre> tag, which will help preserve line feeds for display with documents that were not originally formatted for the web. If true (1) then it means that the document is already in HTML format with possible formatting tags, and no <pre> tag is used to wrap the document.

PRELIMINARY_DOC_FLAG

If it is possible that the source will have documents without text, this can be set to 1. If it is set to 1 another column of metadata will be displayed to users in the Summaries table for that document source in the UI. The heading for that column will be "Final" and it will show a Y in a document row if the document is final (meaning that there is at least some text there) or a N in a document row if it is not final (meaning that the document is blank). Typically this flag would be set to 0. Currently this is based on a source, and there is no setting on a document level (for example, to change the preliminary status based on whether a document is signed or not), although such a feature is possible for a future version.

DISPLAY_NAME

The name of the source as it is displayed in the UI (e.g., "Pathology", "Radiology", "Main EHR").

CSS_DISPLAY_PREFIX

Prefix used internally by CSS components in the UI. This can be anything, but each source must have a unique css_display_prefix. Additionally, it should conform to typical CSS naming conventions (e.g., no spaces, no quotation marks, etc). This is not displayed to the user; it is basically just an ID for the different tabs of the source documents.

DISPLAY_ORDER

Order in which sources appear in the Overview and the tabs within the Summary results page. Each row should have a distinct display order. Start sequential numbering with 0. This must start with 0 since that number is used as the index of the tab to open.

EXTERNAL_SOURCE

Currently not in use. It may be used if documents need to be displayed externally, for example with a PDF viewer outside the browser, and will not be displayed using SOLR’s copy of the document. (1 = yes, external source; 0 = no, no external source). Generally this should always be set to 0.

DOCUMENT_SOURCE Table Example:

source_key user_description html_flag preliminary_doc_flag display_name css_display_prefix display_order external_source

epic

Primary EHR

1

0

Epic EHR

ehr

0

0

rad

Radiology Documents

0

0

Radiology

rad

1

0

path

Pathology Document

0

0

Pathology

path

2

0

DOCUMENT_FIELDS table

Table:

DOCUMENT_FIELDS
Population:

Must be customized during system setup. Must be changed when sources are added or removed, or when new kinds of metadata are added to existing sources.

This table tells EMERSE what data sources use what Solr fields, when to display those fields, and how to display them. For instance, each document field has a name which is shown in the UI when the field is displayed. Each required field in the DOCUMENT_FIELD_EMR_INTENT should have a corresponding "standard" field in this table for every data source in the DOCUMENT_SOURCE table. In this way, sources can have different labels for the same required field.

The primary key of this table is a compound key on the DOCUMENT_SOURCE_KEY and DISPLAY_ORDER columns. This means every column must have a unique DISPLAY_ORDER within its source. This is true even if the column is set to never be visible, in which case any unique value will do.

Some of the data defined here includes the document text itself, but also the metadata fields that will likely vary for each source system (e.g., authoring clinician, clinical service, document identifier, or date of service). In the EMERSE UI, this metadata can be displayed or hidden:

  1. as a column in the summaries table by toggling the SUMMARY_DISPLAY_FLAG, and

  2. as a row in the metadat table atop the full document view by toggling the DISPLAY_FLAG column.

The Summaries table in the UI is the table of documents for a patient and a particular source. Clicking on a row in this table will lead you to the full document view.

For reference, the Overview page in the UI is the view before the Summaries table, where you can view each patient as a row and each source as a column, and a count of how many documents for that patient in that source match the search. One of the view toggles in the Overview view is the Mosaic View, where term colors are shown instead of the matching document count.

The metadata displayed within the EMERSE UI can, for the most part, be ordered by using the DISPLAY_ORDER column. The display order applies to both places in the UI where the data can be displayed, but note that for each of these two locations the system can be setup to display or not display the metadata element.

Column name Description

DOCUMENT_SOURCE_KEY

Specifies the document type key from DOCUMENT_SOURCE table. It should match the values of the SOURCE_KEY column in the DOCUMENT_SOURCE table.

DISPLAY_ORDER

Order in which fields need to appear in the search results, either in the Summaries section of the UI or in a small box above a single displayed document. This should be unique among rows for each source but note that some elements (such as the text of the document itself) would not actually be displayed as a metadata element.

DISPLAY_NAME

Name that appears in the UI

DISPLAY_FLAG

Flag that controls if the field is displayed when the document is displayed. This display of metadata is in a small table above the document when an individual document is shown in the EMERSE UI, when a user drills down to view a complete document. (1 = yes, display; 0 = no, do not display). Note also that for items such as the text of the document, that text will already be displayed and thus it should not be displayed here as a metadata element.

SUMMARY_DISPLAY_FLAG

Flag that controls if the field is displayed in the search results Summary page, which would show up as a metadata coulumn in the Summary results table. (1 = yes, display; 0 = no, do not display). Note that the case_date is a required field for each data source (listed in the SOLR_FIELD_NAME column) and it must have a SUMMARY_DISPLAY_FLAG set to 1, otherwise the browser will report a "no source column in index" error when using EMERSE, since that field is used as the default sorting column. Note also that for items such as the text of the document, it should not be displayed here as a metadata element since the Summaries section would only show the text snippets of search result 'hits'.

TYPE

Either STANDARD or CUSTOM. STANDARD means this field corresponds to a required field in the DOC_FIELD_EMR_INTENT table. CUSTOM means it does not. Every document source must have a document field for each required field in the DOCUMENT_FIELD_EMR_INTENT table. You may have as many custom document fields as needed.

EMR_INTENT

If TYPE is STANDARD for this document field, then this column is set to the NAME of the DOC_FIELD_EMR_INTENT that this field corresponds to. The Solr field for this document field is determined by the EMR intent row. If TYPE is not STANDARD, then this row should be null.

SOLR_FIELD_NAME

If TYPE is CUSTOM for this document field, then this column names the Solr field that corresponds to this document field. If TYPE is STANDARD this field should be null, because the Solr field is determined by the corresponding row in the DOC_FIELD_EMR_INTENT table.

DATATYPE

If TYPE is CUSTOM for this document field, then this column tells EMERSE how to understand the data in the Solr field. If this field is DATE, EMERSE expects the Solr field to hold a millisecond count from Epoch (January 1st, 1970 UTC), and EMERSE displays that date in local time in the UI. If this field is TEXT, then EMERSE expects arbitrary text to appear in the field, and renders that in the UI as-is. If TYPE is not CUSTOM this field should be null, because how the data is displayed is determined by the DOC_FIELD_EMR_INTENT row this document field is mapped to. (What EMR intents correspond to what data types should be obvious from the description of the intent.)

DOCUMENT_FIELDS Table Example:

Shown below is an example document_fields table for three different document sources:

DOCUMENT_SOURCE_KEY DISPLAY_ORDER DISPLAY_NAME DISPLAY_FLAG SUMMARY_DISPLAY_FLAG TYPE EMR_INTENT SOLR_FIELD_NAME DATATYPE

PATHOLOGY

1

Report Id

1

1

STANDARD

RPT_ID

null

null

PATHOLOGY

2

Last Updated

1

1

STANDARD

LAST_UPDATED

null

null

PATHOLOGY

3

Doctor Num

1

1

CUSTOM

null

DR_NUM

TEXT

PATHOLOGY

4

Collection Date

1

1

STANDARD

CLINICAL_DATE

null

null

PATHOLOGY

26

Report Text

0

0

STANDARD

RPT_TEXT

null

null

PATHOLOGY

27

Report Text

0

0

STANDARD

RPT_TEXT_NOIC

null

null

PATHOLOGY

28

Mrn

0

0

STANDARD

MRN

null

null

RADIOLOGY

1

Report Id

1

1

STANDARD

RPT_ID

null

null

RADIOLOGY

2

Last Updated

1

0

STANDARD

LAST_UPDATED

null

null

RADIOLOGY

3

Service Code

1

0

CUSTOM

null

SVC_CD

TEXT

RADIOLOGY

4

Doctor Num

1

0

CUSTOM

null

DR_NUM

TEXT

RADIOLOGY

5

Report Date

1

1

STANDARD

CLINICAL_DATE

null

null

RADIOLOGY

23

Report Text

0

0

STANDARD

RPT_TEXT

null

null

RADIOLOGY

24

Report Text

0

0

STANDARD

RPT_TEXT_NOIC

null

null

RADIOLOGY

25

Mrn

0

0

STANDARD

MRN

null

null

SOLR_INDEX table

Table:

SOLR_INDEX
Population:

At system setup time, the ID column should be set to the name of the directory the document index is in. Other values are updated automatically by the system periodically.

This table stores statistics and metadata of the document index, namely, the name of the directory containing the index (typically documents), the number of patients which have documents, and the date range of documents. This table only ever has one row, since it only describes the one documents index. Only the ID column needs to be set during installation; EMERSE automatically queries the index and populates the other columns. EMERSE looks for the index files at ${lucene.indexPath}/${ID} where ${lucene.indexPath} is the path specified the emerse.properties file, and ${ID} is the value of the ID column in this table. It updates its statistics on the index based on two properties in the emerse.properties file. See batch.updateIndexMinDateFromSolrIndex and batch.updateIndexMaxDateFromSolrIndex in the Batch Updating Begin/End Dates section of the Configuration Guide.

If the path to your document index’s solrconfig.xml is /home/your/indexes/documents/conf/solrconfig.xml, then you should set lucene.indexPath to /home/your/indexes/ and set the ID column to documents.

Column name Description

ID

The directory containing the documents index

START_DATETIME

Start date of clinical documents in the index

END_DATETIME

End date of clinical documents in the index

PATIENT_COUNT

Total distinct MRN’s found in the solr index. Presented in the count in "All Patient" patient list. Updated periodically by the application as a background task.

SOLR_INDEX Table Example:

ID START_DATETIME END_DATETIME PATIENT_COUNT

documents

01.02.2008 00:00:00

31.12.2099 00:00:00

1223829