This guide provides a general overview of the components needed to run EMERSE, how to install them, and verify that they are running. The guide covers step-by-step directions on how to install everything on a virtual machine, so these directions should be used as a high-level guide if EMERSE will be installed in a different type of environment.
For this guide we provide 4 already created Solr indexes to use as well as SQL scripts to not only create, but also populate, the database (all files available upon request). An actual production instance would require the local creation of the Solr indexes, locally populating the patient table, and additional localized configurations dependent on local document sources and metadata. Nevertheless, this guide should provide enough detail to understand how everything is installed and provides the ability to test the final installation to ensure that it is functional.
EMERSE is mainly tested on enterprise linux systems (RHEL and SUSE), so we recommend these or their open source equivalents. Windows based platforms should work as well, provided they are recent enough to have quality Java releases, as EMERSE and most of the components run within a Java Virtual Machine.
To install EMERSE, someone with sys admin skills is desirable. This person should be able to install and configure software, edit database tables, etc. No actual programmer experience is needed for a default installation and configuration of EMERSE. Knowledge of servers, Java, Linux, and Apache software will be useful skills for this installation.
The initial setup described in this guide can probably be done in a day or less. More time will be required for additional customizations and planning related to localized needs, document extraction and indexing, regulatory approval, etc.
A Linux/Unix based server is suggested for installing the application server and for hosting the indexing services. No specific type of storage is required, but in general the server should be connected to the highest speed storage available. EMERSE performs a lot of disk reads to retrieve the documents, so read performance is important. We have conducted some experiments using SSD storage and have found it to provide nearly a two-fold increase in system performance for EMERSE, most noticeably in the area where all documents are retrieved and highlighted for a pre-defined set of patients.
Storage capacity is dependent on number of documents to be indexed. At Michigan Medicine 1.5TB is in use to host approximately 100 million documents in both TXT and HTML format on the production server. An additional 3 TB is available for index optimization, as optimization requires about 2x the size of the index. If your documents are heavily formatted (e.g., RTF instead of TXT), storage requirements may be higher.
EMERSE doesn’t place great demand on the Oracle database, so a relatively small server can be used with 10-50 GB of storage allocated for user tablespaces. Currently at Michigan Medicine we are using only 8 GB of space for production EMERSE with 600+ users and 5 years of data.
For a production install of EMERSE you will probably want to set up a Linux server based on whatever local options are available. For the purposes of this install guide, a Virtual Machine (VM) running Redhat Enterprise Linux (RHEL) will be used to demonstrate one way in which this can be done. When selecting the type of Linux to use, note that the Oracle-XE software is distributed as an RPM file. This means that systems such as Redhat or SUSE would good options, whereas Ubuntu would not be.
Note that we are only using a VM to demonstrate the installation process. We do not recommend using this VM approach for a production-level installation. RHEL can be used at no cost for development purposes. To get started, download the DVD iso file at:
|To download the DVD iso file you will need to set up an account with Redhat and accept the terms and conditions.|
Download and install VirtualBox application locally, which can be found at:
In the VirtualBox menu choose Machine → New
It will then prompt for a name, which can be anything you want. Also choose Red Hat Linux (64-bit), then press Continue. For example:
On the Memory Size screen set the memory to 4096 MB, and press Continue.
On the Hard disk screen, choose the option Create a virtual hard disk now, and press Create.
When it asks for about Hard disk file type choose the VDI (VirtualBox Disk Image) option and press Continue.
On the Storage on physical hard disk screen, choose Dynamically allocated and press Continue.
On the File location and size screen, leave the text alone (it should say EMERSE installation demo) and change the size to 15.00 GB, then press Create.
Now, in the VirtualBox application, make sure the EMERSE installation demo VM is selected and press the green Start arrow.
Once the VM starts it will ask for the location of the Redhat Linux DVD iso file, so browse to the right location where it was downlaoded to select the file. You should see something like:
Once you have located the file, press Start. This will begin installation of the operating system.
The Red Hat software should start loading. You can use the up arrow to select Install Red Hat Enterprise Linux 7.4 (or whatever the current version is), then press Return.
Follow the prompt to choose the language, and then on the Installation Summary screen, do the following.
Click on System to select the Installation Destination. Then, under the Device Selection section, click the hard drive icon where it says "15 GiB…ATA VBOX HARDISK". You may have to click twice until is is blue in color and the checkbox appears over it, as shown below, then click Done.
Then, click where it says Software Selection, and under the Base Environment column select Server with GUI, then press Done (none of the add-ons in the right column need to be selected).
|There is no actual need to have a GUI, but it is easier to work with for these installation instructions.|
You may have to wait a little bit for the system to check to make sure everything can be found, and once it is ready click on the Begin Installation button.
While the installation is underway, click on the Root Password section under User Setings and enter a password, such as
demouser. Then, in the User Creation section, create a user for the system, such as:
Make this user administrator
|These usernames/passwords are not needed by the EMERSE application so they can be anything.|
Once the installation is complete, click on the Reboot button. After the machine has rebooted, click on Licensing Information to accept the license, then click the Finish Configuration button. A few other setting options will then be offered, and then the system is ready to use.
|It may also be necessary to turn on the network within the VM so that files can be downloaded the the VM. To do this, go to Applications → System Tools → Settings → Network and change the Wired option to On.|
The remainder of this guide covers the process of installing and configuring the application server where the EMERSE application will be deployed. Specifics for installing each of these components follows.
For some of the commands shown in this guide, specific version numbers will be shown as examples. Depending on the software component, the version number you download and install may differ slightly from these examples. Make note of these potential differences when executing the instructions on the command line to ensure that they work successfully.
Also note that most of these directions are directed at a Linux installation, but we have provided some details for Windows for those interested in trying to install EMERSE on a Windows server.
For the purposes of this installation guide, all components will be stored in a directory called
app. In reality it can be any directory, but for consistency they will go here in this guide. To make the directory, go to Application → Favorites → Terminal and in the Terminal type:
sudo mkdir /app
In a few cases for the following directions it will be necessary to edit some text files. There are multiple Linux software programs to make these edits (e.g., vi, emacs, etc) but a simple one with this Virtual Machine installation is gedit, which can open the files in a regular graphical interface text editor. To use it, open the file from the command line by typing the following, where filename.txt is the name of the file that should be opened.
sudo gedit filename.txt
It may also be necessary to turn on the internet connection to the virtual machine so that software can be downloaded to it. To do this, go to the upper right corner of the screen and click on the battery/power icon, then click where it says Wired Off and choose Connect.
Java Development Kit/SDK (JDK)
Java Servlet Engine
7.3.1 (only, not higher)
Indexing/Information Retrieval System
Apache Active MQ
A message broker
11g or higher
Express Edition is fine for testing or light production use. Setup scripts will be provided by the EMERSE team upon request
EMERSE WAR file
EMERSE Web Archive File (WAR file) deployment and configuration
Provides database-agonstic data persistence
Provides search capabilities by reading SOLR index files directly
Provides security services
Used for making the demographic summary charts
The first step in installing EMERSE is to download and install a Java Development Kit (JDK) on the server. Java 8 or higher is required.
Operating Systems often include a Java runtime, and is available on the "Path" variable. We recommend explicitly using the JDK suggested in the Required Software table. We do not recommend using openJDK although it is possible it could be used (we have not tested it).
All of the remaining components that will be installed are dependent on a path to the location of the installation of this java JDK installation, so note the location of the path to the JDK installation—it will be needed in subsequent installation steps
After downloading the Java appropriate for the Linux server, move it to the
/app/ directory and unpack the file. Assuming that the file is downloaded to
/home/emerse/Downloads/ then to move the file:
mv /home/emerse/Downloads/jdk-8u181-linux-x64.tar.gz /app/
tar -xvf jdk-8u181-linux-x64.tar.gz
rpm -ivh jdk-8u181-linux-x64.rpm
Windows: After downloading, run the executable. If the target workstation has an existing Java installation that you wish to maintain, unselect the “public JRE” option in the installer as shown below.
Confirm installation of java by typing into Terminal:
If there are multiple versions of Java installed, and the newly installed version is not set as the default, then confirm the specific installation using:
You should see a message that says something like:
java version "1.8.0_181" Java(TM) SE Runtime Environment (build 1.8.0_181-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)
EMERSE is packaged as a Java based WAR file that will be deployed to the Tomcat Servlet engine. Tomcat depends on a Java JDK. Tomcat version 9.0.7+ is required. The Tomcat download page lists a number of components available for download. Only the "Core" software is required. See Required Software section.
From the binary distributions listed on the page, choose the "Core" tar file (for example, the
tar.gz file). Move the tar file from the Downloads directory to the desired directory for installation.
mv /home/emerse/Downloads/apache-tomcat-9.0.12.tar.gz /app/
Extract the tar file using
tar -zxvf apache-tomcat-9.0.12.tar.gz
Ensure Tomcat uses the desired Java runtime (it may not be necessary to do anything if the environmental variables are already set to point to the correct version of Java). To specify the correct Java installation, edit the startup script (
startup.sh) found at
/path/to/tomcat/bin (in this specific example for the installation guide it is in
/app/apache-tomcat-9.0.12/bin) to point to Java installation directory by doing the following:
Add this line:
# ---------------------------------- # Start Script for the CATALINA Server # ---------------------------------- export JAVA_HOME=/app/jdk1.8.0_181/
It has also been observed that EMERSE requires a higher limit on the number of open files than what is typically the default on some Linux systems. This can be set using the
ulimit command as below prior to starting Tomcat, or added directly to the startup script
startup.sh. For example,
# ---------------------------------- # Start Script for the CATALINA Server # ---------------------------------- export JAVA_HOME=/app/jdk1.8.0_181/ ulimit -v unlimited
When the open file limit is set too low, an exception like the following will be thrown when Solr or EMERSE are started.
Caused by: java.io.IOException: Map failed at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:849) at org.apache.lucene.store.MMapDirectory.map(MMapDirectory.java:283) at org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:228) at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:195) at org.apache.lucene.codecs.compressing.CompressingTermVectorsReader.<init>(CompressingTermVectorsReader.java:118) at org.apache.lucene.codecs.compressing.CompressingTermVectorsFormat.vectorsReader(CompressingTermVectorsFormat.java:85) at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:132) at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:96) at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:63) at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:843) at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:53) at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:66)
Download the zip file from binary distributions under the "Core" section.
Unzip this file in a desired directory. This will become the Tomcat installed directory.
startup.bat file found under bin to point to the directory where the JDK was installed.
Since Oracle will be using port 8080, change the Tomcat port from its default (8080) to 8090. Go to the
tomcat/conf folder, and edit the
server.xml file. to change
server.xml file should be located here:
To edit the file:
The part that needs to be changed should look something like:
<Connector port="8090" protocol="HTTP/1.1" connectionTimout="20000" redirectPort="8443" />
To start the server use:
To stop the server use:
For this installation guide it would be:
You may need to use
EMERSE leverages Apache Solr for searching documents. See Required Software section for the specific version to use. The Solr version should be the exact version specified. Otherwise, the sample indexes provided may not be usable, and EMERSE may not function.
After downlodading/moving the file to the desired directory, unzip/untar the file similar to before. For example:
mv /home/emerse/Downloads/solr-7.3.1.tgz /app/
tar -zxvf solr-7.3.1.tgz
There is no need to download the source (
Make sure that Solr is using the correct JDK. It may not be necessary to do anything if the environmental variables are already set to point to the correct version of Java. Nevertheless, you can specify the exact path by editing the file:
solr.in.sh look for the line:
And change the line by removing the comment (#) character and specifying the exact path to the JDK, for example:
To start the server use:
To stop the server use:
In some cases you may need to use
EMERSE requires ActiveMQ to enable background processing of a number of internal tasks. Installation steps are very similar to Apache Tomcat. First, download the recommended file. See Required Software section.
Move the file to the desired directory:
mv /home/emerse/Downloads/apache-activemq-5.15.6-bin.tar.gz /app/
Unpack the ActiveMQ distribution:
tar -xvf apache-activemq-5.15.6-bin.tar.gz
Ensure ActiveMQ uses the desired Java runtime (it may not be necessary to do anything if the environmental variables are already set to point to the correct version of Java). To specify the JDK, do the following:
Modify the file
Add the line (near the beginning of the file, perhaps under the license):
Modify the file
activemq.bat located in
An Oracle database server is needed for EMERSE. For the purposes of getting started, Oracle makes available a free “Express Edition” (Oracle-XE) that is fully functional. This free edition of Oracle supports 1 core and up to 10 GB of disk space, which should be enough to support a few users in a demonstration version, or even light use in a production setting. Oracle express can be downloaded here: Oracle Express-Edition
Any of the available Oracle Database Editions should work with EMERSE. Michigan Medicine uses the Enterprise version in the production environment, but EMERSE does not depend on any Oracle components that would require additional licensing (e.g., replication, Oracle RAC, advanced compression, etc.)
|EMERSE was initially developed at Michigan Medicine for internal use, and Oracle was chosen because it was already licensed by the University. It is the only non-open source component used by EMERSE. It should theoretically be possible to change the database to an open source one, or to another database such as SQL Server, but we do not recommend it at this time due to the effort it would take. Further, this approach is currently untested and unsupported by the EMERSE team.|
Instructions for installing Oracle-XE can be found here:
Download the Linux Oracle Express edition, for example "Oracle Database Express Edition 11g Release 2 for Linux x64".
|You will need to accept the License agreement and have an account (free) set up with Oracle to download the file.|
Move the file to the desired directory:
mv /home/emerse/Downloads/oracle-xe-11.2.0-1.0.x86_64.rpm.zip /app/
unzip the file
It will unzip into a directory called
Run the executable to install the database:
rpm -ivh /app/Disk1/oracle-xe-11.2.0-1.0.x86_64.rpm
|If you encounter an error installing Oracle-XE related to Swap Space (e.g, "The system does not meet the minimum requirements for swap space. Based on the amount of physical memory available, Oracle Database 11g Express Edition requires 2048 MB of swap space…") please see the directions below, Creating a swap file.|
After the installation is complete, follow the directions to configure the application by issuing the following command:
Accept all of the defaults. For the password anything can be entered, but for this example it may be easiest to use
Point your browser to:
Ignore this section if you do not encounter an error related to installigng Oracle-XE
If you encounter an error installing Oracle-XE due to a lack of swap space (see warning, above), you can follow these Redhat Linux directions to increase the swap space on your machine, focusing specifically on the section called"Creating a Swap File". Those directions are briefly summarized below.
In Terminal type the following commands:
dd if=/dev/zero of=/swapfile bs=1024 count=2560000
chmod 0600 /swapfile
The additional swap memory should now be available. You can confirm this by typing in the Terminal:
While not necessary for EMERSE, it may be useful to install the Oracle SQL Developer tool, or another database tool of your liking (e.g., DbVisulizer). The following directions are for installing the Oracle SQL Developer tool. This tool requires the Java JDK so make sure Java is already installed. The SQL Developer software can be downloaded at:
Accept the license and download the Linux RPM. You can run the RPM installer from anywhere, since by default the software will be installed in the
rpm -Uhv sqldeveloper-220.127.116.11.1748-1.noarch.rpm
Run the setup script:
When prompted, enter the full path name of the Java JDK:
The SQL Developer App should then launch automatically. The installation should also create a shortcut to the app that can be reached via Applications → Programming → SQL Developer.
If launching does not work using the GUI, then go to Terminal and type:
We provide 4 indexes with the default distribution. Three of the indexes are populated with sample data to help with testing the initial setup and configuration, and the fourth is an index containing a list of words used for spell-checking, described below.
Two of the indexes (
patient-slave) store data about the patients (not real patients in the files that we distribute), and are meant to be populated from the
PATIENT table within the Oracle database. These indexes are required by EMERSE and the overall structure/definition of them should not be changed even for your own localized installation. The patient demographic data stored within them will automatically be replaced from the database tables when EMERSE is running. The two patient indexes are used to rapidly summarize details used for graphing the demographics.
A third index is called
documents and holds PubMed abstracts (containing no protected health information) as placeholders for sample documents. For your production system you can optionally rename this to something that is more suitable (you would then have to update the name in the Oracle table called
SOLR_INDEX, described in the Data Guide). Additionally, other aspects of the configuration for this document index will have to be changed from the included example to match your local institutional needs (e.g., the metadata types included for each document).
The fourth index is called
dictionary which holds words that are used by EMERSE for spell checking of terms entered by users. This
dictionary index is recommended but is not required. Additionally, no changes should need to be made to this dictionary and it can be used as-is. It is also worth pointing out that the dictionary is in an older Lucene format and not the newer Solr format. This should have no practical implications for use, except that it won’t show up in the standard Solr user interface even though the EMERSE application itself can access it.
Contact the EMERSE team for the download links to these four sample indexes.
Once the four indexes have been downloaded, move them to the default directory within Solr:
mv /home/emerse/Downloads/patient.zip /app/solr-7.3.1/server/solr/
mv /home/emerse/Downloads/patient-slave.zip /app/solr-7.3.1/server/solr/
mv /home/emerse/Downloads/documents.zip /app/solr-7.3.1/server/solr/
mv /home/emerse/Downloads/dictionary.zip /app/solr-7.3.1/server/solr/
Change to the
/app/solr-7.3.1/server/solr/ directory and unzip the three files:
For the demonstration installation the Solr indexes are placed in the default Solr directory. However, for a full production installation we would recommend that these indexes are located outside the Solr installation directory, perhaps on externally connected storage. To specify a directory outside of the default location, open the
If Solr is already running, stop it first:
Then restart it:
There may be times when you have several sets of indexes in various locations (e.g., for testing). In such a case it is possible to start up Solr and have it point to a set of indexes not located in the default location (or in the location specified by
Check to see if the three indexes (
documents) are listed under the Core Selector button:
Select each core to verify that the proper counts are there. For the
patient-slave cores, there should be 1750 "documents" (each representing a patient), and for the
documents core there should be a little over 125,000 documents.
Provided with the EMERSE distribution are a set of files, each containing SQL statements, that create all needed database objects and sample data that will allow the EMERSE application to startup with a default set of database objects, and sample data in the patients, research studies, synonyms and tables. These scripts should be run as the user and schema setup for the EMERSE application (this will be set by each implementing site), and not a system or sysdba user. We recommend a schema named
emerse. The account doesn’t require a DBA role but needs to be able to create database objects such as tables, indexes, sequences and views.
To set up the user/schema for EMERSE, do the following.
Launch SQL Developer.
In SQL Developer click on the green plus-sign icon to make a New Connection… or go to File → New… Database Connection.
Give any name to the Connection name, such as
Administrator Account. The username will be
system and the password will be the same password used when initially setting up Oracle. For the directions in the guide, the password
demouser was used. Press Test to test the connection (it should say Status: Success if it worked). Save the Connection, and then press Connect.
Now create the actual EMERSE application user for Oracle (this is for the EMERSE application, not a user of EMERSE). In the Query Builder window enter:
create user emerse identified by demouser default tablespace USERS quota unlimited on users; grant create session to emerse; grant create table to emerse; grant create sequence to emerse; grant create procedure to emerse;
And then click on the Run Script icon.
If you enter the wrong password to set up the
Create another connection: File → New… Database Connection.
The connection name can be anything, such as
EMERSE connection. For the username and password enter the information used to create the account based on the sql above. In this case, username is
emerse and password is
emerseapplication connection in Oracle
Test the connection by pressing Test, and you should see Status: Success. Check the box to Save Password and then press the Save button to save this connection. Then press Connect.
At this point you should be ready to run the SQL script that sets up the database for EMERSE and loads in sample data (but not the notes since the notes do not get loaded into the database).
Contact the EMERSE team for the download link to the SQL setup script.
Unzip the file, if necessary (the actual filename may differ between versions):
Execute the file in a SQL tool. For example, to run the script in SQL Developer go to File - > Open… and then select the SQL file. After loading the file it should open in a Worksheet tab. Then click on the Run Script icon and wait for the script to finish running. When running the Script it may show a prompt for Select Connection, in which case choose
EMERSE connection and then press OK.
|The script has about 140,000 lines so it will take a while to run.|
The next step in getting EMERSE up and running after initial installation of the application server and configuration of the database with default settings is to deploy the EMERSE WAR file.
Contact the EMERSE team for the download link to the EMERSE WAR file.
After downloading, deploy the file. First, rename the supplied war file to
emerse.war, then copy the war file to the
webapps directory of the Tomcat server. This can be done in one step:
mv /home/emerse/Downloads/emerse2-4.4.war /app/apache-tomcat-9.0.12/webapps/emerse.war
If Tomcat cat is not running, start it:
If Tomcat is using default settings, the WAR file will be exploded into a
number of files in a directory called
emerse. This directory includes all the files needed to run the application, including many of the application settings stored in the
WEB-INF/classes/project.properties file. For example, you may need to make a change to the settings file to reflect the database that will be used. If you have followed our directions for the example setup/installation you likely will not need to change any database connection information. However, you will need to change the pathname to the Solr indexes. Open the
project.properties file and look for the line starting with
lucene.indexPath. Change it to the location where the Solr indexes were copied, for example:
At this point the software part of the installation is complete, but some additional configuration may be required. Minimally, this would include the URL of the Oracle database if it is running on a separate server. Information on configuring EMERSE to locate the Oracle Server is located in the Configuration Guide
First, it would be a good idea to make sure all of the components (Tomcat, Solr, Active MQ) are restarted. Oracle should have been setup to start when the system started up so likely does not need to be launched separately.
To stop the main components:
To re-start the main components:
At this point EMERSE should come up, using the URL:
If the EMERSE application does not come up, the best place to troubleshoot any issues are in log files inside the Tomcat installation:
After all the required software is installed, one can verify the component wiring by running a diagnosis checklist through the EMERSE app. The check list page can be viewed using the URL:
http(s)://hostname<:port if applicable>/emerse/diagnostics.html
The following components are verified on a high level:
Database: Connection to the database is established and a query on sysdate is run.
Lucene: Checks the Lucene path in the configuration file and establishes a connection.
SOLR: Establishes a connection to Solr and runs a simple query.
Active MQ: Establishes a connection to Active MQ configured in the property files.
LDAP: Checks if the LDAP profile is provided and, if true, establishes a connection to the LDAP tree.
SOLR_DATE_FIELD: Checks whether documents in Solr have a populated field for
CLINICAL_DATE, as mapped in the the
MRN_IN_SOLR: Checks whether the MRNs in the PATIENT table exist in the Solr patient index. The intention is to catch a mis-match between the configured field and the populated one.
SOLR_SOURCE_FIELD: Makes sure documents in Solr have a
SOURCEfield, and that all values of that field are mapped in the
SOLR_SCHEMA: Checks to make sure the Solr schema XML files are not changed in ways that the software doesn’t expect.
The Diagnostics page can also show some errors extracted from the logs. However, if the hardened profile is active, the diagnostics page will not be able to show these error messages and it will be necessary to view it in the protected Admin page instead. This Admin page is described in the Administrator Guide.
To test the installation a bit futher, login and try a search:
Login at http://localhost:8090/emerse/. Login with user:
|The initial username and password is provided just to help you get started. To ensure security, it should be changed once you get everything set up.|
Choose anything to get past the Attestation page.
Enter "chest pain" in the Quick Terms Box and press Find Patients.
Click on the button Move patients to Temporary Patient List
Click the Highlight Documents button
Drill down into a cell on the Overview page to see a document with the term "chest pain" highlighted.
At this point, if everything worked, this is a very good indication that EMERSE is up and running well.
Once an initial implementation is complete, local customization work will need to be done including identifying document sources, proper indexing with metadata in Solr, and other configurations related to document display in EMERSE. These are detailed in other guides, such as the EMERSE Configuration and Optimization Guide and the EMERSE Data Guide.