EMERSE Developer/Admin Release Notes

Multiple front-end changes, although these will not be visible to the user.

Bundle (Saved Terms) sharing across the network is now possible.

Improved the admin tool’s system reload page. It should more reliably complete and show progress.

Improved reliability of the network, so that it can better recover from system downtimes. It now will retry connections to the query coordinator.

Synonyms feature is substantially improved

Networking feature is now available for use. This is turned off by default and must be configured locally to turn it on. Note that no additional software needs to be installed to participate in the network.

Updated system to handle longer usernames

Made LDAP authentication checked by default when adding a new user (and added a property for it)

Fixed a rare bug that could prevent demographics from being charted properly

Fixed a bug that would cause errors if a source name had spaces in it

New Term Bundles (Saved Term Lists) are now private by default (previously they had been public by default to encourage sharing)

Other minor bug fixes

Fixed a bug that prevented passwords with some special characters (e.g., '+') from working

Improved security with modifications to the Privileges (this will require a database change)

Admin App has a new Reports feature

Built a Report in the admin app to look for mismatches between patient MRNs in Solr versus the Patient database table

The Clarity UI framework has been updated

Fixed an issue that could occur when setting long passwords for Excel exports

This version is now database 'agnostic' and should be able to work with multiple types of databases including Oracle, MySQL, SQL Server, Maria DB, and more

The EDIT_PT_LIST privilege has been removed since it was not useful.

The VIEW_ALL_PT_COUNT privilege has been renamed to SEARCH_ALL_PT and the privilege has been tweaked to be more functional.

This version now includes a preliminary version of the networking component (which is off by default)

New user interface that leverages Angular Clarity

New version of Solr required

New Solr index settings

New Solr plugin required

Several changes to the database

ActiveMQ has been removed

Security improvements

Improved system performance

Removed the Analytics/Activity tracking since that is no longer available with the new user-interface updates.

Substantial overhaul of the user interface, which now uses the Angular Clarity Design System

ActiveMQ is no longer used for the standard EMERSE implementation

Multiple back-end performance improvements

Solr 8.x is now required (re-indexing will be required for upgrades from prior versions of EMERSE)

Indexing nor normalizes curly single quotes to straight single quotes, resulting in more consistent search results

Security improvement related to remote procedure calls, preventing arbitrary Java code from running

Additional minor security and efficiency fixes based on the output from the Open Web Application Security Project (OWASP) security tool and the Sonar tool

Fixed an issue identified by Sonar in which there was a hard-coded reference to an absolute pathname.

Fixed an inconsistent behavior in how a field would be visible within a single document and the Summary view (related to DISPLAY_FLAG).

Minor fixes based on Sonar scan to improve the efficiency of the code (e.g., use of Long.valueOf(long) instead of new Long(long) constructor.

Java 11 is now required

A link to the user guide is now available from the EMERSE menu (upper right corner of the application). The link is configurable within the project properties. A link to the Admin app is now also available from the same menu, visible only to users with admin privileges.

EMERSE will now work with Java 11. Java 8 is minimum required.

Improved the way in which Lucene indexes are re-opened. Background: Even as more documents are added to an index, Lucene is unable to recognize the newly added documents until the index is closed and then re-opened. Previously a small amount of custom code had to be written and deployed to handle this task. Now the task is built into EMERSE itself, and is configurable within the emerse.properties file.

The admin application has been upgraded to use Angular version 7.

Improved logging of searches for the Overview section of EMERSE. The actual query used is now stored in the logs (rather than a reference to the query stored elsewhere). The query is logged in the USER_SEARCH table in the SEARCH_QUERY column (this column was previously empty).

Improved checking of the system setup, by adding a new check in the diagnostics.html page. It will now also check to see if MRNs in the documents match the Solr MRNs to verify that they match up.

Several unused database tables/columns have been dropped, including:

Improved the way in which invalid logins are handled when logging in directly to the admin app.

Improved the way in which invalid logins are handled when logging in directly to the admin app.

Improved handling of EMERSE accounts to avoid an "unexpected error occurred" message that appeared when some users tried to login.

The user action of exporting a patient list is now recorded in the audit logs.

The dictionary used for spell checking has been updated. This is a Solr index called 'dictionary'.

EMERSE will now warn users if a patient MRN in a Saved Patient list is obsolete, meaning that it is no longer active in the main EHR system (which can occur if a patient has been assigned more than one MRN, and then the duplicate MRNs get merged into just one). An obsolete MRN is noted in the Patient database table with the patient.deleted flag, so this feature can only be leveraged if the Patient table is kept up to date.

Improved handling of the system starting up when LDAP is enabled. Previously if certain properties related to specific U of Michigan assets were not defined, the app would not start up.

Added a logging feature for analytics/activity tracking.

Fixed an issue related to a potential Javascript script attack vulnerability in bundle name/ description sections

Fixed a bug related to merging Venn diagrams that appears in certain browsers such as Safari

Upgraded the admin app to latest version of Angular

Additional Cross-Site Request Forgery (CSRF) protection added through the application

Handle any possibility of a javascript script attack in bundle name and possibly description

Improved handling of characters that can cause problems with file systems when a filename is generated for a Patient List export

Removing the leading zero when inserting a patient list is now configurable using a regular expression with the patientList.stripRegexOnMRNInsert property.

Updated dependencies that had potential vulnerabilities (e.g, jquery)

Additional minor security enhancements (set cookie to HTTP only, turn off auto-complete)

Added a new tab to the Admin UI that provides a way to force an internal synchronization. This normally happens once daily, but it may be needed more often during initial installation/setup.

Fixed a recently introduced bug where the 'Remove' button on the Attestation page was not working in some browsers

Improved handling of cached javascript files

Fixed a recently introduced bug that prevented the demographics charts from loading in certain situations

Updated Spring Security to version 5.0.6

EMERSE dependencies on Solr/Lucene jars are now at version 7.3.1

Improved timeouts by ensuring that the screen switches to the login screen after the system has timed out.

Fixed a bug where logging of a session attestation might sometimes be associated with a previously created user_session instead of the user_session created at the most recent login.

Fixed a bug related to mixed case wildcard searches with the Advanced Terms feature

Improved search performance

Improved the Diagnostics page, which is useful for installations: For example,

Upgraded Tomcat to version 9

Upgraded to Spring version 5

Cleaned up unused Oracle sequences and columns

Improved the ability to search metadata fields in the Advanced Terms feature

Minor bug fixes

Upgraded Solr from version 6.0.0 to 7.2.1

Upgraded the Spring Framework to 4.3.15

Improved the installation/upgrade process that uses the Liquibase output

Minor bug fixes (for example, removing a rare "null pointer" error)

Changed a database table name from release_info to environment_info. Two columns have been removed which were related to the version number and release date. This is so that the table can be modified once at initial setup to record the specific environment (prod, test, etc) without needing to be updated again.

The specific release version number is no longer in a database table but instead embedded in the WAR file distribution. This is to help ensure that the version number will be automatically distributed/updated with each software release.

Cron jobs for scheduling some events are now more easily configurable using the properties file.

The Administrator app now shows the logs for the current environment, which may be helpful when trouble-shooting.

Improved search performance for the Overview feature, by making changing to how the messaging broker handled requests from multiple users.

Fixed some javascript code that prevented some documents from being displayed in certain Chrome browsers.

The default font was changed to the Google font "Muli". This should be easier to read. This change was also made so that the font could be downloaded and included with the app to ensure greater consistency in the look of the app across browsers and operating systems.

Fixed an error on the administrator page that could occur when changing roles for a user that had multiple roles assigned to them

Changed the handling of the lucene.indexPath setting in the project.properties file so that the code can work correctly whether there is or is not a trailing slash on the pathname.

Added a new Log tab to the Admin page. This is currently a placeholder for functionality that will come later to help with reporting issues that may be recorded in the logs.

Improved server timeout handling that was preventing some users from obtaining results for time-intensive searches.

Added a new database table (release_info) to keep track of the version number (which is shown in the browser’s title bar and the EMERSE About menu), as well as information about the specific environment (dev, test, prod, etc).

Added a display option for the upper right of the EMERSE application that enables display of the specific environment that is active (dev, test, prod). This can be turned off for the production system with a database flag (using display_on_ui).

Removed help balloons from the application. Also removed the tables user_help_complete which kept track of which balloons were to be shown for each user as well as the table help_balloon which stored the actual content of the help balloons. These database table changes are incorporated in the SQL LiquiBase update script so should be removed automatically when updating from a prior version.

Various IE 11 browser fixes for compatibility with the Administrator Page

Improved functionality on the Administrator Page

Fixed a null pointer error that could occur if the patient Solr index was not synced up with the patients in the patient table or when the all patient search join query did not return any rows.

Improved the security for browser inactivity by forcing a logoff after X minutes.

Made the forced logout for inactivity configurable in the project.properties file. Two new properties are now defined: application.idle.timeout and application.warn.length.

Made contact/help information within the application (in the About menu and the Feedback menu) configurable. It will now just open a new e-mail within the user’s email client, but admins can define where the e-mail is sent in the project.properties file using the email property

Fixed bug to ensure that only users with an admin role can open the admin page

Improved reliability of ActiveMQ that was causing issues of system non-responsiveness with queries that Lucene could not handle. This involve adding JMX support and refactoring some of the Spring integration.

Improved messaging for users when Lucene encountered a query it could not handle

Various tables and column names have been renamed to better reflect their current purpose. Examples include changing document_index to document_source, renaming lucene_shards to SOLR_INDEX, and renaming columns in Document_fields: Lucene_name to schema_field_name.

Made removal of leading zeroes configurable when uploading patient medical record numbers. There is now a new property in emerse.properties: patientList.stripLeadingZerosOnMRNInsert. See 'EMERSE Configuration and Optimization Guide' for details.

Database build now based on Liquibase, which should make implementations as well as future upgrades easier to support.

Diagnostic page added to project for installation troubleshooting. The page is available to admins and can be found at diagnostics.html.

Changes were made to the project.properties file in order to make additional elements configurable, such as how MRNs are formatted.