Search the OSCAR Documentation
< All Topics

Lab Download Integration

Share this

Preface

These are instructions on how to set up electronic data transfer (EDT) for downloading your lab reports in HL7 format from each individual major laboratory, and uploading them to your OSCAR EMR. This is the original method of getting electronic lab data to your EMR without the need for OntarioMD certification. You vendor may have other newer methods of downloading lab data such as Excelleris.

If you need an expert to assist you with automated Lab Download or support, please contact us.

Document Version History

  • v1.0 – Initial public release to oscarmanual.org – Jan 12, 2016
    © 2016 by Peter Hutten-Czapski MD under the Creative Commons Attribution-Share Alike 3.0 Unported License
  • v1.1 – Initial OSCAR 15 release – Mar 5, 2016
  • v2.0 – Ported over to worldemr.org – Oct 11, 2022 (KC Lai)
  • v2.1 – Added instructions on setting up lab download from lab vendors (KC Lai)
  • v2.2 – Updated extensively and documented keypair generator by Adrian Starzynski – Jun 11, 2024

Prerequisites

  • You are familiar with Windows Command line, dealing with source code and Java build tools, and administrative access to a server or Windows workstation.
  • You are familiar with how to setup a Windows virtual machine for the dedicated lab downloader. Or, you have a dedicated secure Windows or Mac workstation that can be kept on 24/7.
  • You have a static IP for your clinic system that runs the lab downloader. Some labs restrict lab download to known static IP’s of authorized clinics/physicians. You need to contact your internet provider and pay a little extra for a static IP. If you static IP changes (upgrade services or new provider), remember to update the labs of your new static IP.

Overview

As a physician, you may order labs and patients may go to one of three major laboratory vendors in Ontario: Lifelabs (CML/MDS), Dynacare (Gamma Dynacare Medical Laboratory), and AlphaLabs. There are smaller labs, but it may not be worthwhile to get set up with their lab download accounts (if they have any in the first place).

The general process is this: Each lab will have a SFTP service that you can download HL7 lab files (electronically coded lab data) to a folder on your dedicated lab download computer or virtual machine. An HL7 transport utility (Mule) will upload the HL7 lab files from the folders to your EMR via a secure channel with private-public keypair in OSCAR for each lab vendor. The EMR will then process the HL7 lab data and file to the corresponding patients, and will show up in your InBox for review.

These instructions are mainly for a Windows system, but you can try to adapt the instructions for Mac or Linux.

Get Electronic Data Transfer (EDT) accounts

LifeLabs

  • Contact a LifeLabs Client Products Specialist by emailing [email protected] and tell them that you are “a medical doctor that is self-hosting your own OSCAR EMR server” and that you need to be set up with the “LifeLabs Results Distribution System (LRDS)” and be given a “four digit route” and be given SFTP credentials (if possible) to download the labs.
  • You need to get the form “Electronic Results Request Form” to fill out and return to LifeLabs.
    • Vendor: specify your clinic name
    • Contact: specify yourself
    • PC/Server Operating System: Windows Pro  or Mac (unfortunately, there is no Linux version)
    • EMR Software Version: OSCAR McMaster / Community Edition v19 (or whatever you are using)
    • Switch (Winblast to SFTP): leave this blank
    • New Install: checkmark this
    • Client information: fill this section out with your physicians that need lab download to your system (you can add multiple physicians now or later to your account)
  • LifeLabs will ask you for your OSCAR EMR vendor, and you will have to reply to them again and clarify that you are your own self-hosted vendor and need to be able to download the HL7 files yourself to upload to your EMR. Remind them that you are a MD medical doctor (not some random IT person), that you do not have OLIS, and that you need to be able to download labs to do your doctor work.
  • LifeLabs will not actually give you your SFTP credentials, but will pre-package a WinSCP-type program (“proprietary SFTP client”) for you to install that has your unique ID and password and some custom scripts to be able to access their secure SFTP server to download HL7 lab files from your account to your computer. This is a rather ancient and cumbersome method, but LifeLabs has no incentive to pay developers to update this original EDT system when they have OLIS or Excelleris.
  • You may also need to provide LifeLabs with your static IP address from where you are downloading the lab to.
  • LifeLabs will send you a one time download link via a secure email. Do not click this link yet!  Proceed with this next steps in the document to set up your dedicated lab download workstation  or virtual machine, before you continue to set up the LifeLabs downloader. The software download link is a one-time access, and it may be time limited, so you can consider setting up your lab download system first before requesting the link. If the link expires, you can contact your LifeLabs Client Products Specialist to send it to you again.

Dynacare

  • Contact Gamma Dynacare Medical Laboratories and tell them that you are a medical doctor, and that you are requesting to be set up with an “eResults SFTP account to download HL7 lab files for importing to your EMR“. Dynacare likes physicians to use their web-based Contact Us form (choose the option for Healthcare Providers and Hospitals). They don’t have a specific general email to contact. But once you get the email of one specialist, you can build a rapport with them and they can be quite helpful.
  • If you are asked which system you are interested in being set up, tell them that you need the “Version 2.0 Batch Service (EMR) Users“. You can also request and account for the web based Online Transaction Service User (v 3.x), but this is more for downloading individual lab results for one patient at a time (not what you need for EMR).
  • You will be asked to complete and sign a Dynacare eResults – Terms of Use Agreement.
    • Organization / Clinic: your clinic name
    • User Name: give yourself a unique download username
    • Email: your email
    • Results to be: choose “Downloaded into your EMR”
      • EMR Provider: self-hosted
      • Name of software: OSCAR EMR
      • Phone: your phone for tech contact
  • You will be given a SFTP username, password, and the website of their SFTP server. Please keep this credentials private and confidential!
  • You may also need to give them your fixed IP address from where you will be doing the batch lab download from.

AlphaLabs

  • Contact AlphaLabs and ask them for an SFTP lab download account.
  • You will be given an SFTP username, password, and the website of their SFTP server. Please keep this credentials private and confidential!

Create a Virtual Machine to download and process labs

Although you can set up any Windows PC or a Mac to be the lab downloader, we would advise that you set up a dedicate machine for this task. This computer would need to be secured in a physical location, and would need to be running 24/7.

An even better solution, would be to set up virtual machine on your server. Follow the same instructions on setting up a virtual machine for OSCAR, but instead of installing OSCAR, install your own copy of Windows Pro. This lab download method has been tested to work with Windows XP Pro. We recommend Pro versions because it allows you to remote desktop to the Windows and manage the downloader files easily. It has not been tested to work with newer Windows 7 / 10. For this reason, we would recommend dedicating a virtual machine to do the lab download, and not allow anything else or anyone else to use this machine to limit security vulnerabilities.

  1. Set up a Windows workstation or virtual machine to be the dedicated lab downloader.
  2. If you don’t know how to set up the downloaders to run as a Service, you could set up Windows to automatically log-in to an account on startup, and then immediately Lock Screen (to switch out but stay logged in)

Setup the Lab Downloaders

LifeLabs

  • In your Windows or Mac dedicated lab download workstation or virtual machine, use the link in the LifeLabs email to download the LRDS software.
  • Install the LRDS software on the machine.
  • The installer will create the following folders
    • C:\lifelabs
    • C:\DATA\LIFELABS
  • The program to run to download labs is “C:\lifelabs\HL7Main.exe”
  • Create a “Scheduled Task” in Windows or Mac to regularly run this program “C:\lifelabs\HL7Main.exe” every hour (or however often you want to check for new labs). Remember to allow this Task to run as administrator.
  • The HL7 files will be dropped in to the folder “C:\DATA\LIFELABS”
  • The LifeLabs downloader will check for new labs, download the HL7 files, and then generate their “CURHST” file which they use to validate whether a lab was successfully downloaded before deleting it off their server.

Additional Info for LifeLabs download:

The LifeLabs client software they provide has the IP address along with the keys. You have to have it generate a CURHST.0 ack file. Here’s the script for generating this file: Perl script for responding to the downloaded LifeLabs report

Dynacare

  • Create a directory “C:\DATA\GDML” to store your log files and incoming HL7 files from Dynacare
  • Create a directory “C:\home\GammaDynacareScript” to hold the custom lab download scripts
  • Use the custom Python script (created by KC Lai) to automatically download from Dynacare.
  • Edit the batch file and the Python script to match your instance (Dynacare SFTP username and password)
    • user=”username”
    • pw=”password”
  • Create a “Scheduled Task” in Windows or Mac to regularly run KC Lai’s batch file (for logging the execution of Ian Pun’s Python script) on an hourly schedule.

AlphaLabs

  • Create a directory “C:\DATA\ALPHA” for log files and incoming HL7 lab files from AlphaLabs
  • Create a directory “C:\home\AlphaLabScript” for the custom lab download script
  • Install WinSCP so you can run a custom download script from SFTP servers
  • Open WinSCP and make an FTP connection to AlphaLabs SFTP server with your username and password (given by AlphaLabs)
  • Copy down the SSH host key fingerprint to the authorized AlphaLab’s SFTP server. You will need this to put in to the custom script to automatically connect without manually confirming an authorized server each time the script runs.
  • Create a WinSCP script in “C:\home\AlphaLabScript\AlphaLabDownloadScript.txt” with the following text (be sure to edit the username, password, domain name of the SFTP server, and the hostkey fingerprint details to suit your instance). This text file must be kept secure (as it contains plaintext passwords) and should not be accessible by anyone except yourself or a trusted IT person.
# Connect to AlphaLabs and Download labs
# by Kevin Lai (2-July-2019)
# Accept any hostkey:
# open -hostkey=* sftp://username:[email protected]
# Accept verified hostkey:
open -hostkey="ssh-rsa 1024 e3:4f:5h:63:g3:5h:6j:6g:2g:k8" sftp://username:[email protected]
# Change directory
cd /outbox
# Download files and delete from source when downloaded
get -delete "*.*" "c:\DATA\Alpha\Alpha%TIMESTAMP#yyyymmddhhnnss%-*.*"
# Download files but leave on server
#get "*.*" "c:\DATA\Alpha\Alpha%TIMESTAMP#yyyymmddhhnnss%-*.*"
# Logoff
close
# Quit WinSCP
exit
  • Create a Windows batch file and save it to “C:\home\AlphaLabScript\AlphaLabDownloadBatch.bat” with the following text:
REM Batch file for downloading AlphaLabs HL7 labs
REM by Kevin Lai (2019)
REM Logs output with timestamp
echo off
REM Determining the datestamp for backup folder:
for /f "tokens=2 delims==" %%a in ('wmic OS Get localdatetime /value') do set "dt=%%a"
set "YY=%dt:~2,2%" & set "YYYY=%dt:~0,4%" & set "MM=%dt:~4,2%" & set "DD=%dt:~6,2%"
set "HH=%dt:~8,2%" & set "Min=%dt:~10,2%" & set "Sec=%dt:~12,2%"
set "datestamp=%YYYY%%MM%%DD%" & set "timestamp=%HH%:%Min%:%Sec%"
set "fullstamp=%YYYY%-%MM%-%DD%_%HH%%Min%%Sec%"
echo on
echo datestamp: "%datestamp%"
echo timestamp: "%timestamp%"
echo fullstamp: "%fullstamp%"
echo Running AlphaLabDownloadBatch.bat on... "%fullstamp%" >> c:\DATA\Alpha\Alpha-log.txt
"c:\Program Files (x86)\WinSCP\winscp.com" /ini=nul /script=AlphaLabDownloadScript.txt /log=winscp-log.txt /loglevel=0 >> c:\DATA\Alpha\Alpha-log.txt
  • Create a “Scheduled Task” in Windows or Mac to regularly run KC Lai’s script “C:\home\AlphaLabScript\AlphaLabDownloadBatch.bat” on an hourly schedule.

Installing Mule Uploader to EMR

Although you can now take the HL7 lab files download from the above scripts found in “C:\DATA\…” manually upload them in to your OSCAR EMR via “InBox -> HL7 Lab Upload” and select the correct Lab type depending on which lab vendor the file is from:

  • ALPHA is AlphaLabs
  • GDML is Dynacare
  • MDS/LifeLabs is LifeLabs

We recommend you install Mule which automatically takes the HL7 files in a specific directory and uploads it to your EMR. You may need to adapt the instructions for your situation (ie. version of Java or Maven, 32bit or 64bit installer etc)

Labwork can be pushed into OSCAR with the HL7 transport utility from a Windows workstation. Use similar instructions to install on a Linux machine. Alternatively, the SOAP web service LabUploadService can be used to send contents of HL7 files.

Installing the Infrastructure Packages

Java

The version below has been tested, you can use a newer one, just ensure that it is the same type. The wrapper that starts this Mule needs a 32 bit Java.

The following example uses the file pattern for 32 bit Oracle Sun Java 8 update 66.

jdk-8u66-windows-i586.exe

The typical windows installation process will place the executable files in c:\Program Files (x86)\Java\jdk1.8.0_66\jre

Maven

Maven is tool that you will use to compile a java JAR file configured for your system. Download the latest maven from apache. It will be in a bin tar which you need to extract using either a command line tool such as TarTool or a GUI extractor such as 7-zip. Extract to an appropriate location such as Program Files.

TarTool apache-maven-3.3.9-bin.tar.gz "C:\Program Files\maven-3.3.9"

The maven package is thus extracted into a subdirectory.

Git

Git is a software configuration management (SCM) tool that you can use to retrieve the source files for this program from the repository.

You can download either the 32bit or 64bit installer from https://git-scm.com/download/win

Then use Git to clone the source files into an appropriate location.

mkdir C:\home
cd C:\home
git clone https://git.code.sf.net/p/oscarmcmaster/hl7_file_management oscarmcmaster-hl7_file_management

Or you can download and extract the ZIP file: oscarmcmaster-hl7_file_management

Configuration of Packages

Environmental Variables

You should set some system variables so that Windows knows where to find the various components

  1. Open the ‘System’ control in the ‘Control Panel’
  2. Under the ‘Advanced’ tab click the ‘Environment Variables’ button
  3. In the ‘System variables’ window click the ‘New…’ button
    1. Variable Name: MULE_HOME
    2. Variable Value: C:\home\oscarmcmaster-hl7_file_management\mule-1.3.3
    3. Click ‘OK’
  4. In the ‘System variables’ window click the ‘New…’ button
    1. Variable Name: MAVEN_HOME
    2. Variable Value: C:\Program Files\maven-3.3.9
    3. Click ‘OK’
  5. In the ‘System variables’ window highlight the ‘Path’ variable and click the ‘Edit…’ button
    1. add ‘;%MULE_HOME%\bin;%MAVEN_HOME%\bin’ to the end of the ‘Variable Value’
    2. Click ‘OK’
  6. Click ‘OK’

OSCAR Keys

Key Pair Generator

Log into the OSCAR server and go to Administration>System Management>Key Pair Generator.

Input ‘oscar’ without the quotes as the service name and press the ‘Create Key Pair’ button. If you received the error: “Failed: The oscar key pair has already been created”. It is ok to ignore it.

About OSCAR Lab Keys

For lab services to communicate with OSCAR via HL7, OSCAR requires matching keys for security. You first need to generate a key for OSCAR itself (call it ‘oscar’), then you generate the other keys.

The public/private key headings might be a little misleading right now. The Oscar key is one shared key which authenticates who the server is to other people, that’s why there’s only one Oscar key (Oscar server is always the legitimate server to everyone). The “private keys” are service keys, which are keys specific to each service. So if you need 5 different lab services you have 5 different service keys so you can control access to each service individually (ex. disable a single connection).

The Oscar key is stored in the oscarKeys database table, while the service keys are stored in the publicKeys table.

You can generate the Oscar key manually using RSA Key Generator tool and Remove Line Breaks From Text tool to remove the line breaks, and place into the oscarKeys table.

Service Keys

After you have generated a public key for Oscar, you can create a new service key using the Key Manager.

Note: if Oscar doesn’t have a public key and you try to generate a service key, you will get the error ‘Failed: Could not retrieve public key from oscar’.

Create Key Pair

Input your service as the service name and select the message type from the drop down menu and click the ‘Create Key Pair’ button. A window will pop up asking you to save ‘keyPair.key’. Remember the location where you saved it. (ie. “C:\home\oscar-keypairs\keypair_lifelabs.key” or “C:\home\oscar-keypairs\keypair_gdml.key”)

If you wish to use multiple message formats such as MDS / Lifelabs, Dynacare (GDML) and AlphaLabs, you will need a different service key pair for each message format.

Note: The keypair file saved contains line feeds code (hidden CR/LF) which must be removed for Mule to read the keys correctly. In other words, you need to delete the hidden return/line feed that separates each line of the key so that the key is on one single line with no word wrap. Use an editor such as Notepad++ to edit the keyfile and can save in Linux format and also show you the hidden CR/LF line feeds. The default Windows Notepad does not save well in Linux format and so the keyfile may fail.

Configure Properties File

Edit the file ‘LabProperties.properties’ in the ‘src/main/resources’ directory to use your own directories. If the directories do not exist they will be created at runtime. Also make sure to include the address of the OSCAR server and the location of ‘keyPair.key’ that you saved in the previous step. Instructions are included at the top of the file, example settings are listed below. Note the UNIX style forward slash usage “c:/” for directory paths.

Example settings for a Windows system downloading from two labs (LifeLabs and Dynacare)

incomingHL7dir1 = c:/DATA/LIFELABS
incomingHL7dir2 = c:/DATA/GDML
incomingHL7dir3 = 
errorHL7dir = c:/home/messages/errorHL7dir
completedHL7dir = c:/home/messages/completedHL7dir

keyLocation1 = c:/home/oscar-keypairs/keypair_lifelabs.key
keyLocation2 = c:/home/oscar-keypairs/keypair_gdml.key 
keyLocation3 =

oscarURL = https://your.oscaremr.com:8443/oscar/lab/newLabUpload.do
# smtpServer for notifying errors (optional)
smtpServer = smtp.server.com:25
senderEmailAddress = [email protected]
recipientEmailAddress = [email protected]

*Note the oscarURL is from OSCAR>Administration>System Management>Key Pair Generator>Lab Upload url (you may need to change the server name/port to be accessible from the Windows machine)

Configure Config File

Depending on how many different message formats you wish to use you will have to use a different config file. If you are downloading just one lab then you can leave mule-config.xml as is. Otherwise use the template for the number of message formats (eg. mule-config_two-formats.xml for when you are downloading from Lifelabs and Dynacare), and save to ‘src/main/resources/mule-config.xml’. You will not usually need to edit the source template.

Build the Package

Open the command line by selecting ‘Run…’ from the start menu and typing in ‘cmd’ and clicking ‘OK’

Navigate to the ‘oscarmcmaster-hl7_file_management’ directory within the command line.

Set JAVA_HOME to point to the JDK using the following command ( be sure the directory is correct ):

set JAVA_HOME=c:\Program Files (x86)\Java\jdk1.8.0_66\jre

Use maven to build the source:

mvn package

copy the resultant jar to the mule library (the existing jar in source needs to be overwritten):

copy target/IncomingHL7Management-1.0-SNAPSHOT.jar mule-1.3.3/lib/user/IncomingHL7Management-1.0-SNAPSHOT.jar

Leave the command line open for the next step.

Start Mule

Type the following into the command line:

mule install

Mule is now installed as a Windows service, it will start when windows boots. It can be started or stopped using the following commands:

mule start
mule stop

Mule can be removed from being a service with the following command:

mule remove

Changing the Mule configuration

If you want to add a third lab or change the config file, remember to stop Mule, remove Mule as a service; and then go back to editing the properties file, generating the Java package, and reinstalling the updated Mule.

mule stopmule remove

Other Notes

Original instruction for Importing HL7 Labs by Peter Hutten-Czapski initially posted to oscarmanual.org
Supplementary instructions from Marc Dumontier that might be helpful as well.

Credits

  • Dr. Peter Hutten-Czapski
  • Marc Dumontier
  • Dr. Kevin Lai
  • Dr. Ian Pun
Table of Contents