Health Information Systems: MedSun Small Sample Survey Summary
MedSun: Newsletter #54, November 2010

Survey Topic: Health Information Systems
Year Conducted: 2009


Background
Computerized patient care systems are used by many health care facilities. Food and Drug Administration, Center for Devices and Radiological Health (CDRH) is interested in learning more about the experiences of hospital staff using these patient care systems, especially on the patient care side involving Computerized Physician Order Entry (CPOE). In order to understand the issues that hospitals have with these systems before, during and after implementation, a survey was conducted with health care professionals from five facilities. The focus of the survey was on the functionality of the systems and the topics discussed were the impact of the system, usability, user satisfaction, and problem resolution with systems that facilities have implemented. Another goal of the survey was to better understand any potential issues involving health information systems. Nine staff members representing several hospital departments were interviewed and the information that follows is a summary of the interviews.

Summary
Generally, the respondents had mixed feelings about their health information system’s usability. However, users did say that the impact of CPOE systems have changed workflow dramatically for the better. Documentation quality has improved and one respondent believed their medication errors have been reduced by 30%.

User dislikes include pop-up features because they can cause alert fatigue. Other dislikes include implementing complicated order sets and the fact that multiple applications have a different look and feel, even within the same vendor’s products. User likes include the way the system delivers decision support. They also liked the ability to use admission order sets, like medication adjudication, and have automated this process for patients’ discharge from the hospital.

Training for users ranged from no training for some staff to 24 hours for IT staff. Training time also depended on the role of the employee. Physicians generally received the least training, followed by nurses and other non-physicians. IT staff and other super-users received the most training. Also, there was usually more training for big software releases.

In general, experiences with system reliability have been good. Unplanned downtime is rare and normally very short with an estimated uptime close to 100%. When forced to return to paper orders during downtime they see more errors, including, incomplete orders, missing medication routes, and incorrect abbreviations. Staff realize how much safer CPOE is, versus a paper system. Experience with recovery from outages was also good.

Most respondents were unaware of any adverse event reports with a CPOE system. Most reports of system malfunction were related to human error. Types of other problems included those with system defaults and how patient data interfaces with the application.

Future plans for respondents’ health information systems included adding bar-coding of medications, expansion to the emergency room, and incorporating infusion pumps in a pharmacy management system. Respondents are also working to assure that their electronic medical records (EMR) are going to adhere to the evolving standards for the exchange of data with other facilities.

Small scale surveys are one of many tools the Agency uses to evaluate the public health impact of the potential problems associated with the use of medical devices. Because these surveys utilize a small sample size that is not statistically derived, the results provide only qualitative and not quantitative results. Additionally, FDA continues to receive adverse event reports from its Medical Device Reporting program, and will continue to make use of the literature and other published information. FDA scientific, medical, nursing and engineering staff are made aware of the survey results as needed. If FDA believes a possible public health problem may be emerging from the results of a survey, it will combine those results with data gained from the other sources and may convene an Ad Hoc group of clinical and manufacturing representatives to discuss further actions.

FDA will work with the manufacturers and health care professional organizations to make important information known about medical devices to the clinical community.


****************************************************
Small Sample Survey Questions

{Legend: questions asked to Clinicians = XX; IT staff = IT; Management = M}

1. Impact/Usability

Does the system meet the expectations of users (i.e.- does it support the workflows of clinicians)
•Does the system functionality match your requirements? (XX, IT, M)
•How is the system compatible with other systems/software/IT infrastructure? (IT)
•How well the system is compatible with your original processes/workflows? (XX, IT, M)
User likes and dislikes (Clinical Decisions)
•List top three areas of system that you like and dislike? (XX, IT, M)
•During installation and configuration does the clinician have the ability to override system constraints such as dose amount, frequency, etc(XX, IT)

2. Training

User training received on how to use of the system, before and after system deployment
•How much training was provided for use of the CPOE system (XX, IT, M)
•How would you rate the quality of the training received. On a scale of 1- 5 (with 5 being the highest) (XX, IT, M)
Training of IT staff and implementation teams related to deployment of the system
•Who installed and implemented your system? (i.e.- the hospital IT department vs. the manufacturer vs. 3rd-party consultant vs. other) (IT, M)
•Does the manufacturer verify/validate that the system is installed/implemented correctly? (IT, M)
•Does the manufacturer provide appropriate support/training to install/implement if done by hospital or hospital representatives? (IT, M)
•When the manufacturer installs an update/patch, do they validate/verify that the system continues to function as intended? (IT, M)
•Does the hospital or manufacturer validate/verify that other vendor products in use at the hospital (integrated or interfaced) are not adversely affected when their CPOE device is installed/implemented? (IT, M)

3. Reliability

How reliable are these systems (System reliability)
•Have you experienced any system outages and at what frequency? (XX, IT, M)
oDaily, weekly, monthly, yearly
•How long does these outages last? (XX, IT, M)
•Is recovery from outages stable or unstable? (?, IT, M)
What are the procedures in place during any system downtime, power outages, etc (System maintenance/upgrades)
•Is there a controlled policy in place for system upgrades/updates/maintenance software releases? (IT, M)
•What is the frequency of the updates? (XX, IT, M)
•What is the stability of the system after software maintenance is completed? (XX, IT, M)
ounstable, relatively unstable, no difference, relatively stable, stable
•Is there a validation procedure for updates before deploying the changes on the production system (on actual data)? (IT, M)
Disaster recovery/emergency plan
•How often does your data have to be retrieved from backup? (IT, M)
•What was the quality of system stability after data restoration from backup? (XX, IT, M)
•What policies and procedures are in place in event of long outages? (XX, IT, M)

4. Problem resolution

Categories/Types of problems encountered
•Have you encountered problems with your CPOE systems? (XX, IT, M)
•If so, what kind of problems have you faced? (XX, IT, M)
•Have you had any adverse events associated with these problems? (XX, IT, M)
•Have you reported any Medical Device Reports (MDRs) associated with these problems? (XX, IT, M)
Procedures for reporting problems encountered and improvement requests
•How long does it take to resolve these problems (XX, IT, M)
•Does your facility have the ability to request feature enhancements? (XX, IT, M)
•What procedures are followed feature enhancements request and implementation by the vendor? (XX, IT, M)

5. Other

Management oversight
•Is there an Audit process for workflow, data integrity and adherence to standards of the hospital (XX, IT, M)
•What are typical findings (most prevalent) (XX, IT, M)



MedSun Newsletters are available at www.fda.gov/cdrh/medsun