Patients admitted to inpatient facilities are at risk for acute physiologic deterioration. This can lead to prolonged hospitalization, admission to the ICU, and even cardiorespiratory arrest (1,2). Worsening in a patient’s clinical condition often remains undetected for hours prior to escalation of care (1). Attempts at recognizing deterioration early have been developed and range from simple alerts based on vital sign alterations to trend analysis and complex early warning scores (EWS) (3). These efforts have been combined with multidisciplinary rapid response teams (RRTs) aimed at timely intervention and prevention of cardiorespiratory arrest (2). Owing partly to limitations of the aggregate risk scores and partly to variable RRT availability and composition, these efforts have not led to consistent improvements in outcomes (4).
Health information technology (HIT) is broadly defined as the incorporation of various information sources, data, and technology to facilitate improved communication and decision-making (5). The widespread implementation of electronic medical records (EMRs) has allowed access to larger quantities of clinical data and utilization of prediction analytics (6). EMR-based alarms have emerged to support timely detection of acute conditions such as sepsis, acute kidney injury (AKI), and respiratory failure (7–9). Digital clinical decision support has also been created to help standardize the approach and management of deteriorating patients (10).
Although a recent meta-analysis reported EMR improved patient safety by reducing medication errors and adverse drug reactions, that study did not reveal any improvement in mortality (11). Another meta-analysis focused on a broad range of HIT in the inpatient setting and did not demonstrate reduction in hospital mortality or length of stay (LOS) either (12).
Much work has been done to develop systems identifying actionable deterioration when a patient may benefit from early attention and action from clinicians. However, patient outcomes from HIT supporting early detection of patients with actionable worsening conditions remain unknown. The objective of this systematic review (SR) and meta-analysis was to evaluate the impact of HIT for early detection of patient deterioration on patient mortality and LOS in the acute care hospital setting. This systematic evaluation may help clinicians and institutions to make informed decisions about the utilization and implementation of HIT within process and workflow systems across acute care clinical settings.
MATERIALS AND METHODS
The results of the study were reported using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statements (13) (Supplemental Table 1, https://links.lww.com/CCM/H90). The Covidence software (Veritas Health Innovation, Melbourne, Australia) was used for data collection (14).
Data Sources and Search Strategy
A comprehensive search of several databases from 1990, when robust information technology infrastructure in hospitals became more widespread, to January 19, 2021, was conducted. The databases included MEDLINE and Epub Ahead of Print, In-Process & Other Non-Indexed Citations and Daily, Embase, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, and Scopus. The search strategy was designed and conducted by an experienced librarian with input from study investigators. Controlled vocabulary supplemented with keywords was used to search for studies of interest. The actual strategy listing all search terms used and how they were combined is available in Supplemental Table 2 (https://links.lww.com/CCM/H91). The additional sources included gray literature search and reference mining.
Study Selection
We included studies that enrolled patients hospitalized on inpatient floors, in ICU, or evaluated in the emergency department (ED). Eligible studies assessed HIT for early detection of and notification about patients experiencing deterioration or at high risk of deterioration, as an intervention. Comparison groups received usual care in the same study settings. Eligible studies reported at least one end point of interest: hospital LOS, ICU LOS, or mortality at any time point.
We excluded studies that used an HIT intervention not detecting deterioration, and developed or validated an HIT intervention only without implementation into practice.
Titles, abstracts, and full texts of identified studies were independently reviewed by pairs of reviewers (S.H., K.L., Y.P., H.L., A.T., A.K.B.) using prespecified eligibility criteria. Disagreements were resolved by a third reviewer (S.H., V.H.) or through group discussion to reach consensus.
Data Extraction
Study details of included articles were abstracted by two independent reviewers (K.L., Y.P., H.L., A.T.) using a standardized data extraction form. Additional reviewers (S.H., A.K.B.) resolved disagreements. Data abstracted included study timeline, setting, population and size, intervention description, and outcomes (Supplemental Appendix 1, https://links.lww.com/CCM/H92).
Outcome Measures
The primary outcome was difference in hospital mortality between the intervention and comparison groups. The secondary outcomes were hospital LOS, ICU LOS, ICU mortality, and mortality at other commonly reported time points. All outcomes were prespecified.
Data Synthesis and Analysis
When possible, we extracted or calculated the odds ratios (OR) and corresponding 95% CIs for binary outcomes (mortality). We used adjusted OR when available. For continuous outcomes (LOS), we calculated mean differences (MDs) using 95% CIs.
The DerSimonian and Laird random effect method was used for quantitative synthesis of data when at least three eligible articles included the desired outcome. Meta-analyses were performed separately for randomized controlled trials (RCTs) and for pre-post studies.
We evaluated heterogeneity between studies using the I2 statistics. I2 0–30% was categorized as low heterogeneity, 31–60% as moderate, and greater than 60% as substantial heterogeneity (15).
To explore potential sources of heterogeneity, we conducted predetermined subgroup analyses based on the study setting (ED, hospital floor, or ICU), type of patient deterioration identified by HIT (sepsis, AKI, and others), and risk of bias (ROB).
We also conducted post hoc analysis of RCTs to assess possible changes in the cumulative evidence about the effect of HIT on hospital mortality over time. Sensitivity analyses were conducted to assess robustness of the synthesized results. Analyses were performed using OpenMeta Analyst—an open-source, cross-platform software for advanced meta-analysis (16). Two-tailed p value of less than 0.05 was considered statistically significant.
Risk of Bias/Quality Assessment
The ROB was assessed by Drs. Herasevich and Herasevich using the Revised Cochrane ROB tool for randomized trials (17) and The Risk Of Bias In Nonrandomized Studies—of Interventions assessment tool (18).
We evaluated the strength of evidence using Grading of Recommendations Assessment, Development, and Evaluation approach (19). Per standard grading, evaluation of RCTs was initially considered as high quality of evidence and observational studies as low quality of evidence. To assess potential modifying factors affecting the strength of evidence, we evaluated methodological limitations of included studies, precision, directness, consistency, and publication bias (19).
RESULTS
Study Selection
The search strategy identified 2,767 studies with 44 additional studies identified through additional searches (20). After removing duplicates, 2,810 papers were screened using titles/abstracts. Following screening, 2,552 abstracts were removed, and 258 papers remained for full-text review. Among the final set of 30 studies, 21 contained quantitative data and were included in the meta-analyses for one or more outcomes. See PRISMA diagram (Fig. 1) for study selection, phases, and reasons for exclusion.
Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram.
Eligible Studies and Participant Characteristics
Supplemental Table 3 (https://links.lww.com/CCM/H93) summarizes the characteristics of the 30 eligible studies. Most of the studies were conducted in the United States, six were conducted in Europe, and two in Asia. Twenty-three studies were single center, and seven were multicenter.
Eighteen studies were based on hospital floors (6,10,21–36) and five in ICU (37–41). Two studies examined HIT implementation in the ED (42,43) and then analyzed outcomes among those hospitalized following ED presentation. Five studies were based on both ICU and the hospital floor (44–48).
Seven studies were RCTs, including two cluster-randomized trials (22,41) and five individually randomized trials (29,39,46–48). Twenty-three studies used a pre- and a postimplementation design (6,10,21,23–28,30–38,40,42–45).
Seven studies evaluated HIT for detection of AKI (21,32,37,38,46–48), 10 were designed for early detection of sepsis or systemic inflammatory response syndrome (10,24,25,27,33,35,39,42–44), and the remaining 13 studies for other types of deterioration (6,22,23,26,28–31,34,36,40,41,45) such as respiratory or other physiologic deterioration. There was a lack of uniformity in how deterioration was quantified with some investigators using scores or criteria for clinical syndromes and some using changes in vital signs, but all used robust approaches to define deterioration (Supplemental Table 4, https://links.lww.com/CCM/H94).
Baseline characteristics in intervention and comparison groups in included studies were similar. The median study duration was 1.5 years with wide variation from 2.5 months to 12 years.
Outcome Measures
Some studies assessed outcomes of interest among the entire study cohort, whereas other studies only assessed the outcomes among those patients meeting the criteria for deterioration both in intervention and comparison groups. Thus, we conducted two types of meta-analyses: one evaluating the mortality and hospital LOS for all included study patients (entire study cohort) and one evaluating only those patients who reached the alert threshold defined for each study and, therefore, detectable by the HIT.
All outcomes for eligible studies are summarized in Supplemental Table 4 (https://links.lww.com/CCM/H94). However, we limited our analysis to the primary and secondary outcomes described above. We conducted separate meta-analyses for RCTs and pre-post studies for each outcome.
Risk of Bias/Quality Appraisal
Among the RCTs, the overall ROB was low or moderate for most studies due to lack of blinding among clinicians and outcome assessors (Supplemental Table 5, https://links.lww.com/CCM/H95). In the pre-post studies, ROB was moderate or high for most studies due to potential confounding and incomplete reporting of study results (Supplemental Table 6, https://links.lww.com/CCM/H96).
Pooled effect size and quality of evidence for hospital mortality and LOS are reported in Supplemental Table 7 (https://links.lww.com/CCM/H97). The quality of evidence of included studies was low due to methodological limitations, inconsistency, and imprecision.
Mortality
All included studies assessed mortality as an outcome, although at different time points.
Hospital Mortality.
Twenty-eight of the 30 studies (6,10,21–27,29–45,47,48) reported hospital mortality. Sixteen studies assessing hospital mortality were evaluated in the meta-analyses. Of these, 11 (6,30,31,34,37,39–41,43,45,47) reported hospital mortality for the entire study cohort, two studies (33,42) reported the outcome only for those patients meeting deterioration criteria, and three (21,22,35) reported both.
Entire Cohort.
In the meta-analysis of four RCTs, the implementation of HIT for early detection of patient deterioration was not associated with a significant decrease in hospital mortality (OR, 0.99 [95% CI, 0.80–1.21]) (Fig. 2).
Meta-analyses on hospital mortality in patients who received the intervention (Health Information Technology for early detection of deterioration) compared with usual care. Entire study cohort. A, Randomized controlled trials. B, Nonrandomized (pre-post) studies. C, Sensitivity analysis of the pre-post studies. The size of the data markers represents the weight each study has in the pooled result.
Heterogeneity within this subset of studies was moderate and can be partially explained by the difference in types of deterioration detected by HIT.
The meta-analysis of 10 pre-post studies demonstrated a significant association between the use of HIT and improved mortality (OR, 0.78 [95% CI, 0.70–0.87]) (Fig. 2). The heterogeneity was moderate in this group and may be attributed to the difference in types of deterioration detected (Supplemental Fig. 1, https://links.lww.com/CCM/H98; legend, https://links.lww.com/CCM/H92). Sensitivity analysis demonstrated the stability of the pooled effect size and only a marginal improvement in heterogeneity (Fig. 2).
Study Participants Meeting Criteria for Deterioration.
Implementation of HIT was not associated with a statistically significant decrease in hospital mortality in three RCTs (22,29,48). Meta-analysis was not performed as one study did not include sufficient data.
Meta-analysis of five pre-post studies demonstrated a significant association between HIT and a decrease in hospital mortality (OR, 0.92 [95% CI, 0.87–0.97]) (Fig. 3). The heterogeneity within this subset was low.
Meta-analysis on hospital mortality in patients who met the criteria for deterioration among those who received the intervention (Health Information Technology for early detection of deterioration) compared with usual care. Pre-post studies. The size of the data markers represents the weight each study has in the pooled result.
Additional mortality outcomes are reported in Supplemental Appendix 1 (https://links.lww.com/CCM/H92).
Hospital LOS
Twenty-three of the 30 included studies assessed hospital LOS as an outcome (6,10,21–23,25–31,33–37,39–41,44,46,47). Sixteen studies included quantitative data for evaluation in the meta-analysis. Of these, 11 (26,27,30,31,34,37,39–41,46,47) reported the hospital LOS for the entire study cohort, three studies (23,29,33) reported the hospital LOS only for those patients who met the criteria for deterioration, and four (21,22,25,35) reported both.
Entire Cohort
In the meta-analysis of five RCTs, no significant difference in hospital LOS was found (MD, 0.10 [95% CI, –0.07 to 0.27]) (Fig. 4). The heterogeneity was low in this group of studies.
Meta-analyses on hospital length of stay in patients who received the intervention (Health Information Technology for early detection of deterioration) compared with usual care. Entire study cohort. A, Randomized controlled trials. B, Nonrandomized (pre-post) studies. C, Sensitivity analysis of the pre-post studies. The size of the data markers represents the weight each study has in the pooled result.
Meta-analysis of 10 pre-post studies demonstrated significant association of HIT with reduced LOS (MD, –0.29 [95% CI, –0.51 to –0.07]) (Fig. 4). However, the heterogeneity in this set of studies was substantial and could not be fully explained by difference in ROB, study settings, or types of detected deterioration (Supplemental Fig. 2, https://links.lww.com/CCM/H99; legend, https://links.lww.com/CCM/H92). One obvious outlier, Olchanski et al (40), compared two cohorts with time difference in 4 years, and its results were likely affected by the practice changes over time. Sensitivity analysis showed that following removal of this study, no significant association between HIT and improvement in hospital LOS was demonstrated (MD, –0.15 [95% CI, –0.33 to 0.03]) (Fig. 4).
Study Participants Meeting Criteria for Deterioration.
Two RCTs evaluating hospital LOS among patients meeting criteria for deterioration (22,29) did not demonstrate significant improvement in LOS.
However, in the meta-analysis of four pre-post studies, HIT implementation was associated with a significant reduction in hospital LOS (MD, –0.29 [95% CI, –0.48 to –0.11]) (Fig. 5).
Meta-analysis on hospital length of stay in patients who met the criteria for deterioration among those who received the intervention (Health Information Technology for early detection of deterioration) compared with usual care. Pre-post studies. The size of the data markers represents the weight each study has in the pooled result.
Additional LOS outcomes are reported in Supplemental Appendix 1 (https://links.lww.com/CCM/H92), Supplemental Figure 3 (https://links.lww.com/CCM/H100; legend, https://links.lww.com/CCM/H92), and Supplemental Figure 4 (https://links.lww.com/CCM/H101; legend, https://links.lww.com/CCM/H92).
DISCUSSION
In this SR and meta-analyses, we evaluated the impact of HIT for early detection of patient physiologic deterioration on hospital mortality and LOS. We included 30 studies assessing patients in acute care hospital settings. There was variability in setting, interventions, type of deterioration detected, and outcome measurement approaches. We conducted multiple analyses to compare similar study designs and groups with similar outcome approaches (studies reporting outcomes for entire study cohorts and only for patients meeting deterioration criteria).
We found that HIT for early detection of patient deterioration was not associated with a reduction in hospital mortality or LOS in the RCTs and associated meta-analyses. In the meta-analyses of pre-post studies, HIT intervention was significantly associated with improved hospital mortality and hospital LOS. ICU LOS did not change significantly with HIT interventions. LOS can be a challenging outcome measure due to competing risk of mortality and the potential of including those with a short survival time who have died (49).
There have been several SRs and meta-analyses exploring the impacts of HIT on patient outcomes. However, those studies differed from our study in several ways. The study by Varghese et al (50) focused on computerized decision support system (DSS) implementations and found positive but not clinically important improvements in patient outcomes. That study noted a lack of rigorous RCTs to assess clinical decision support. The SR by Despins (51) focused on detection of sepsis only and noted that current performance variability affected the impact on patient outcomes. Two more SRs have focused on a broad range of HIT including EMR, DSS, computerized physician order entry, and surveillance systems (“sniffers”) and have not demonstrated improvements in hospital mortality or LOS (11,12). In contrast to other SRs, we focused on the subset of HIT specifically designed for early detection of deterioration that had been implemented in acute care settings.
A notable finding of our work overall is the difference between the conclusions of the RCTs and the pre-post studies. HIT implementation was not associated with improvements in hospital mortality in the RCTs that are considered the gold standard of research and a rigorous approach to avoid confounding (52). The studies supporting the use of HIT were generally pre-post studies, and the conclusions from these studies need to be considered carefully due to the high probability of confounding outlined below.
We identified several categories of potential cofounders that may have played an important role in the improved outcomes in the pre-post studies in our SR. These were: 1) training and education of staff (6,37), 2) broad quality improvement projects in which the HIT was just one component (10,23,43), 3) change management assessments and general improvements over time (30,35,40,45), 4) complex multicomponent or multifaceted interventions that also included DSS and dashboards (40,45), and 5) the Hawthorne effect (10,23,34,37,53).
Although studies often reported that there were no known significant changes in the clinical practice during the study period, they were likely still prone to bias and influenced by time and overall improvements in practice. For example, one of the two studies demonstrating the highest benefit of HIT on hospital mortality (45) evaluated COVID patients early in pandemic, and it is likely that improvement in mortality was due to advances in COVID patient management rather than to HIT implementation (54). Another study compared a postimplementation cohort with historical controls from four years prior to implementation (40).
Undoubtedly, the mechanism by which HIT was integrated to the clinicians’ workflow is important. However, similar approaches to HIT integration may yield different results. Six studies in this SR evaluated HIT implementation to supplement RRT activations in settings where RRT activations were the standard of care. Of those, three pre-post studies demonstrated a decrease in hospital mortality associated with HIT intervention (23,30,34). Possible factors associated with the positive effect on hospital mortality included off-site nurse review to filter alerts before contacting the RRT, alerting bedside staff as well as RRT members, and probable improvements in practice over a prolonged study period. The other three studies (two pre-post studies that evaluated sepsis-related outcomes, and one RCT) did not demonstrate any significant improvement in hospital mortality (24,29,35).
Our SR has several strengths. We performed meta-analyses of studies reporting meaningful patient outcomes: LOS and mortality, rather than more immediately and easily measurable surrogate markers such as time to RRT activation, ICU transfer, or specific interventions, which helped us form robust conclusions (55). Study settings included all relevant acute care hospital populations: floor, ICU, and ED, and most studies were large. We only included studies assessing HIT that had been implemented in practice as opposed to studies that described development or validation of an HIT to assess “real-world” use of HIT and its effects on patient outcomes (41). Although our SR includes a broad range of settings and populations, we hoped this work would provide relevant insights across the spectrum of acute care.
Important limitations are as follows. Heterogeneity was found to be moderate or substantial in the meta-analyses of the studies evaluating hospital mortality, hospital, and ICU LOS among the entire study cohorts (Figs. 2 and 4; Supplemental Fig. 1, https://links.lww.com/CCM/H98; Supplemental Fig. 2, https://links.lww.com/CCM/H99; Supp lemental Fig. 3, https://links.lww.com/CCM/H100; Supplemental Fig. 4, https://links.lww.com/CCM/H101 [legend, https://links.lww.com/CCM/H92]). This heterogeneity was mostly attributed to the difference in types of deterioration detected and study flaws related to temporal and practice changes. HIT for the detection of patient deterioration included distinct types of electronic systems using data from continuous bedside monitoring, EMR, and other electronic documentation. There was not a uniform definition of criteria for clinical deterioration across all studies. The most common conditions identified were AKI, early sepsis, or physiologic deterioration based on EWS or vital signs parameters. The difference in baseline states and standards of care across study settings may also affect the effect of HIT implementation.
However, although type of deterioration, modality of assessment, and disease states differed, all HIT implementation mechanisms required emergent responses by the clinical team as an integral part of the intervention and were designed to alert the teams to deterioration earlier than usual practice.
The certainty of evidence of the included studies was low, mostly due to methodological limitations and inconsistency. Some studies described unadjusted results, and some results were imprecise including wide CIs. Therefore, it is possible that other unmeasured factors influenced the effectiveness of the intervention, potentially under- or overestimating the true impact.
Improved results after HIT implementation in the pre-post studies may be attributed more to practice advances and quality improvement initiatives rather than to HIT implementation itself.
CONCLUSIONS
In this SR and meta-analysis, the implementation of HIT for early detection of deterioration in acute care settings was not significantly associated with improved mortality or LOS in the meta-analyses of RCTs. In the meta-analyses of pre-post studies, HIT was associated with improvement in hospital mortality and hospital LOS; however, these results should be interpreted with caution. We believe the differences in patient outcomes between the findings of the RCTs, and pre-post studies may be secondary to multiple potential confounding factors including practice advances and quality improvement initiatives rather than to HIT implementation itself.
REFERENCES