Skip main navigation

Hospital-Based Quality Improvement Interventions for Patients With Acute Coronary Syndrome

A Systematic Review
Originally publishedhttps://doi.org/10.1161/CIRCOUTCOMES.118.005513Circulation: Cardiovascular Quality and Outcomes. 2019;12:e005513

    Abstract

    Background:

    Quality improvement initiatives have been developed to improve acute coronary syndrome care largely in high-income country settings. We sought to synthesize the effect size and quality of evidence from randomized controlled trials (RCTs) and nonrandomized studies for hospital-based acute coronary syndrome quality improvement interventions on clinical outcomes and process of care measures for their potential implementation in low- and middle-income country settings.

    Methods and Results:

    We conducted a bibliometric search of databases and trial registers and a hand search in 2016 and performed an updated search in May 2018 and May 2019. We performed data extraction, risk of bias assessment, and quality of evidence assessments in duplicate. We assessed differences in outcomes by study design comparing RCTs to nonrandomized quasi-experimental studies and by country income status. A meta-analysis was not feasible due to substantial, unexplained heterogeneity among the included studies, and thus, we present a qualitative synthesis. We screened 5858 records and included 32 studies (14 RCTs [n=109 763] and 18 nonrandomized quasi-experimental studies [n=54–423]). In-hospital mortality ranged from 2.1% to 4.8% in the intervention groups versus 3.3% to 5.1% in the control groups in 5 RCTs (n=55 942). Five RCTs (n=64 313) reported 3.0% to 31.0% higher rates of reperfusion for patients with ST-segment–elevation myocardial infarction in the intervention groups. The effect sizes for in-hospital and discharge medical therapies in a majority of RCTs were 3.0% to 10.0% higher in the intervention groups. There was no significant difference in 30-day mortality evaluated by 4 RCTs (n=42 384), which reported 2.5% to 15.0% versus 5.9% to 22% 30-day mortality rates in the intervention versus control groups. In contrast, nonrandomized quasi-experimental studies reported larger effect sizes compared to RCTs. There were no significant consistent differences in outcomes between high-income and middle-income countries. Low-income countries were not represented in any of the included studies.

    Conclusions:

    Hospital-based acute coronary syndrome quality improvement interventions have a modest effect on process of care measures but not on clinical outcomes with expected differences by study design. Although quality improvement programs have an ongoing and important role for acute coronary syndrome quality of care in high-income country settings, further research will help to identify key components for contextualizing and implementing such interventions to new settings to achieve their desired effects.

    Systematic Review Registration:

    URL: https://www.crd.york.ac.uk/PROSPERO/. Unique identifier: CRD42016047604.

    WHAT IS KNOWN

    • Hospital-based quality improvement programs have been implemented to improve quality of care for acute coronary syndrome, particularly in high-income country settings.

    • The evidence base evaluating the efficacy of these programs on process of care measures and clinical outcomes has largely been derived from nonrandomized studies, though randomized controlled trials of quality improvement interventions have been undertaken more recently.

    WHAT THIS STUDY ADDS

    • This systematic review synthesizes the evidence base for hospital-based acute coronary syndrome quality improvement interventions on process of care measures and clinical outcomes.

    • Randomized trial data show more modest effects on process measures like reperfusion rates and medication use compared with nonrandomized studies without clear effects on improving clinical outcomes.

    • This study demonstrates substantial heterogeneity across study reports, expected differences in effect size by study type, and estimated direction and magnitude of effects, which are relevant to settings where quality improvement programs may be newly implemented, including low- and middle-income countries.

    Introduction

    In 2015, the estimated global prevalence of ischemic heart disease was 111 million (95% uncertainty interval: 101 to 122 million) with 7.3 million global cases of fatal acute myocardial infarction (95% uncertainty interval: 6.8–7.8 million).1 In response to delays and deficiencies in acute myocardial infarction and acute coronary syndrome care associated with high morbidity and mortality rates, professional organizations have developed quality improvement initiatives. These quality improvement programs are complex interventions that frequently include clinical pathways, audits, performance feedback, education, and checklists. Nonrandomized studies have evaluated the efficacy of various hospital-based acute coronary syndrome quality improvement interventions on clinical outcomes and process of care measures. There is evidence for temporal improvement of evidence-based management and outcomes for acute coronary syndrome, including a reduction in disparities of care. For example, the joint American Heart Association’s and American College of Cardiology Chest Pain-MI Registry (formerly known as ACTION-Registry) demonstrated temporal improvements in process of care measures from 2006 to 2014, such as the use of aspirin (94%–99%), β-blockers (93%–98%), and lipid-lowering medications (85%–99%) at discharge.2

    To overcome the potential confounding and uncertainty inherent in nonrandomized studies and to understand which components of these complex quality improvement interventions are effective, several teams have performed randomized or quasi-randomized trials of quality improvement interventions, largely in high-income countries. However, questions remain about their generalizability across and implementation in different settings, including low- and middle-income countries.

    The objective of this systematic review was to estimate the effect size and quality of evidence for hospital-based acute coronary syndrome quality improvement interventions on clinical outcomes and process of care measures using data from randomized controlled trials (RCTs) and to summarize differences in the effect estimates between RCTs and nonrandomized studies. We also contextualize the findings on how quality improvement interventions may be particularly useful in low- and middle-income country health systems where the presentation and management of acute coronary syndrome is more heterogeneous than in high-income countries, evidence-practice gaps are frequently greater, and clinical outcomes are generally, but not always, poorer.3

    Methods

    We developed and published our systematic review protocol on the international prospective register of systematic reviews (PROSPERO)4 a priori and performed our review according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines illustrated in the Figure. All supporting data and methods used for this systematic review are available within the article and supplemental files and can be used to replicate the study.

    Figure.

    Figure. Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow chart of included studies. RCT indicates randomized controlled trial.

    Search Methods

    In November 2016, we conducted a bibliometric search of 9 databases. We hand-searched references of included trials to identify additional studies. This search was updated in May 2018 and May 2019 to include trials that may have been published since the initial search. We placed no restrictions on language of publication. Supplement I in the Data Supplement provides the detailed list and search strategies for each database.

    Included Studies

    Two authors (Drs Bahiru and Agarwal) independently conducted title and abstract screening. Differences between the 2 initial reviewers about inclusion of studies were resolved by consensus or review with a third author (Dr Huffman). We included individual- and cluster-level RCTs and nonrandomized quasi-experimental studies of acute coronary syndrome quality improvement interventions. We included a variety of interventions including audit and feedback reporting systems, admission and discharge checklists, chart case management, patient educational or behavioral change materials, health care quality training that are directed as the hospital system, doctors, nurses, or allied health professionals, or information management systems with the goal of being inclusive in the type and target of intervention. The classifications of the included study settings into high-, middle-, and low-income were made based on the World Bank’s Atlas calculation methods using gross national income per capita.5

    Study Outcomes

    We included a combination of clinical outcomes and process of care measures for our outcomes. The coprimary outcomes included rates of (1) in-hospital major adverse cardiovascular events (fatal and nonfatal myocardial infarction, fatal and nonfatal stroke, and major bleeding, combined and separate), (2) reperfusion for patients with ST-segment elevation myocardial infarction (STEMI), and (3) in-hospital and discharge medical therapy including anti-platelets, anticoagulants, β-blockers, and statins (combined and separate). Secondary outcomes included (1) time from hospital presentation to initial ECG, (2) time to reperfusion (STEMI only), (3) 30-day and 1-year major adverse cardiovascular events. We also attempted to evaluate rates of behavioral counseling for diet, activity, and tobacco cessation, uptake of quality improvement intervention components, patient-level health-related quality of life, patient-related costs as additional secondary outcomes; however, we did not identify any studies that reported these outcomes.

    Data Extraction

    Data extraction, risk of bias assessment, and quality of evidence assessments were performed in duplicate by 2 authors (Drs Bahiru and Agarwal) using standardized forms. Differences were resolved by consensus or review with a third author (Dr Huffman). The risk of bias assessment was performed using the Cochrane Risk of Bias Tool across the domains of selection, performance, detection, attrition, reporting, and other biases. The quality of evidence assessment was performed using the Grading of Recommendations Assessment, Development and Evaluation framework checklist, which accounts for issues related to internal validity (risk of bias, inconsistency, imprecision, and publication bias) and to external validity, such as directness of results.6

    Statistical Analyses

    This systematic review presents a qualitative, narrative synthesis of data from both individual- and cluster-level RCTs and nonrandomized quasi-experimental studies of acute coronary syndrome quality improvement interventions. We sought to perform a meta-analysis, but did not do so because of substantial, unexplained heterogeneity across the different studies.

    Results

    Summary of Included Studies

    The Figure demonstrates the PRISMA flowchart. After de-duplication, we identified 5858 records to screen using our search methods. We excluded 5727 studies through title/abstract screening and reviewed the full texts of the 131 remaining studies. We excluded 94 records after full-text review. Five of the 37 studies that met the inclusion criteria were ongoing trials, and thus, we present 32 studies in this systematic review. Among the included studies, we identified 14 RCTs (2 individual level and 12 cluster level) consisting of 109 763 patients (Table 1).7–20 We also identified and included 18 studies (n=54 423) that used either a nonrandomized controlled or uncontrolled quasi-experimental study design (Supplements II and III in the Data Supplement).21–38 Twenty-two of the 32 studies were conducted in high-income countries including 10 in the United States, 5 in Australia, 6 in Western Europe, and 1 in Canada; while the remaining 10 studies were conducted in middle-income countries including 3 in China, 3 in India, 2 in Taiwan, 1 in Brazil, and 1 in Iran.

    Table 1. Summary of Included Randomized Controlled Trials

    StudySettingNPopulationIntervention and Comparator1o and Key 2o Outcomes
    Bailey, 2000–20017Patient-level; MissouriI: 488 patients, C: 365 patients, TP: 853AMIIntervention: Pharmacist-led recommendations for secondary prevention at discharge.Guideline-directed discharge medications.
    Comparator: Usual care.
    Berner, 1999–20008Cluster; AlabamaI1: 7 hospitals, I2: 8 hospitals, C: 6 hospitals, TP: 2210Unstable anginaIntervention1: QI intervention developed and implemented by opinion leaders.ECG within 20 minutes after arrival, guideline-directed in-hospital and discharged medications.
    Intervention 2: General blinded hospital performance feedback.
    Comparator: Usual care.
    Berwanger, 2011–20129Cluster; urban hospitals in BrazilI:19 hospitals, C: 17 hospitals, TP: 1150AMIIntervention: Reminders, checklists, case management, and educational materials.Guideline-directed in-hospital and discharge medications, in-hospital and 30-day MACE outcomes
    Comparator: Usual care.
    Du, 2007–201010Cluster; urban hospitals in ChinaI: 32 hospitals, C: 38 hospitals, TP: 3500AMIIntervention: Clinical care pathway.Guideline-directed discharge medications, reperfusion therapy, and in-hospital MACE
    Comparator: Usual care.
    Flather, 2007–200911Cluster; France, Italy, Poland, Spain, United KingdomI: 19 hospitals, C: 19 hospitals, TP: 2622AMIIntervention: QI tools specific to individual hospitals led by senior cardiologists.Guideline-directed in-hospital and discharge medications, risk stratification, coronary-angiography
    Comparator: Usual care.
    Guenancia, 2012–201312Multicenter patient-level; Burgundy, FranceI: 286 patients, C: 286 patients, TP: 572NSTEMIIntervention: GRACE score+clinical assessment to guide clinical decision.Guideline-directed in-hospital medications and in-hospital MACE.
    Comparator: Clinical assessment only.
    Heller, 1996–199813Cluster; New South Wales, AustraliaI: 19 hospitals, C: 19 hospitals, TP: 3242AMI, angina, chest painIntervention: Educational sessions performance feedback.Guideline-directed in-hospital and discharge medications, coronary angiography, and echocardiography.
    Comparator: Educational session only.
    Huffman, 2014–201716Cluster; stepped wedge; Kerala, IndiaI: 63 hospitals, C: 63 hospitals, TP: 21 374AMIIntervention: Multi-component QI toolkit.30-day MACE, health-related quality of life, in-hospital, and discharge medical therapy.
    Comparator: Usual care.
    Kinsman, 2008–200914Cluster; rural Victoria, AustraliaI: 3 hospitals, C: 3 hospitals, TP: 108AMI eligible for thrombolysisIntervention: Clinical pathways, reminders, education sessions, audit, and feedback.Eligible patients with AMI receiving thrombolysis, time to thrombolysis, and ECG.
    Comparator: Usual care.
    Lytle, 2008–201015Cluster; multiple participants in GWTG registryI: 19 hospitals, C: 17 hospitals, TP: 19 579STEMI or NSTEMIIntervention: Targeted feedback reports based on 3 lowest performing metricsGuideline-directed in-hospital and discharge medications, reperfusion therapy.
    Comparator: Standard performance feedback.
    Sauaia, 1994–199617Cluster; ColoradoI: 9 hospitals, C: 9 hospitals, TP: 1367AMIIntervention: Written feedback+2-hour on-site performance feedback.Guideline-directed in-hospital and discharge medications, and reperfusion within 12 hours of arrival.
    Comparator: Written feedback only.
    Soumerai, 1992–199618Cluster; MinnesotaI: 20 hospitals, C: 16 hospitals, TP: 5347AMIIntervention: Small and large group discussion, informal consultations revision of protocols, clinical pathways.Guideline-directed in-hospital and discharge medications.
    Comparator: Usual care.
    Tu, 2001–200519Cluster; Ontario, CanadaI: 42 hospitals, C: 39 hospitals, TP: 18 492AMIIntervention: Early feedback.12 process of care indicators for AMI.
    Comparator: Delayed feedback.
    Wu, 2011–201420Cluster; stepped wedge; ChinaI: 101 hospitals, C: 101 hospitals, TP: 29 346Final diagnosis of AMIIntervention: QI team, clinical pathways, education session, audit, and feedback.In-hospital mortality, in-hospital MACE, 16 key performance indicators.
    Comparator: Usual care.

    Total number of participants: 109 763. 1° indicates primary; 2°, secondary; AMI, acute myocardial infarction; C, comparator; GRACE, Global Registry of Acute Coronary Events; GWTG, Get With The Guidelines Registry; I, intervention; MACE, major adverse cardiovascular events; NSTEMI, non-ST elevation myocardial infarction; QI, quality improvement; STEMI, ST-segment-elevation myocardial infarction; and TP, total number of participants.

    Risk of Bias Assessment of RCTs

    Summaries of trial-specific risk of bias assessment and documentation supporting risk of bias assessment for included RCTs are listed in Supplement IV in the Data Supplement. Six of 14 RCTs had low risk of selection bias based on reported methods of sequence generation and or allocation concealment,9,10,14,16,19,20 whereas 8 studies had unclear or high risk of selection bias.7,8,11–13,15,17,18 None of the 14 trials blinded the study personnel, and thus had a high risk of performance bias. Though only one RCT20 blinded the outcome assessors, we determined there to be a low risk of detection bias for the remaining 13 RCTs given the objective nature of these outcomes, which are less likely to be influenced by unblinding of outcome assessors.39 We categorized 9 trials as having low risk of attrition bias due to differential missingness across groups,7,8,10–12,14,16,17,20 while 4 trials were unclear risk of bias for this domain.13,15,18,19 Six studies had low risk of reporting bias based on previously published protocols and adherence to those protocols,9,10,14–16,20 while 7 had unclear risk of reporting bias,7,8,11–13,17,18 and one study had high risk of reporting bias.19 We also identified 4 studies as having high risk of recruitment bias due to randomization at the cluster level with recruitment at the individual level.7,12,13,18

    Summary of Findings by Outcome

    We summarized selected outcomes from individual studies of included RCTs in Table 2, and a comprehensive summary is listed in Supplement V in the Data Supplement. We present a summary of findings in Table 3. Outcomes from nonrandomized studies are summarized in Supplements VI through VIII in the Data Supplement.

    Table 2. Summary of Outcomes of Randomized Controlled Trials

    OutcomeTrialEvent Rates, No (%)Significance
    InterventionComparatorEffect (95% CI)*P Value
    In-hospital
    MACE
    Berwanger933 (5.5)38 (7.0)OR: 0.72 (0.36 to 1.43)0.35
    Du1092 (5.8)122 (6.4)RR: 1.12 (0.58 to 2.14)0.74
    Guenancia1226 (9.1)31 (10.8)OR: 1.59 (0.61 to 4.17)0.49
    Wu20559 (3.8)655 (4.4)aOR: 0.93 (0.75 to 1.15)NR
    In-hospital
    mortality
    Berwanger929 (4.8)28 (5.1)OR: 0.82 (0.37 to 1.82)0.62
    Du1041 (2.6)78 (4.1)RR: 1.60 (0.97 to 2.64)0.07
    Guenancia126 (2.1)11 (3.8)OR: 1.16 (0.68 to 2.01)NR
    Huffman16321 (2.8)331 (3.3)aOR: 0.98 (0.82 to 1.17)NR
    Rates of reperfusion for STEMIDu10290 (42.7)229 (31.8)RR: 1.24 (0.98 to 1.55)0.70
    Huffman164805 (71.0)5067 (73.2)OR: 1.24 (1.06 to 1.46)NR
    Kinsman14Thrombolysis;
    Baseline: 80%;
    Post-intervention: 78%
    Thrombolysis;
    Baseline: 96%;
    Post-intervention: 84%
    NRI: 0.86,
    C: 0.19
    Lytle15730 (97.2)228 (94.2)NR0.03
    Sauaia17Baseline: 12 (55.0);
    Post-intervention: 9 (75.0)
    Baseline: 31 (84);
    Post-intervention: 4 (44)
    Control 6.5× worse compared to baselineI: 0.01,
    C: 0.02
    Tu19% change (95% CI):
    6.7 (−0.8 to 14.2)
    % change (95% CI):
    7.2 (−0.5 to 15.1)
    Absolute % difference:
    3.3 (−5.7 to 12.4)
    0.47
    Wu201414 (48.9)1683 (52.2)aOR: −2.2 (−4.7 to 0.30)NR
    30-day total mortalityBerwanger942 (7.0)46 (8.4)OR: 0.79 (0.46 to 1.34)0.38
    Huffman16445 (3.9)509 (5.1)aOR: 0.87 (0.75 to 1.00)NR
    Sauaia17Baseline: 81 (19.0);
    Post-intervention: 33 (15.0)
    Baseline: 85 (17.0);
    Post-intervention: 46 (22.0)
    NRNR
    Tu19Absolute % change:
    (95% CI): −1.9 (−3.8 to −0.1)
    Absolute % change:
    (95% CI): 0 (−2.3 to 2.3)
    Absolute % difference: (95% CI): −2.5 (−4.9 to 0.1)0.50
    30-day MACEBerwanger949 (8.1)55 (10.1)OR: 0.76 (0.45 to 1.27)0.30
    Huffman16445 (3.9)645 (6.4)OR: 0.92 (0.81 to 1.04)NR

    aOR indicates adjusted odds ratio; MACE, major adverse cardiovascular events; NR, not reported; NS, not significant; and STEMI, ST-segment-elevation myocardial infarction.

    *Odds ratios or adjusted odd ratios represent the effect estimates of the intervention compared with the control.

    †Absolute percentage differences represent the effect estimates using a difference in difference analysis.

    Table 3. Summary of Outcomes and Quality of Evidence of Randomized Controlled Trials

    Hospital-Based Acute Coronary Syndrome Quality Improvement Interventions Versus Usual Care
    OutcomesEffect on OutcomeStudies/Total ParticipantsQuality of the EvidenceComments
    In-hospital MACEAbsolute in-hospital mortality ranged from 2.1% to 4.8% in the intervention vs 3.3% to 5.1% in the control and the unadjusted mortality rates were 0.3% to 1.7% lower in the intervention groups.9,10,12,16,205 RCTs, TP: 55 942Moderate*Unblinded studies*
    Rates of reperfusion for STEMIIn 5 studies, absolute rate of reperfusion was 3% to 10% higher in the intervention except for one outlier study with 31% higher rate in the intervention. Two studies showed no difference.10,14–17,19,207 RCTs, TP: 93 659Moderate*Unblinded studies* with some inconsistency
    Rates of in-hospital and discharge medical therapyIn-hospital medical therapy: Effect estimates ranged from no difference to 15.2% except one outlier study with 31% increase in the intervention vs 9.1% in the control.8,9,11,13,16,17,19,208 RCTs, TP: 79 803Moderate*Unblinded studies* with some inconsistency
    Discharge medical therapy: Effect estimates ranged from no difference to 7.2% higher in the intervention groups.7–11,15–2011 RCTs, TP: 100 511Moderate*Unblinded studies* with some inconsistency
    Door to ECG timeOne RCT showed 10% higher rate of ECGs done in time in the intervention group. One small sized study showed no difference between the intervention and control.14,202 RCT, TP: 29 454Low*Unblinded study* Unable to assess for inconsistency
    Door to any reperfusion for STEMI timeTwo studies reported a 2% to 7% higher rate of reperfusion under 90 minutes in the intervention group. Three studies reported no difference in reperfusion time between groups.10,14–16,205 RCTs, TP: 73 908Low*Unblinded studies* Significantly variable in outcome measures
    30-Day MACEThe effect estimates ranged from 3.9% to 15% and 5.1% to 22% in the intervention and control groups, respectively.9,16,17,194 RCTs, TP: 42 384Low*§Unblinded studies* Significant inconsistency in estimates

    GRADE Working Group grades of evidence.6 High quality: We are very confident that the true effect lies close to that of the estimate of the effect. Moderate quality: We are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low quality: Our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect. Very low quality: We have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect. MACE indicates major adverse cardiovascular events; STEMI, ST-segment-elevation myocardial infarction; and TP, total number of participants.

    *Downgraded due to study limitations.

    †Downgraded due to inconsistency.

    §Downgraded due to imprecision.

    ‡ECG done in time means the patient received the first ECG within 10 minutes of hospital arrival.

    In-Hospital Major Adverse Cardiovascular Events

    Five RCTs (n=55 942) assessed the effect of hospital-based quality improvement interventions on in-hospital major adverse cardiovascular events consisting of fatal and nonfatal myocardial infarction, fatal and nonfatal stroke, and major bleeding, combined and separate).9,10,12,16,20 The overall absolute rate of in-hospital mortality ranged from 2.1% to 4.8% in the intervention groups compared with 3.3% to 5.1% in the control groups. The unadjusted mortality rates were 0.3% to 1.7% lower in the intervention groups compared with the control (Table 2). In comparison, 7 nonrandomized quasi-experimental studies (n=42 013) showed an absolute event rate reduction in in-hospital mortality ranging from 0.2% to 13% post-intervention (Supplements VI through VIII in the Data Supplement).21,23,31,34,36–38

    Rates of Reperfusion for STEMI

    Seven RCTs (n=93 659) assessed the effect of hospital-based acute coronary syndrome quality improvement interventions on rates of reperfusion for patients with STEMI.10,14–17,19 Five RCTs (n=64 313) showed an overall 3.0% to 31.0% higher absolute rate of reperfusion in the intervention groups compared with the control groups.10,15–17,19 Two RCTs (n=29 454) showed no difference between the intervention and control groups.14,20 One17 of the 5 RCTs (n=1367) was an outlier with 31% higher rate in reperfusion in the intervention group compared with the other 4,10,15,16,19,20 (n=62 946) which showed a 3.0% to 10.9% higher rate of reperfusion in the intervention groups (Table 2). Five nonrandomized controlled and uncontrolled quasi-experimental studies (n=28 083) showed no increase in rates of reperfusion postintervention (Supplements VI through VIII in the Data Supplement).21,27,36–38

    Rates of In-Hospital and Discharge Medical Therapy

    Table 3 and Supplement V in the Data Supplement describe the results for in-hospital and discharge medical therapy. Eight RCTs (n=79 803)8,9,11,13,16,17,19,20 evaluated in-hospital aspirin, β-blocker, and anticoagulant use. The effect estimates reported in 7 studies ranged from no difference to 15.2% higher rates in the intervention, and one outlier RCT8 (n=2210) showed a substantially larger effect on in-hospital aspirin and anticoagulation therapy in the intervention group. Eleven RCTs (n=100 511)7–11,15–20 evaluated discharge medical therapy, including aspirin, β-blocker, statin, and angiotensin-converting enzyme-inhibitor/angiotensin receptor blockers (ACE-I/ARB). The effect estimates from 9 trials ranged from no difference to 7.2% higher rates in the intervention groups, and one outlier RCT18 (n=5347) showed a substantially larger effect on discharge aspirin and β-blocker use in the intervention group. One RCT10 (n=3500) reported combined recommended discharged therapies showed a 11% absolute higher rate in the intervention compared with control (unadjusted RR [95% CI], 1.23 [1.06–1.42]; P=0.007).

    In contrast, the results from nonrandomized quasi-experimental studies showed a 2.6% to 25% increase of in-hospital medical therapy and a 2.0% to 80.0% increase in discharge medical therapy with most studies reporting a >10% increase in in-hospital or discharge medical therapy post-intervention (Supplements VI through VIII in the Data Supplement).22–24,27,30,33,37,38

    Hospital Presentation to ECG Time

    One RCT (n=29 346)20 showed 10% higher rate of ECGs completed in time, that is, within 10 minutes after arrival, in the intervention group compared with the control (adjusted odds ratio, 1.12 [0.90–1.39]) while another RCT (n=108) showed no difference in door to ECG time between the intervention and control groups (Supplement V in the Data Supplement).14 Four nonrandomized quasi-experimental studies21,25,31,38 (n=5058) showed minimal differences.

    Door to any Reperfusion Time for Patients With STEMI

    Five RCTs (73 908) evaluated door to any reperfusion time for patients with STEMI.10,14–16,20 Three RCTs10,14,16 (n=24 983) reported no difference in mean or median door to balloon time, while 2 RCTs15,20 (n=48 925) showed an absolute 2.0% to 7% higher rate of reperfusion in <90 minutes in the intervention groups compared with the control groups. In contrast, 7 nonrandomized quasi-experimental studies21,25,26,31,33,34,37,38 (n=7039) showed a significant reduction in door to any reperfusion time or an increase in rates of reperfusion within 60 minutes of presentation (Supplements VI through VIII in the Data Supplement).

    30-Day and 1-Year Major Adverse Cardiovascular Events

    Four RCTs (n=42 384) reported 30-day mortality rates of 3.9% to 15% in the intervention groups compared with the 5.1% to 22.0% in the control groups9,16,17,19 The 30-day mortality rates from the more recent 3 RCTs9,16,19 are comparable and <10% in comparison to one RCT which reported a markedly higher 15% and 22% 30-day mortality rates in the intervention and control groups, respectively. This relatively small RCT (n=1397) was completed between 1994 to 1996, and the lower 30-day mortality rates in the more recent trials may be a reflection of time trends in improvements in clinical outcomes of acute coronary syndrome due to better clinical management. One nonrandomized quasi-experimental study (n=420) showed a 2.5% reduction of total 30-day mortality.34 No RCTs reported differences in 1-year major adverse cardiovascular event rates. In contrast, 4 nonrandomized quasi-experimental studies (n=14 824) showed a 1.2% to 4.0% lower mortality rate at 1 year in the intervention groups (Supplements VI through VIII in the Data Supplement).22,24,30,34

    Ten of the 14 RCTs were conducted in high-income countries, and 4 were conducted in middle-income countries. Overall, there were no consistent significant differences in the effect estimates on clinical outcomes and process of care measures between the high-income and middle-income countries. Additionally, 12 out of the 18 quasi-experimental studies were conducted in high-income countries while the remaining were in middle-income countries. There were significant variabilities in the representation of high-income versus middle-income countries for each study outcome and together with the inconsistencies in the effect estimates, they limit the ability to confidently assess differences in outcomes by country income status from the quasi-experimental studies.

    Study Quality Assessment

    Table 3 describes the outcome-specific quality of evidence assessment for RCTs. We graded the quality of evidence moderate for 4 out of the 7 outcomes and low or very low for the remainder, downgrading because of study limitations and between-study heterogeneity. We also present the quality of evidence for nonrandomized quasi-experimental studies in Supplement VIII in the Data Supplement, which were considered very low given study designs, study limitations, and heterogeneity.

    Discussion

    This systematic review is the first, to our knowledge, of RCTs and nonrandomized quasi-experimental studies on hospital-based quality improvement interventions for patients with acute coronary syndrome on clinical outcomes and process of care measures. There was substantial heterogeneity across studies in the types of in-hospital quality improvement interventions studied, which limited our ability to identify what types of interventions were most efficacious. Despite a large number of RCTs that reported on the primary outcomes, the heterogeneity in how the results are reported limited the ability to perform a pooled analysis and thus, we present a qualitative analysis of the data.

    Overall, we found the quality of the evidence from RCTs moderate to low, which showed modest to no effect of the interventions studied on clinical outcomes, including in-hospital and 30-day mortality and combined major adverse cardiovascular events. In contrast, nonrandomized studies demonstrated larger, but overall modest effect sizes on clinical outcomes. Similarly, RCTs showed modest to no effect of the interventions on rates of reperfusion for patients with STEMI and rates of guideline-directed in-hospital and discharge medical therapy, although overall the effects sizes were higher compared with the effects on clinical outcomes. Nonrandomized quasi-experimental studies showed a greater effect size on process of care outcome measures in comparison to RCTs, although with greater inconsistency in the size of the effect estimates. Overall, both randomized and nonrandomized studies demonstrated larger effect estimates for process of care measures compared with clinical outcomes. Only 13 out of 32 studies (7 RCTs and 6 nonrandomized studies) reported clinical outcome measures and only 4 RCTs reported the primary clinical outcomes of this review (ie, in-hospital mortality and in-hospital major adverse cardiovascular events). The evidence base for acute coronary syndrome quality improvement interventions on clinical outcomes could be improved if future studies include clinical outcomes measures more consistently to help identify and test which interventions may have greater impact on clinical outcomes. There was significant heterogeneity in the interventions included in this review, ranging from education programs, targeted performance feedback, clinical pathways, and audits among others, which did not allow to assess specific interventions that are potentially more efficacious than others. One important area of future research will be process evaluation of existing interventions to better understand which interventions might be more effective in different clinical settings.

    A range of high-income, high- and low- middle-income countries were represented in this review, both in the randomized and nonrandomized studies. There was no significant difference in the effect estimates with the various interventions studied between high-income and middle-income countries, including Brazil, China, and India. However, low-income settings, including countries in sub-Saharan Africa, are not well represented in the studies included in this systematic review. Despite the growing burden of ischemic heart disease, there is minimal understanding on implementation and utilization of evidence-based acute coronary syndrome management in low-income countries.3 Low-income countries may potentially have higher gain, both in clinical and process of care outcome measures, from acute coronary syndrome quality improvement interventions compared with middle- or high-income countries that typically have higher baseline use of guideline-directed management and lower event rates. Therefore, having more low-income countries represented in future clinical trials could help understand which clinical settings may benefit the most from quality improvement interventions.

    Implementation of acute coronary syndrome quality improvement interventions in the context of low-income countries, in addition to process of care and outcome evaluations, need to also consider structural interventions, including at the health worker (eg, adequate staffing and training), hospital (eg, functioning diagnostic and treatment equipment), and prehospital (eg, available emergency response system) levels to enhance performance. Evidence from a systematic review that assessed strategies to improve health care provider performance in low-and middle-income countries shows that the efficacy of strategies to improve health care provider performance in low resource settings was highly variable. The effect estimates were the largest for multifaceted strategies that incorporated several elements including improving infrastructure, training and group problem solving, and emphasizes the need for future research to generate better evidence using standardized methodologies of outcome analysis and robust study designs including RCTs.39

    Strengths and Limitations

    There are several strengths to this systematic review, including providing a summary of multiple experimental study designs. The concurrent summary of evidence from RCTs and nonrandomized quasi-experimental studies also allows for comparison of the evidence between the different study designs. Title screening, data extraction, and quality assessments were performed in duplicate to minimize error, and a prespecified protocol before the initiation of review was published to guide the search strategy and minimize the risk of bias.

    This review also has limitations. First, the study duration of the RCTs may not have been implemented long enough to observe changes in health systems, culture, and attitude from providers and administrators that could influence the implementation of interventions, quality of care, and outcomes. Although this review included a wide range of countries of varying economic status, there remains underrepresentation of low-income countries, which limits the generalizability of the findings to those settings potentially most in need of health system strengthening. Understanding effective elements of quality improvement interventions is important to improve quality and safety of acute coronary syndrome in diverse clinical and resource settings. Second, the process of performing RCTs of quality improvement interventions could have led to improvements in baseline care through a Hawthorne effect. Third, the review tried to synthesize complex interventions, which may not be possible.

    Conclusions

    This systematic review demonstrates that RCTs of hospital-based acute coronary syndrome quality improvement interventions have a modest effect on process of care measures but not on clinical outcomes. Overall, nonrandomized quasi-experimental studies showed larger effect sizes compared with randomized clinical trials, particularly for process of care measures. Understanding which components of quality improvement interventions are more effective and their role in low-resource settings, which were largely not included in these trials, would be important future directions. Further research will also help to identify key components for contextualizing successful acute coronary syndrome quality improvement interventions to new settings.

    Footnotes

    The Data Supplement is available at https://www.ahajournals.org/doi/suppl/10.1161/CIRCOUTCOMES.118.005513.

    Ehete Mikael G. Bahiru, MD, Division of Cardiology, Department of Medicine, David Geffen UCLA School of Medicine, 650 Charles E. Young Dr S A2-237 CHS, MC 167917, Los Angeles, CA 90095. Email

    References

    • 1. Benjamin EJ, Virani SS, Callaway CW, Chamberlain AM, Chang AR, Cheng S, Chiuve SE, Cushman M, Delling FN, Deo R, de Ferranti SD, Ferguson JF, Fornage M, Gillespie C, Isasi CR, Jiménez MC, Jordan LC, Judd SE, Lackland D, Lichtman JH, Lisabeth L, Liu S, Longenecker CT, Lutsey PL, Mackey JS, Matchar DB, Matsushita K, Mussolino ME, Nasir K, O’Flaherty M, Palaniappan LP, Pandey A, Pandey DK, Reeves MJ, Ritchey MD, Rodriguez CJ, Roth GA, Rosamond WD, Sampson UKA, Satou GM, Shah SH, Spartano NL, Tirschwell DL, Tsao CW, Voeks JH, Willey JZ, Wilkins JT, Wu JH, Alger HM, Wong SS, Muntner P; American Heart Association Council on Epidemiology and Prevention Statistics Committee and Stroke Statistics Subcommittee. Heart disease and stroke statistics-2018 update: a report from the American Heart Association.Circulation. 2018; 137:e67–e492. doi: 10.1161/CIR.0000000000000558LinkGoogle Scholar
    • 2. Masoudi FA, Ponirakis A, de Lemos JA, Jollis JG, Kremers M, Messenger JC, Moore JWM, Moussa I, Oetgen WJ, Varosy PD, Vincent RN, Wei J, Curtis JP, Roe MT, Spertus JA. Trends in U.S. cardiovascular care: 2016 report from 4 ACC national cardiovascular Data registries.J Am Coll Cardiol. 2017; 69:1427–1450. doi: 10.1016/j.jacc.2016.12.005CrossrefMedlineGoogle Scholar
    • 3. Vedanthan R, Seligman B, Fuster V. Global perspective on acute coronary syndrome: a burden on the young and poor.Circ Res. 2014; 114:1959–1975. doi: 10.1161/CIRCRESAHA.114.302782LinkGoogle Scholar
    • 4. PROSPERO International Prospective Register of Systematic Reviews. Hospital-Based Quality Improvement Interventions for Patients with Acute Coronary Syndrome: A Systematic Review.www.crd.york.ac.uk/PROSPERO/DisplayPDF.php?ID=CRD42016047604. Accessed November 23, 2018.Google Scholar
    • 5. The World Bank. World Bank Country and Lending Groups.https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups. Accessed June 24, 2018.Google Scholar
    • 6. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann HJ; GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations.BMJ. 2008; 336:924–926. doi: 10.1136/bmj.39489.470347.ADCrossrefMedlineGoogle Scholar
    • 7. Bailey TC, Noirot LA, Blickensderfer A, Rachmiel E, Schaiff R, Kessels A, Braverman A, Goldberg A, Waterman B, Dunagan WC. An intervention to improve secondary prevention of coronary heart disease.Arch Intern Med. 2007; 167:586–590. doi: 10.1001/archinte.167.6.586CrossrefMedlineGoogle Scholar
    • 8. Berner ES, Baker CS, Funkhouser E, Heudebert GR, Allison JJ, Fargason CA, Li Q, Person SD, Kiefe CI. Do local opinion leaders augment hospital quality improvement efforts? A randomized trial to promote adherence to unstable angina guidelines.Med Care. 2003; 41:420–431. doi: 10.1097/01.MLR.0000052977.24246.38CrossrefMedlineGoogle Scholar
    • 9. Berwanger O, Guimaraes HP, Laranjeira LN, Cavalcanti AB, Kodama AA, Zazula AD, Santucci EV, Victor E, Tenuta M, Carvalho V, Mira VL, Pieper KS, Weber B, Mota LH, Peterson ED, Lopes RD. Effect of a multifaceted intervention on use of evidence-based therapies in patients with acute coronary syndromes in Brazil: the BRIDGE-ACS randomized trial.JAMA. 2012; 307:2041–2049.CrossrefMedlineGoogle Scholar
    • 10. Du X, Gao R, Turnbull F, Wu Y, Rong Y, Lo S, Billot L, Hao Z, Ranasinghe I, Iedema R, Kong L, Hu D, Lin S, Shen W, Huang D, Yang Y, Ge J, Han Y, Lv S, Ma A, Gao W, Patel A; CPACS Investigators. Hospital quality improvement initiative for patients with acute coronary syndromes in China: a cluster randomized, controlled trial.Circ Cardiovasc Qual Outcomes. 2014; 7:217–226. doi: 10.1161/CIRCOUTCOMES.113.000526LinkGoogle Scholar
    • 11. Flather MD, Babalis D, Booth J, Bardaji A, Machecourt J, Opolski G, Ottani F, Bueno H, Banya W, Brady AR, Bojestig M, Lindahl B. Cluster-randomized trial to evaluate the effects of a quality improvement program on management of non-ST-elevation acute coronary syndromes: the European Quality Improvement Programme for Acute Coronary Syndromes (EQUIP-ACS).Am Heart J. 2011; 162:700.e1–707.e1. doi: 10.1016/j.ahj.2011.07.027CrossrefGoogle Scholar
    • 12. Guenancia C, Stamboul K, Hachet O, Yameogo V, Garnier F, Gudjoncik A, Cottin Y, Lorgis L. Clinical effectiveness of the systematic use of the GRACE scoring system (in addition to clinical assessment) for ischaemic outcomes and bleeding complications in the management of NSTEMI compared with clinical assessment alone: a prospective study.Heart Vessels. 2016; 31:897–906. doi: 10.1007/s00380-015-0695-8CrossrefMedlineGoogle Scholar
    • 13. Heller RF, D’Este C, Lim LL, O’Connell RL, Powell H. Randomised controlled trial to change the hospital management of unstable angina.Med J Aust. 2001; 174:217–221.CrossrefMedlineGoogle Scholar
    • 14. Kinsman LD, Rotter T, Willis J, Snow PC, Buykx P, Humphreys JS. Do clinical pathways enhance access to evidence-based acute myocardial infarction treatment in rural emergency departments?Aust J Rural Health. 2012; 20:59–66. doi: 10.1111/j.1440-1584.2012.01262.xCrossrefMedlineGoogle Scholar
    • 15. Lytle BL, Li S, Lofthus DM, Thomas L, Poteat JL, Bhatt DL, Cannon CP, Fonarow GC, Peterson ED, Wang TY, Alexander KP. Targeted versus standard feedback: results from a randomized quality improvement trial.Am Heart J. 2015; 169:132–141.CrossrefMedlineGoogle Scholar
    • 16. Huffman MD, Mohanan PP, Devarajan R, Baldridge AS, Kondal D, Zhao L, Ali M, Krishnan MN, Natesan S, Gopinath R, Viswanathan S, Stigi J, Joseph J, Chozhakkat S, Lloyd-Jones DM, Prabhakaran D; Acute Coronary Syndrome Quality Improvement in Kerala (ACS QUIK) Investigators. Effect of a quality improvement intervention on clinical outcomes in patients in India with acute myocardial infarction: the ACS QUIK Randomized Clinical Trial.JAMA. 2018; 319:567–578. doi: 10.1001/jama.2017.21906CrossrefMedlineGoogle Scholar
    • 17. Sauaia A, Ralston D, Schluter WW, Marciniak TA, Havranek EP, Dunn TR. Influencing care in acute myocardial infarction: a randomized trial comparing 2 types of intervention.Am J Med Qual. 2000; 15:197–206. doi: 10.1177/106286060001500503CrossrefMedlineGoogle Scholar
    • 18. Soumerai SB, McLaughlin TJ, Gurwitz JH, Guadagnoli E, Hauptman PJ, Borbas C, Morris N, McLaughlin B, Gao X, Willison DJ, Asinger R, Gobel F. Effect of local medical opinion leaders on quality of care for acute myocardial infarction: a randomized controlled trial.JAMA. 1998; 279:1358–1363. doi: 10.1001/jama.279.17.1358CrossrefMedlineGoogle Scholar
    • 19. Tu JV, Donovan LR, Lee DS, Wang JT, Austin PC, Alter DA, Ko DT. Effectiveness of public report cards for improving the quality of cardiac care: the EFFECT study: a randomized trial.JAMA. 2009; 302:2330–2337. doi: 10.1001/jama.2009.1731CrossrefMedlineGoogle Scholar
    • 20. Wu Y, Li S, Patel A, Li X, Du X, Wu T, Zhao Y, Feng L, Billot L, Peterson ED, Woodward M, Kong L, Huo Y, Hu D, Chalkidou K, Gao R. Effect of a quality of care improvement initiative in patients with acute coronary syndrome in resource-constrained hospitals in China: a randomized clinical trial.JAMA Cardiol. 2019; 4:418–427. doi: 10.1001/jamacardio.2019.0897CrossrefMedlineGoogle Scholar
    • 21. Alexander T, Mullasari AS, Joseph G, Kannan K, Veerasekar G, Victor SM, Ayers C, Thomson VS, Subban V, Gnanaraj JP, Narula J, Kumbhani DJ, Nallamothu BK. A system of care for patients with ST-segment elevation myocardial infarction in India: the Tamil Nadu-ST-Segment Elevation Myocardial Infarction Program.JAMA Cardiol. 2017; 2:498–505. doi: 10.1001/jamacardio.2016.5977CrossrefMedlineGoogle Scholar
    • 22. Aziz EF, Javed F, Pulimi S, Pratap B, De Benedetti Zunino ME, Tormey D, Hong MK, Herzog E. Implementing a pathway for the management of acute coronary syndrome leads to improved compliance with guidelines and a decrease in angina symptoms.J Healthc Qual. 2012; 34:5–14. doi: 10.1111/j.1945-1474.2011.00145.xCrossrefMedlineGoogle Scholar
    • 23. Carlhed R, Bojestig M, Wallentin L, Lindström G, Peterson A, Aberg C, Lindahl B; QUICC study group. Improved adherence to Swedish national guidelines for acute myocardial infarction: the Quality Improvement in Coronary Care (QUICC) study.Am Heart J. 2006; 152:1175–1181. doi: 10.1016/j.ahj.2006.07.028CrossrefMedlineGoogle Scholar
    • 24. Carlhed R, Bojestig M, Peterson A, Aberg C, Garmo H, Lindahl B; Quality Improvement in Coronary Care Study Group. Improved clinical outcome after acute myocardial infarction in hospitals participating in a Swedish quality improvement initiative.Circ Cardiovasc Qual Outcomes. 2009; 2:458–464. doi: 10.1161/CIRCOUTCOMES.108.842146LinkGoogle Scholar
    • 25. Chen KC, Yen DH, Chen CD, Young MS, Yin WH. Effect of emergency department in-hospital tele-electrocardiographic triage and interventional cardiologist activation of the infarct team on door-to-balloon times in ST-segment-elevation acute myocardial infarction.Am J Cardiol. 2011; 107:1430–1435. doi: 10.1016/j.amjcard.2011.01.015CrossrefMedlineGoogle Scholar
    • 26. Dai X, Meredith D, Sawey E, Kaul P, Smith SC, Stouffer GA. A Quality Improvement Program for recognition and treatment of inpatient ST-segment elevation myocardial infarctions.JAMA Cardiol. 2016; 1:1077–1079. doi: 10.1001/jamacardio.2016.3031CrossrefMedlineGoogle Scholar
    • 27. Ellerbeck EF, Kresowik TF, Hemann RA, Mason P, Wiblin RT, Marciniak TA. Impact of quality improvement activities on care for acute myocardial infarction.Int J Qual Health Care. 2000; 12:305–310. doi: 10.1093/intqhc/12.4.305CrossrefMedlineGoogle Scholar
    • 28. Fakhr-Movahedi A, Soleimani M, Ghazvininejad R, Maher MK, Ghorbani R. Effect of patient-focused clinical pathway on anxiety, depression and satisfaction of patients with coronary artery disease: a quasi-experimental study.Iran Red Crescent Med J. 2015; 17:e29933. doi: 10.5812/ircmj.29933CrossrefMedlineGoogle Scholar
    • 29. Kuo FY, Huang WC, Chiou KR, Mar GY, Cheng CC, Chung CC, Tsai HL, Jiang CH, Wann SR, Lin SL, Liu CP. The effect of failure mode and effect analysis on reducing percutaneous coronary intervention hospital door-to-balloon time and mortality in ST segment elevation myocardial infarction.BMJ Qual Saf. 2013; 22:626–638. doi: 10.1136/bmjqs-2012-001288CrossrefMedlineGoogle Scholar
    • 30. Fonarow GC, Gawlinski A, Watson K. In-hospital initiation of cardiovascular protective therapies to improve treatment rates and clinical outcomes: the University of California-Los Angeles, Cardiovascular Hospitalization Atherosclerosis Management Program.Crit Pathw Cardiol. 2003; 2:61–70. doi: 10.1097/01.HPC.0000077071.32488.ecCrossrefMedlineGoogle Scholar
    • 31. Khot UN, Johnson ML, Ramsey C, Khot MB, Todd R, Shaikh SR, Berg WJ. Emergency department physician activation of the catheterization laboratory and immediate transfer to an immediately available catheterization laboratory reduce door-to-balloon time in ST-elevation myocardial infarction.Circulation. 2007; 116:67–76. doi: 10.1161/CIRCULATIONAHA.106.677401LinkGoogle Scholar
    • 32. Lai CL, Fan CM, Liao PC, Tsai KC, Yang CY, Chu SH, Chien KL. Impact of an audit program and other factors on door-to-balloon times in acute ST-elevation myocardial infarction patients destined for primary coronary intervention.Acad Emerg Med. 2009; 16:333–342. doi: 10.1111/j.1553-2712.2009.00372.xCrossrefMedlineGoogle Scholar
    • 33. Prabhakaran D, Jeemon P, Mohanan PP, Govindan U, Geevar Z, Chaturvedi V, Reddy KS. Management of acute coronary syndromes in secondary care settings in Kerala: impact of a quality improvement programme.Natl Med J India. 2008; 21:107–111.MedlineGoogle Scholar
    • 34. Scholz KH, Maier SK, Jung J, Fleischmann C, Werner GS, Olbrich HG, Ahlersmann D, Keating FK, Jacobshagen C, Moehlis H, Hilgers R, Maier LS. Reduction in treatment times through formalized data feedback: results from a prospective multicenter study of ST-segment elevation myocardial infarction.JACC Cardiovasc Interv. 2012; 5:848–857. doi: 10.1016/j.jcin.2012.04.012CrossrefMedlineGoogle Scholar
    • 35. Robinson MB, Thompson E, Black NA. Evaluation of the effectiveness of guidelines, audit and feedback: improving the use of intravenous thrombolysis in patients with suspected acute myocardial infarction.Int J Qual Health Care. 1996; 8:211–222. doi: 10.1093/intqhc/8.3.211CrossrefMedlineGoogle Scholar
    • 36. Scott IA, Eyeson-Annan ML, Huxley SL, West MJ. Optimising care of acute myocardial infarction: results of a regional quality improvement project.J Qual Clin Pract. 2000; 20:12–19.CrossrefMedlineGoogle Scholar
    • 37. Scott IA, Coory MD, Harper CM. The effects of quality improvement interventions on inhospital mortality after acute myocardial infarction.Med J Aust. 2001; 175:465–470.CrossrefMedlineGoogle Scholar
    • 38. Scott IA, Denaro CP, Hickey AC, Bennett C, Mudge AM, Sanders DC, Thiele J, Flores JL. Optimising care of acute coronary syndromes in three Australian hospitals.Int J Qual Health Care. 2004; 16:275–284. doi: 10.1093/intqhc/mzh051CrossrefMedlineGoogle Scholar
    • 39. Rowe AK, Rowe SY, Peters DH, Holloway KA, Chalker J, Ross-Degnan D. Effectiveness of strategies to improve health-care provider practices in low-income and middle-income countries: a systematic review.Lancet Glob Health. 2018; 6:e1163–e1175. doi: 10.1016/S2214-109X(18)30398-XCrossrefMedlineGoogle Scholar