Patient-Reported Outcomes in Cardiology: Comparison of Two Programs to Assess Angina Burden in Coronary Artery Disease
Circulation: Cardiovascular Quality and Outcomes
Goals and Vision of the Program
Angina predicts outcomes from coronary artery disease, including mortality, morbidity, reduced quality of life, and increased healthcare spending.1 However, patient symptoms are underreported and incompletely documented, limiting understanding of the efficacy of therapies.2 Angina can be reliably assessed using patient-reported outcomes measures (PROMs): short, well-validated questionnaires assessing symptoms and quality of life.3,4 However, implementation barriers continue to hinder routine use of PROMs. We describe two strategies designed to evaluate the feasibility of PROMs assessment. In strategy 1, we implemented a system for collecting the Seattle Angina Questionnaire-7 (SAQ-7), 7 questions evaluating angina; the Rose Dyspnea Scale, 4 questions evaluating dyspnea; and the Patient Health Questionnaire-2, 2 questions evaluating depression, via electronic self-administered surveys before elective coronary angiography at the Beth Israel Deaconess Medical Center, and at 3 follow-up times. In strategy 2, we created a system for phone-based administration of the SAQ-7 in CAD patients receiving care at most of Massachusetts General Hospital’s (MGH) primary care clinics and affiliated health centers.
Design and Implementation of the Initiative
Strategy 1
We conducted a pilot study, December 18, 2016, to January 19, 2018, administering web-based versions of the SAQ-7, the Rose Dyspnea Scale, and Patient Health Questionnaire-2 via tablets to patients presenting for nonemergent coronary angiography.5 Catheterization laboratory nurses and front-desk staff provided tablets to patients in the waiting room or preprocedure holding area. Tablets were configured to display the survey and a brief study explanation with minimal assistance. Responses were transmitted via WiFi to an encrypted Research Electronic Data Capture database and uploaded by a research coordinator (L. Valsdottir) as portable document format files to the electronic health record (EHR). A project request, currently pending, was submitted to the Beth Israel Deaconess Medical Center to create a workflow to automatically upload surveys. Patients desiring to use their own mobile device were given a survey link. Participants were automatically sent emails at 30 days, 6 months, and 12 months after the procedure with requests to complete the questionnaire again. Nonresponders were contacted by phone by a research coordinator (L. Valsdottir). Catheterization laboratory, nursing, and front-desk staff and clinicians (N=40) attended a 10-minute training by study staff before study initiation to learn about the goals of the initiative, their role in its conduct, and advice for troubleshooting (Figures I and II in the Data Supplement). Attendance was documented and efforts were made to contact those unable to attend. Nursing and administrative leadership were included in all planning meetings.
Strategy 2
We conducted a pilot study, February 1, 2017, to July 31, 2017, to assess the feasibility of administering the SAQ-7 to MGH primary care patients of age ≥30 years with CAD receiving care through 1 of 15 primary care clinics and community health centers. Patients were excluded if dementia was documented in the EHR, or they were deemed unable to complete a telephone survey due to dementia (ascertained based on caregiver or family report), hearing impairment, or language barrier.
We obtained a list of patients with CAD from the MGH Primary Care Practice Based Research Network, a database containing EHR data for 161 000 patients receiving primary care through the MGH network. SAQ-7 surveys were administered by trained population health coordinators (PHCs) as an extension of their current work. Seven full-time PHCs are employed by MGH, each working with 1 to 3 MGH primary care clinics, ensuring patients receive appropriate disease screening and reach disease targets. Our strategy was to expand the pool of PHCs so each would be able to commit time to survey administration weekly. We hired an eighth PHC who assumed responsibility for part of the PHCs’ workload, enabling each PHC to devote 6 hours weekly to survey administration.
Seven PHCs administered surveys in English, 1 in Spanish. One study investigator (Dr Blumenthal) administered up to 10 surveys weekly on evenings or weekends per patient request. PHCs were given the option to administer the SAQ-7 via secure email (Patient Gateway) per preference. PHCs were instructed to perform ≥1 follow-up call for patients not answering or returning a voice message within 24 hours. All responses were recorded in the EHR. We aimed to survey 50 patients from each of the 15 primary care clinics. We drafted scripts and protocols for contacting patients, administering surveys, and recording survey responses in the EHR, and trained PHCs in their use (Figures IV through X in the Data Supplement).
For patients reporting concerning symptoms, we developed a protocol to promptly identify them and contact a cardiologist to evaluate need for urgent medical evaluation. These included survey responses or informal patient reports of daily, severe, or progressive angina, dyspnea, increasingly frequent episodes of angina, or rest angina.
Success of the Initiative
Strategy 1
As of January 19, 2018, a total of 613 baseline surveys were collected on 1448 elective coronary angiography procedures (average response rate 42.3%). Of the 613 patients completing a baseline survey, 156 (25.4%) underwent percutaneous coronary intervention with the remainder undergoing coronary angiography alone (Figure III in the Data Supplement). Of those eligible for follow-up at the time of this report, 56.1% (331/590) completed the 30-day survey, 50.3% (186/370) completed the 6-month survey, and 60.6% (40/66) completed the 1-year survey. A total of 479 respondents (77.0%) provided an email address on the baseline survey, but only 27.3% (161/590) of participants completed a follow-up survey by email at 30 days, 23.5% (87/370) at 6 months, and 22.7% (15/66) at 1 year. The remaining surveys were conducted by phone, representing slightly more than half of completed follow-up surveys.
The main reason for noncompletion of follow-up surveys was lack of response to email and phone contact, accounting for 34.4% (203/590) of participants at 30 days, 36.5% (135/370) at 6 months, and 21.2% (14/66) at 1 year. The majority of other nonresponders either declined the survey (30 days: 12/590; 6 months: 16/370; 1 year: 4/66), did not speak English (30 days: 10/590; 6 months: 7/370; 1 year: 4/66), or provided an incorrect email address or phone number (30 days: 15/590; 6 months: 23/370; 1 year: 2/66). A minority were hard of hearing or cognitively impaired (30 days: 3/590; 6 months: 2/370; 1 year: 1/66), or had died (30 days: 2/590; 6 months: 1/370; 1 year: 1/66). Respondents were healthier than nonrespondents, and those with a positive stress test were more likely to respond to the survey (Table I in the Data Supplement). Thus, it is possible that expectation of need for PCI could have influenced participation.
Strategy 2
PHCs intended to survey 4789 CAD patients, but excluded 687 patients (14.3%) at the time of survey administration (553 [11.5%] because of no longer seeing an MGH-affiliated primary care physician, being in hospice, a nursing home, or hospitalized; 77 (1.6%) for hearing impairment; 53 (1.1%) for dementia; and 4 (0.1%) for difficulty with speech). Of the 4102 remaining individuals, 1612 (34%) completed the SAQ-7 (Table II in the Data Supplement). Of these, 1598 (99.1%) completed the SAQ-7 by phone and 14 (0.9%) used the Patient Gateway (Figure IV in the Data Supplement). Among the 2490 nonresponders, reasons for nonresponse included: 900 (28.3%) did not answer the phone; 888 (27.9%) did not return a voice message from the PHC; 408 (12.9%) refused or did not complete the survey; 213 (6.7%) requested follow-up at a later date but never completed the survey; and 37 (1.2%) did not respond to the SAQ-7 when administered via Patient Gateway. In 44 cases (1.4%), the EHR contained inaccurate contact information. We surveyed at least 50 patients for 14 of 15 participating primary care clinics. Median clinic-specific and PHC-specific survey completion rates were 34.9% (range: 16.7%–51.1%) and 35.2% (range: 9.3%–46.2%), respectively. Eighteen surveys (1.0%) met criteria for a concerning survey response. Follow-up within 1 week was arranged for 2 patients; none went to the emergency room.
Local Challenges in Implementation
Strategy 1
The most significant barriers were technological. Tablets required reconfiguration daily to ensure data privacy, making integration into workflow difficult. Additionally, patient disability or poor dexterity limited electronic PROMs completion. Patients were allowed to use their own device to complete surveys, and family members were encouraged to help, but many declined or elected to complete a paper version. Additionally, PROMs were only available in English, a barrier to non-English speakers.
Strategy 2
Contacting patients proved challenging, as many patients were busy during daytime hours. Language barriers also limited administration to non-English, non-Spanish speaking patients. Additionally, PHCs were not initially prepared to explain to asymptomatic patients reasons for completion, so some patients refused to complete the survey. PHCs were subsequently trained to provide an explanation to these patients, mitigating this barrier. The authors are considering further research to understand reasons for the differential response rates among PHCs.
Translation to Other Settings
Phone-based administration of PROMs may be particularly useful for evaluating symptoms longitudinally among several patient populations. Health systems participating in risk-based contracts including a mandate to assess patient-reported health status may also find it useful to develop web- or phone-based PROMs programs. Further comparison of web- versus phone-based strategies is warranted to elicit which strategies work best locally.
Summary of the Experience, Future Directions, and Challenges
We describe the motivation, design, implementation, and early outcomes of 2 complementary programs to evaluate the feasibility of collecting PROMs in the inpatient and outpatient setting among patients with CAD (Figure).

In strategy 1, PROMs were administered by tablet to patients undergoing coronary angiography. Response rates were reasonable (42.3%), and the innovation demonstrated that PROMs can be administered in a semi-automated fashion in a busy catheterization laboratory. Challenges included competing responsibilities among staff, hospital technological requirements that did not integrate well into workflow, and lack of incentives for participation. While feedback was given on low response rates to staff, the perception of increased workload was a sufficient barrier to implementation. Whether staff incentives for participation could improve response rates is unknown.
In strategy 2, the SAQ-7 was administered by phone to a population of primary care patients with stable CAD, with 2 findings: (1) the program’s 34% response rate compares favorably with the 15% to 20% response rates achieved when the SAQ-7 is administered via tablet device to patients during outpatient clinic appointments with MGH providers (unpublished data), and (2) the infrastructure for PROMs administration can be embedded within existing population health management programs. Challenges included contacting patients, responding promptly to concerning surveys, and off-hours administration. The human capital needed to administer PROMs by phone could substantially limit efforts to scale this approach.
Challenges common to both programs include integration into busy clinical workflows and engaging patients in follow-up. Both innovations confirmed higher success rates with phone contact compared with email, including in elderly, comorbid populations. These two innovations demonstrate the need to standardize workflow for PROMs administration and underscore the importance of providing training and support to administering personnel. Integrating PROMs into daily workflow may not be possible without increasing staff capacity to offset additional workload. Both innovations achieved low absolute response rates suggesting that full participation may not be realistic. Strategies for improving response rates include tailoring surveys to participants’ language and literacy levels, conducting interviews face-to-face, setting deadlines for participation, and using non-monetary incentives. Additionally, it is possible that iterative improvement through Plan-Do-Study-Act cycles may improve response rates in the future, similar to observational registries. Using a structured approach, such as the Reach-Effectiveness-Adoption-Implementation-Maintenance schematic (www.RE-AIM.org) could result in improved responses. While clinicians were encouraged to share PROMs with patients and use them for counseling, it is unclear how many clinicians chose to do so, and whether this was associated with increased follow-up response rates. Moreover, while the current projects were designed to evaluate the feasibility of PROMs collection, with appropriate training, routine integration of PROMs into clinical workflow as a vital sign may help to improve response rates and clinical utilization. Both MGH and the Beth Israel Deaconess Medical Center have plans to continue and modify existing programs through adding staffing and electronic work-arounds, but these interventions may not apply to all institutions. Until evidence-based strategies exist to improve response rates and overcome common barriers to PROMs administration, institutions are encouraged to develop pilot programs to start collecting PROMs to identify the most effective local strategies for administration. Through pooled knowledge and expertise, we can start to build the evidence base for extracting meaningful and reproducible information from our most important and underutilized resource, the patient.
Acknowledgments
The authors would like to acknowledge Kim Fong for his contribution to this work and the Centers for Healthcare Delivery Science at Beth Israel Deaconess Medical Center.
Supplemental Material
File (circcvqo_circcqo-2018-004794_supp1.pdf)
- Download
- 6.97 MB
References
1.
Arnold SV, Morrow DA, Lei Y, Cohen DJ, Mahoney EM, Braunwald E, Chan PS. Economic impact of angina after an acute coronary syndrome: insights from the MERLIN-TIMI 36 trial. Circ Cardiovasc Qual Outcomes. 2009;2:344–353. doi: 10.1161/CIRCOUTCOMES.108.829523
2.
Arnold SV, Grodzinsky A, Gosch KL, Kosiborod M, Jones PG, Breeding T, Towheed A, Beltrame J, Alexander KP, Spertus JA. Predictors of physician under-recognition of angina in outpatients with stable coronary artery disease. Circ Cardiovasc Qual Outcomes. 2016;9:554–559. doi: 10.1161/CIRCOUTCOMES.116.002781
3.
Spertus JA, Winder JA, Dewhurst TA, Deyo RA, Fihn SD. Monitoring the quality of life in patients with coronary artery disease. Am J Cardiol. 1994;74:1240–1244.
4.
Beltrame JF, Weekes AJ, Morgan C, Tavella R, Spertus JA. The prevalence of weekly angina among patients with chronic stable angina in primary care practices: the Coronary Artery Disease in General Practice (CADENCE) Study. Arch Intern Med. 2009;169:1491–1499. doi: 10.1001/archinternmed.2009.295
5.
McNamara RL, Spatz ES, Kelley TA, Stowell CJ, Beltrame J, Heidenreich P, Tresserras R, Jernberg T, Chua T, Morgan L, Panigrahi B, Rosas Ruiz A, Rumsfeld JS, Sadwin L, Schoeberl M, Shahian D, Weston C, Yeh R, Lewin J. Standardized Outcome Measurement for Patients with Coronary Artery Disease: Consensus From the International Consortium for Health Outcome Measurement (ICHOM). J Am Heart Assoc. 2015;4:e001767. doi: 10.1161/JAHA.115.001767
Information & Authors
Information
Published In
Circulation: Cardiovascular Quality and Outcomes
PubMed: 30571330
Copyright
© 2018 American Heart Association, Inc.
History
Published in print: November 2018
Published online: 5 November 2018
Keywords
Subjects
Authors
Disclosures
None.
Sources of Funding
Dr Blumenthal reports receiving funding for this work through a Population Health Management Innovation Pilot Award from Partners Healthcare and the John S. LaDue Memorial Fellowship at Harvard Medical School. Dr Yeh is funded by a grant from the National Heart, Lung, and Blood Institute (1R01HL136708-01). Dr Strom is funded by a grant from the American Heart Association (18CDA34110267). Dr Wasfy reports grant support from the National Institutes of Health and Harvard Catalyst (KL2 TR001100) and a grant from the American Heart Association (18CDA34110215). Dr Yeh reports additional support through an Innovation Grant from the Center for Healthcare Delivery Sciences at Beth Israel Deaconess Medical Center for the current project, as well as grant support from Boston Scientific and Abiomed, and consulting fees from Abbott, Medtronic, and Teleflex, outside the submitted work. Dr Wasfy is medical director for population health which includes work implementing patient-reported outcomes measures at the Massachusetts General Physicians Organization.
Metrics & Citations
Metrics
Citations
Download Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Select your manager software from the list below and click Download.
- Introduction of Patient-reported Outcome Measures in a Cardiac Surgery Center, The Thoracic and Cardiovascular Surgeon, (2025).https://doi.org/10.1055/a-2509-0430
- Manejo Invasivo versus Conservador de Pacientes com IAMSSST com Idade ≥ 75 Anos: Comentários, Arquivos Brasileiros de Cardiologia, 120, 12, (2023).https://doi.org/10.36660/abc.20230364
- Patient‐Reported Outcomes in Venous Thromboembolism: A Systematic Review of the Literature, Current Challenges, and Ways Forward, Journal of the American Heart Association, 12, 23, (2023)./doi/10.1161/JAHA.123.032146
- Temporal Trends and Factors Associated With the Inclusion of Patient‐Reported Outcomes in Heart Failure Randomized Controlled Trials: A Systematic Review, Journal of the American Heart Association, 10, 21, (2021)./doi/10.1161/JAHA.121.022353
- Relation of the Number of Cardiovascular Conditions and Short-term Symptom Improvement After Percutaneous Coronary Intervention for Stable Angina Pectoris, The American Journal of Cardiology, 155, (1-8), (2021).https://doi.org/10.1016/j.amjcard.2021.06.007
- Association of Syndemic Unhealthy Alcohol Use, Smoking, and Depressive Symptoms on Incident Cardiovascular Disease among Veterans With and Without HIV-Infection, AIDS and Behavior, 25, 9, (2852-2862), (2021).https://doi.org/10.1007/s10461-021-03327-4
- Prevalence of Angina Among Primary Care Patients With Coronary Artery Disease, JAMA Network Open, 4, 6, (e2112800), (2021).https://doi.org/10.1001/jamanetworkopen.2021.12800
- Strengthening the Learning Health System in Cardiovascular Disease Prevention: Time to Leverage Big Data and Digital Solutions, Current Atherosclerosis Reports, 23, 5, (2021).https://doi.org/10.1007/s11883-021-00916-5
- Assessing health-related quality of life among patients with peripheral artery disease: A review of the literature and focus on patient-reported outcome measures, Vascular Medicine, 26, 3, (317-325), (2020).https://doi.org/10.1177/1358863X20977016
- Association between procedure appropriateness and patient-reported outcomes after percutaneous coronary intervention, Heart, 106, 6, (441-446), (2019).https://doi.org/10.1136/heartjnl-2019-315835
- See more
Loading...
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Personal login Institutional LoginPurchase Options
Purchase this article to access the full text.
eLetters(0)
eLetters should relate to an article recently published in the journal and are not a forum for providing unpublished data. Comments are reviewed for appropriate use of tone and language. Comments are not peer-reviewed. Acceptable comments are posted to the journal website only. Comments are not published in an issue and are not indexed in PubMed. Comments should be no longer than 500 words and will only be posted online. References are limited to 10. Authors of the article cited in the comment will be invited to reply, as appropriate.
Comments and feedback on AHA/ASA Scientific Statements and Guidelines should be directed to the AHA/ASA Manuscript Oversight Committee via its Correspondence page.