Feasibility Platform for Stroke Studies: An Online Tool to Improve Eligibility Criteria for Clinical Trials
Abstract
Background and Purpose—
Eligibility criteria are a key factor for the feasibility and validity of clinical trials. We aimed to develop an online tool to assess the potential effect of inclusion and exclusion criteria on the proportion of patients eligible for an acute stroke trial.
Methods—
We identified relevant inclusion and exclusion criteria of acute stroke trials. Based on these criteria and using a cohort of 1537 consecutive patients with acute ischemic stroke from 3 stroke centers, we developed a web portal feasibility platform for stroke studies (FePASS) to estimate proportions of eligible patients for acute stroke trials. We applied the FePASS resource to calculate the proportion of patients eligible for 4 recent stroke studies.
Results—
Sixty-one eligibility criteria were derived from 30 trials on acute ischemic stroke. FePASS, publicly available at http://fepass.uni-muenster.de, displays the proportion of patients in percent to assess the effect of varying values of relevant eligibility criteria, for example, age, symptom onset time, National Institutes of Health Stroke Scale, and prestroke modified Rankin Scale, on this proportion. The proportion of eligible patients for 4 recent stroke studies ranged from 2.1% to 11.3%. Slight variations of the inclusion criteria could substantially increase the proportion of eligible patients.
Conclusions—
FePASS is an open access online resource to assess the effect of inclusion and exclusion criteria on the proportion of eligible patients for a stroke trial. FePASS can help to design stroke studies, optimize eligibility criteria, and to estimate the potential recruitment rate.
Introduction
Randomized controlled trials are necessary to evaluate the effectiveness of new stroke therapies.1 The patient population included in a randomized controlled trial is defined by its eligibility criteria.2 Restrictive inclusion criteria provide a homogeneous study population but also lead to difficulties in enrolling patients and thus increasing the trial’s duration and costs. Less stringent inclusion criteria improve patient accrual and the generalizability of the study results, but at the expense of study population homogeneity and statistical power. Eligibility criteria are therefore a key factor for the feasibility and validity of acute stroke trials as well as for demonstrating treatment effects. However, methods to optimize eligibility criteria and to reliably estimate the proportion of eligible patients are not available to date.
Our aim was to develop an open access online tool to assess the effect of inclusion and exclusion criteria on the proportion of patients eligible for a clinical stroke trial. Such resource could help to estimate the recruitment rate for a specific stroke study and could be used to improve stroke trial design with respect to optimizing eligibility criteria. At its initiation, the feasibility platform for stroke studies (FePASS) is based on a large cohort of acute stroke patients from 3 stroke centers. For a first application of FePASS, we estimated the proportion of patients eligible for recent stroke trials and assessed the effect of varying their major determinants for eligibility.
Methods
Identification of Relevant Inclusion and Exclusion Criteria
For identification of relevant inclusion and exclusion criteria, we systematically searched the databases Clinicaltrials.gov and The Internet Stroke Center (both searched in November 2009) for trials investigating treatments for acute ischemic stroke.3,4 The search strategy for Clinicaltrials.gov used the terms stroke for the search term Conditions, Open studies for the search term Recruitment, All studies for the search term Study Result, Interventional Studies for the search term Study Type, and Phase 2 and Phase 3 for the search term Phase. In The Internet Stroke Center, we searched interventional studies on acute ischemic stroke using the term Ongoing for the search term Status and the terms Phase 2 and Phase 3 for the search term Phase. We included studies if they investigated acute therapies, defined by treatment initiation within 24 hours after stroke onset.
Inclusion and exclusion criteria of the identified trials were extracted from Clinicaltrials.gov and The Internet Stroke Center. Some eligibility criteria were summarized, for example, on neurological disorders, that would confound the neurological evaluation. We waived eligibility criteria that were listed for only 1 of the included studies or that were only relevant for specific interventions, for example, a history of amyloid angiopathy (for a study on a thrombolytic agent) or allergy to the treatment under investigation.
Data Collection
We retrospectively analyzed data of all ischemic stroke patients admitted between April and May 2014 at the Inselspital, University of Bern, Switzerland, between December 2013 and June 2014 at the Western Infirmary, University of Glasgow, United Kingdom, and between December 2007 and December 2009 at the University Hospital of Münster, Germany (all participating hospitals are tertiary care centers). Transient ischemic attack patients were only included if they had continuing symptoms on admission. Data on variables derived from eligibility criteria identified as described above were obtained from the patients. Information gathered included sociodemographic characteristics, time of symptom onset, duration of symptoms, reperfusion therapy, mechanical ventilation, whether patient had a transient ischemic attack, infarct localization, and infarct size, whether patients had a stroke subtype additional to ischemic stroke, neurological symptoms, National Institutes of Health Stroke Scale (NIHSS) on admission, past medical history, comorbidities, laboratory findings, current medication, suitability for MRI, modified Rankin Scale before stroke, life expectancy before stroke, and pregnancy. Anonymized data were extracted from university servers with permission according to local regulations. The identity of the individual patients was completely anonymous. Therefore, no specific informed consent was needed from the patients.
Development of the FePASS Online Resource
Based on the characteristics of relevant eligibility criteria that were identified, we developed a text-based data structure to formalize a flexible representation of eligibility criteria in general. This data structure is used to interpret, represent, and evaluate the given criteria electronically. To support the patient data import, an Excel template was defined and provided. This schema can be processed by the system and transforms patient data into the given representation of eligibility criteria. On the basis of the given data structure, the underlying relational database can be set up easily. The relational database stores the patient data, which was imported via Excel, and provides a short response time for querying. Finally, we developed the web portal FePASS that uses the representation of eligibility criteria, presents them to the user, and queries the user’s combination of criteria and values to the database.
Estimation of the Proportion of Patients Eligible for Recent Trials and Effect of Eligibility Criteria Variation
We used the FePASS resource to calculate the proportion of patients eligible for 2 recently completed (Albumin in Acute Ischemic Stroke Trial [ALIAS] part 2, AX200 for the Treatment of Ischemic Stroke 2 [AXIS 2]) and 2 ongoing (Cooling Plus Best Medical Treatment Versus Best Medical Treatment Alone for Acute Ischaemic Stroke [EuroHYP-1], Solitaire FR as Primary Treatment for Acute Ischemic Stroke [SWIFT PRIME]) trials on acute ischemic stroke based on their inclusion and exclusion criteria. Because of later initiation, EuroHYP-1 and SWIFT PRIME were not among the trials whose inclusion and exclusion criteria were used for the development of the FePASS resource. We calculated the coverage of eligibility criteria for these 2 trials by FePASS. We then used the variability-of-values feature of FePASS to assess the effect of varying values of relevant eligibility criteria on the proportion of suitable patients for the 4 trials.
Statistical Analyses
The system’s underlying database is based on first-order logic, which only uses quantified variables. In particular, only Boolean values (true or false), finite numbers and dates without any variation are used to determine a patient’s attribute’s presence or value. Hence, FePASS determines the exact total number of eligible patients by iterating through both the list of eligibility criteria and patients in the database. For each pair of eligibility criteria and patient attribute, it checks whether the attribute fits to the given criterion. If it does, the next pair will be checked. If they do not match, the patient is evaluated as not eligible. Only if all pairs of criteria and attributes match, the patient is considered eligible and is added to the sum of eligible patients. Statistical variations do not occur when using FePASS on the same database. Variations to other cohorts can be derived independently by using the FePASS’s results and this publication’s cohort description as given in Table and adjusting it to the local cohort.
| Patients With Ischemic Stroke or TIA | |
|---|---|
| Demographics | |
| Age, mean (SD), y | 69.1 (14.4) |
| Women, n (%) | 696 (45.3) |
| NIHSS, median (IQR) | 3 (7) |
| Symptom onset | |
| Time from symptom onset to admission (mean±SD), h | 4.22 (5.32) |
| Time from last known well time to admission (mean±SD), h | 11.83 (7.80) |
| Time of symptom onset unknown, n (%) | 409 (26.6) |
| Reperfusion therapy, n (%) | |
| Intravenous thrombolysis | 225 (14.6) |
| Intra-arterial thrombolysis | 66 (4.3) |
| Mechanical intervention | 46 (3.0) |
| TIA, n (%) | 239 (15.5) |
| mRS preceding the stroke, n (%) | |
| 0 | 752 (48.9) |
| 1 | 306 (19.9) |
| 2 | 170 (11.1) |
| 3 | 125 (8.1) |
| 4 | 162 (10.5) |
| 5 | 14 (0.9) |
The number for mRS does not sum to column total because of missings. mRS indicates modified Rankin Scale; NIHSS, National Institute of Health Stroke Scale; and TIA, transient ischemic attack.
To calculate the potential increase of eligible patients for each of the 4 analyzed trials, we conducted the following strategy: (1) map the trial’s eligibility criteria as exactly as possible to the list of criteria offered by FePASS; (2) calculate the number of eligible patients; (3) apply variations to certain criteria, such as age, NIHSS, or modified Rankin Scale; (4) recalculate the exact number of eligible patients; and (5) divide absolute numbers to determine the proportional change caused by criteria variations.
Results
The search strategy yielded 30 acute ischemic stroke studies (Table I in the online-only Data Supplement). According to the information given by Clinicaltrials.gov and The Internet Stroke Center, 26 of these studies used a specific age range as inclusion criterion. The maximum allowed time from symptom was specified for all studies. Twenty-two studies used the NIHSS to limit initial symptom severity. Pre-stroke disability was restricted by 20 (66.7%) studies. Infarct localization and infarct size were used as eligibility criteria by 10 (33.3%) and 11 (36.7%) studies, respectively. In 12 (40%) studies, cotreatment with thrombolysis was allowed.
Overall, a total of 1537 patients were included in the database. Main characteristics of the patients are given in Table. Mean age of patients was 69.1 years; 696 (45.3%) were women. Two hundred and thirty-nine patients (15.5%) had a transient ischemic attack. Data on variables derived from eligibility criteria were >99% complete. Exceptions were NIHSS and derived variables (missing 2.9%), international normalized ratio (missing 4.0%), and partial thromboplastin time (missing 3.0%).
The developed data structure is described with the text-based format JavaScript Object Notation,5 which represents data in key-value pairs in a standardized way. Thus, the data structure is both human-readable and electronically processable. In the data structure, eligibility criteria are aggregated into 16 groups, each of them addressing another medical context, for example, demographic data, Infarct localization and size, or Past medical procedures (Table I in the online-only Data Supplement). Each of the 61 eligibility criteria is mapped to either a date or a numeric value, where dates are used to collect, present, and query a range of time for the patient’s submission and numbers represent Boolean values (0=false/no and 1=true/yes) or discrete values, such as age or NIHSS. Additionally, all criteria are configured to be combinable (eg, to search for male or female patients or both genders) or exclusive (eg, to choose to have the condition Patient has had a previous stroke as an inclusion or exclusion criterion).
The web portal, publicly available at http://fepass.uni-muenster.de, presents the 16 criteria groups in an accordion-like interface (Figure 1). The user is able to open each group and choose the options that determine a patient’s eligibility. Depending on the underlying data, either the (non)existence of a patient characteristic or specific values or thresholds can be selected to include or exclude patients meeting these conditions. All criteria ignored by the user are considered not relevant for the feasibility analysis.
By pressing the search button, FePASS directly collects the chosen eligibility criteria (values), queries the database, and presents the results instantaneously. The portal displays the ratio of patients found eligible in per cent, based on the current data set in the database. A Details panel in the upper right corner offers a detailed analysis of the adjustable criteria, which currently are age, symptom onset time, NIHSS, and modified Rankin Scale before the stroke (Figure 1B). Because the user is not only able to set a value range for these criteria but to adjust the threshold of both minimum and maximum values, the detailed analysis presents the outcome for these adjustments directly for each step in a tabular view. For example, if patients between 40 and 60 years of age are eligible and the adjustment allows moving downwards by 10 and upwards by 5, the detailed view shows query outcomes for all combinations of a minimum age from 30 to 40 years and a maximum age from 60 to 65 years.
A database query is generated by combining the user’s choices. If a criterion has been picked as an inclusion criterion, the respective datum item in the database has to be true or 1 to determine a patient eligible. It has to be false or 0 for exclusion criteria, respectively. Regarding criteria with discrete values (eg, age), a patient is determined eligible if his/her respective datum item is within the given range. The combination of all criteria is run against the database to determine the eligibility of all patients and calculate the ratio of eligible patients. Criteria flagged as no restriction are not considered. By default, all criteria are set to no restriction, which leads to 100% of the patients being eligible in the beginning.
We applied FePASS to 4 recent stroke trials. The proportions of trial-eligible patients were 4.3%, 3.3%, 11.3%, and 2.1% for ALIAS part 2, AXIS 2, EuroHYP-1, and SWIFT PRIME, respectively. The initial protocol of AXIS 2 was amended during the study period. The NIHSS entry limit was lowered from 8 to 6, and patients with previous thrombolysis were allowed to participate. In the FePASS cohort, these changes lead to a relative increase of the proportion of eligible patients by 50% (absolute increase from 2.2% to 3.3%).
FePASS covered 16 (40%) of 40 eligibility criteria of the EuroHYP-1 trial and 26 (63%) of 41 criteria of the SWIFT PRIME trial. Variation of the upper age limit by 3 years increased the proportion of eligible patients per analyzed trial between 3.03% and 4.65% as shown in Figure 2A. Figure 2B shows the extension of the therapeutic time window. A variation by 1 hour increases in the proportion of eligible patients by 2.33% to 3.54%. NIHSS variation by 3 points increases the proportion of eligible patients by 42.86% to 139.39% and variation of the pre-stroke modified Rankin Scale by 2 points by 16.28% to 36.28%. Combination of all variations increased the proportions between 76% and 200% per trial (SWIFT PRIME, 76%; EuroHyp1, 104%; Axis 2, 200%; Alias Part 2, 195%).
Discussion
We developed FePASS, an open access online resource to estimate for the proportion of stroke patients who would meet specific trial’s eligibility criteria. The underlying data structure allows modifications of inclusion and exclusion criteria according to changes of eligibility criteria of future trials. In addition, further stroke patient cohorts can be added easily to the database of FePASS. By applying FePASS to recent clinical trials, we estimated that the proportion of patients eligible for participation in these trials ranges between 2.1% and 11.3%. Even slight variations of inclusion criteria led to significant changes in these proportions.
FePASS allows to estimate the number of patients who can be recruited to a given study. The determination of eligibility criteria for a planned trial certainly depends on the treatment that will be investigated, for example, the maximum time from symptom onset to treatment initiation for recanalization therapies.6 However, knowledge of the effect of inclusion criteria on the proportion of eligible patients is important for choosing the number of centers for a trial and for determining the expected duration. Moreover, minor variation of a range of inclusion criteria can increase the proportion of eligible patients without decreasing the power of a study to demonstrate efficacy.
Difficulties in patient recruitment to large clinical stroke trials are well described.2 For instance, in the recently published AXIS 2 trial, the inclusion criteria were changed soon after recruitment of the first 21 patients as a result of slow recruitment.7 A meta-analysis on a large number of stroke studies showed that recruitment rates for acute stroke trials are significantly influenced by the maximum allowable time from stroke onset to study enrollment and by the exclusion of patients with mild stroke.2 Based on a cohort of 1322 stroke patients, Taylor et al developed regression curves relating the proportion of patients within each of 3 eligibility criteria (NIHSS, age, and time to treatment).8 The model demonstrated that lowering the baseline limits of the NIHSS and extending the treatment window would have a great effect on the proportion of patients eligible for a stroke trial. However, most patients are ineligible for a trial because of >1 criterion violation.9 Thus, modification of a range of criteria is necessary to change the proportion of eligible patients, as enabled by FePASS. The same approach as used for FePASS was recently developed for estimating the proportion of patients suitable for studies on intracerebral hemorrhage.10
Our study has strengths and limitations. The identified inclusion and exclusion criteria are derived from a reasonable number of clinical phase II and III studies, thus covering the most relevant eligibility criteria. In addition, FePASS is based on a cohort of a large number of unselected and consecutively admitted patients. However, despite this large sample, its scope might be limited because patient data are derived from only 3 tertiary care stroke centers. Nevertheless, the current FePASS cohort has a relatively good generalizability indicated by a similar distribution of patient characteristics compared with large, population-based cohorts.11,12 A further strength of the FePASS resource is that it is open access.
Conclusions
We have developed an open access online resource, FePASS, that enables assessment for the effect of inclusion and exclusion criteria on the proportion of patients eligible for an acute stroke trial. FePASS can help to design stroke studies, optimize eligibility criteria, and estimate the potential recruitment rate.
Supplemental Material
File (str_stroke-2014-007124_supp1.pdf)
- Download
- 70.29 KB
References
1.
Kidwell CS, Liebeskind DS, Starkman S, Saver JL. Trends in acute ischemic stroke trials through the 20th century. Stroke. 2001;32:1349–1359.
2.
Elkins JS, Khatabi T, Fung L, Rootenberg J, Johnston SC. Recruiting subjects for acute stroke trials: a meta-analysis. Stroke. 2006;37:123–128.
3.
ClinicalTrials.gov. A service of the U.S. National Institutes of Health. http://clinicaltrials.gov. Accessed December 4, 2013.
4.
The Internet Stroke Center. http://www.strokecenter.org/trials. Accessed December 4, 2013.
5.
Standard ECMA-262. http://www.ecma-international.org/publications/standards/Ecma-262.htm. Accessed January 29, 2014.
6.
Hacke W, Kaste M, Bluhmki E, Brozman M, Dávalos A, Guidetti D, et al.; ECASS Investigators. Thrombolysis with alteplase 3 to 4.5 hours after acute ischemic stroke. N Engl J Med. 2008;359:1317–1329.
7.
Ringelstein EB, Thijs V, Norrving B, Chamorro A, Aichner F, Grond M, et al.; AXIS 2 Investigators. Granulocyte colony-stimulating factor in patients with acute ischemic stroke: results of the AX200 for Ischemic Stroke trial. Stroke. 2013;44:2681–2687.
8.
Taylor A, Castle A, Merino JG, Hsia A, Kidwell CS, Warach S. Optimizing stroke clinical trial design: estimating the proportion of eligible patients. Stroke. 2010;41:2236–2238.
9.
LaRue LJ, Alter M, Traven ND, Sterman AB, Sobel E, Kleiner J. Acute stroke therapy trials: problems in patient accrual. Stroke. 1988;19:950–954.
10.
Fonville AF, Samarasekera N, Hutchison A, Perry D, Roos YB, Al-Shahi Salman R. Eligibility for randomized trials of treatments specifically for intracerebral hemorrhage: community-based study. Stroke. 2013;44:2729–2734.
11.
Xian Y, Fonarow GC, Reeves MJ, Webb LE, Blevins J, Demyanenko VS, et al. Data quality in the American Heart Association Get With The Guidelines-Stroke (GWTG-Stroke): results from a national data validation audit. Am Heart J. 2012;163:392–398, 398.e1.
12.
Schwamm LH, Ali SF, Reeves MJ, Smith EE, Saver JL, Messe S, et al. Temporal trends in patient characteristics and treatment with intravenous thrombolysis among acute ischemic stroke patients at Get With The Guidelines-Stroke hospitals. Circ Cardiovasc Qual Outcomes. 2013;6:543–549.
Information & Authors
Information
Published In
Copyright
© 2014 American Heart Association, Inc.
Versions
You are viewing the most recent version of this article.
History
Received: 16 August 2014
Revision received: 3 October 2014
Accepted: 9 October 2014
Published online: 13 November 2014
Published in print: January 2015
Keywords
Subjects
Authors
Disclosures
Dr Schäbitz is an inventor of patent applications claiming the use of granulocyte colony-stimulating factor for the treatment of stroke.
Metrics & Citations
Metrics
Citations
Download Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Select your manager software from the list below and click Download.
- Development of dual-mode ELISA based on ALP-catalyzed APP hydrolysis for IL-6 detection, Journal of Pharmaceutical and Biomedical Analysis, 236, (115754), (2023).https://doi.org/10.1016/j.jpba.2023.115754
- Clinical trial recruiters’ experiences working with trial eligibility criteria: results of an exploratory, cross-sectional, online survey in the UK, Trials, 22, 1, (2021).https://doi.org/10.1186/s13063-021-05723-6
- Factors influencing harmonized health data collection, sharing and linkage in Denmark and Switzerland: A systematic review, PLOS ONE, 14, 12, (e0226015), (2019).https://doi.org/10.1371/journal.pone.0226015
- Prevalence of Imaging Biomarkers to Guide the Planning of Acute Stroke Reperfusion Trials, Stroke, 48, 6, (1675-1677), (2017)./doi/10.1161/STROKEAHA.117.016759
- User Satisfaction Evaluation of the EHR4CR Query Builder: A Multisite Patient Count Cohort System, BioMed Research International, 2015, (1-11), (2015).https://doi.org/10.1155/2015/801436
Loading...
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Personal login Institutional LoginPurchase Options
Purchase this article to access the full text.
eLetters(0)
eLetters should relate to an article recently published in the journal and are not a forum for providing unpublished data. Comments are reviewed for appropriate use of tone and language. Comments are not peer-reviewed. Acceptable comments are posted to the journal website only. Comments are not published in an issue and are not indexed in PubMed. Comments should be no longer than 500 words and will only be posted online. References are limited to 10. Authors of the article cited in the comment will be invited to reply, as appropriate.
Comments and feedback on AHA/ASA Scientific Statements and Guidelines should be directed to the AHA/ASA Manuscript Oversight Committee via its Correspondence page.