Frequency and Determinants of Nonpublication of Research in the Stroke Literature

Data were collected on all abstracts accepted for presentation at the year 2000 ISC meeting, using a standardized form. This information included type of presentation (platform or poster), regions of origin for authors, pharmaceutical or industry sponsorship, sample size, ischemic versus hemorrhagic stroke, whether a treatment was studied, whether a treatment was standard of care (ie, intravenous tissue plasminogen activator or aspirin) or investigational, type of study (observational, clinical trial, experimental, etc), mode of data collection (prospective or retrospective), imaging modalities used, clinical outcome measures, and laboratory outcomes.

There is no standardized definition of a positive result across the broad range of study types in this analysis. We modified a commonly used definition⁴: results were positive if they met the following criteria: (1) the studied variable produced beneficial results, or supported the abstract hypothesis or objective, and either (2a) the probability value for the variable effect was reported as <0.05, or (2b) no test of statistical significance was reported regarding the variable effect.

The National Library of Medicine PubMed database was searched in August 2005 for evidence of subsequent publication of abstracts in full-manuscript form. Searches used pertinent abstract keywords and first, second, or last author names. Data on publication date, journal, and reference were collected. Time to manuscript publication was calculated relative to the February 2000 date of the ISC.

For all abstracts which were either not located in PubMed, or for which it was not clear whether the publication found related directly to the abstract, a standardized questionnaire was emailed to the abstract authors. Author contact information was obtained first by searching scholarly databases for other listed publications by abstract authors (names cross-verified with affiliations), by searching the American Academy of Neurology 2005 membership address book, and by using the Google search engine. If an email address was not available for the primary author, the email address of the second or last author on the abstract was used. Questionnaires were designed to confirm/disconfirm nonpublication and to determine whether full-length publication was planned or no longer planned.

The individual impact of each independent predictor variable was measured by Fisher Exact or χ² to compute probability values. Fifteen variables were evaluated in a backward logistic regression to determine which factors influence publication rates. In the multivariate analysis, candidate variables entered into the model using a liberal P<0.15 criterion.

A total of 353 abstracts were presented at the 2000 ISC and published in the January 2000 issue of Stroke: Journal of the American Heart Association. Of these, 108 were oral presentations and 245 were posters. On initial search of the PubMed database, 189 abstracts were identified as published as full articles. An additional 31 full articles were identified as published via questionnaires returned by the authors for a total of 202 published articles (62.3% of total).

The frequency of publication of abstracts for each factor studied for association with publication is listed in Table 1. Publication rates were greater for oral abstracts than posters (75.7% versus 56.3%, P=0.02). Publication rates varied substantially among different topic categories (P=0.03). The category “results of recent clinical trials” had the highest rate of publication, 100%, and “community/risk factors” the lowest, 48%.

Positive results were reported in 220/353 (62.3%) by the predefined criteria. Presentation of positive results did not affect publication rate (64.0% versus 59.4%, P=0.39). Positive results were not a factor in publication rate when controlling for type of presentation (oral 79.7% versus 69.8%, P=0.24; poster 57.5% versus 54.1%, P=0.61).

The majority of abstracts (71%) had a listed author affiliated with a United States institution. In addition 17% of abstracts had an author with European affiliation, 10% with Asian/Australian affiliation, 8% with Canadian affiliation and 0.3% with a South/Central American affiliation. Although author region was not associated with publication rate on univariate analysis, abstracts with European authors were more likely to be published relative to those with a US author in the multivariate model (odds ratio 2.2, 95% CI, 1.1 to 4.4).

The majority of abstracts had university affiliation (269/353, 76.2%). Abstracts with university affiliation tended to be published more frequently than those without (64.7% versus 54.8%, P=0.10). The 29 abstracts pertaining to novel therapies were published at a higher rate than those on other topics (82.8% versus 60.5%, P=0.018). Factors with no univariate association with publication rates were: sample size, prospective design, retrospective design, primarily ischemic stroke, primarily hemorrhagic stroke, multicenter collaboration, and industry sponsorship.

Abstracts were published as articles in 57 different journals. The most common journals of publication are listed in Table 2. The median time to publication from presentation was 15 months (interquartile range 7 to 27, range: 18 to 69 months; Figure). Among fully published studies, 73% were published within 2 years of the abstract presentation. Time to full length publication did not differ between posters and oral presentations (14 months versus 16 months, P=0.451).

Multivariate logistic regression analysis (Table 3) revealed that factors independently associated with increased likelihood of full length publication were oral presentation, university affiliation, and European region of origin. Community risk factors as a presentation category was independently associated with reduced likelihood of full-length publication. A trend to association with increased likelihood of full-length publication was seen for Canadian region of origin, presentation category of vascular pathophysiology and new therapy as study topic. Trends to association with reduced odds of full-length publication were observed for presentation categories of diagnosis and recovery.

Among the 161 abstracts initially identified as unpublished or without clear evidence of publication after initial PubMed database search, author email address was identified in 157 of 161. Repeated attempts were made to contact study authors with a standardized questionnaire to determine the fate of these abstracts. At least 1 author of 105/157 (66.8%) responded to the survey (Table 4). Based on author responses, 31 additional abstracts were determined to have been published and 74 were confirmed as not published. Of the 74 confirmed unpublished abstracts, 73% were never submitted for publication, 16% were submitted and rejected, and 11% were in various stages of submission, review, and revision. The commonly reported reasons for nonpublication are shown in Table 4 and included lack of time, lack of coauthor participation, on-going study, methodological limitations, other publications in the field, and low priority given to publication.

We found that 1 of every 3 abstracts presented at the 2000 ISC was not published as a full manuscript within 5 years. This nonpublication rate is relatively low compared with rates found in several other subspecialties (supplemental Table I, available online at http://stroke.ahajournals.org). A recent meta-analysis has shown an overall nonpublication rate of 56% for abstracts presented at academic meetings.¹ The relatively lower rate of nonpublication for ISC stroke abstracts may reflect the high scientific quality of the conference, but may also in part reflect a high survey respondent rate bolstered by our repeated attempts to contact study authors to confirm full article publication.

Poster abstract presentations were less likely to be published than oral presentations, which is consistent with findings from prior studies.¹ This difference likely reflects the fact that oral presentation abstracts were deemed by meeting peer reviewers to have the greatest relevance and scientific merit, as well as the overall greater quantity of poster abstracts compared with oral abstracts at scientific meetings.

Publication rates of abstracts varied by presentation category with vascular pathophysiology, experimental ischemia, and recent clinical trials abstracts published at high rates and community risk factors, recovery, and diagnosis abstracts at lower rates. A variety of factors likely underlie these differences, including greater competing duties for clinical than basic scientist researchers and a privileging of therapeutic over population and diagnostic research in the clinical research community.

The majority of abstracts were published as full articles within 2 years of presentation. Of note, however, publication of abstracts from the 2000 ISC continued through the 2005 year. At the time of the writing of this article, there were an additional 8 abstracts in submission, revision, or prepublication designation according to author survey results, so the final conversion to full-length publication rate among this corpus of studies may increase slightly by the end of the decade.

This study did not identify positive-outcome bias across the broad class of stroke publications studied. Positive-outcome bias refers to the preferential publication of research with positive outcomes.⁴ The lack of a positive-outcome bias in this study likely in part reflects study of only accepted abstracts and not submitted abstracts. It could well be that a positive-outcome bias may exist in the selection of abstracts for presentation to the ISC. Positive-outcome bias has been documented in acceptance rates of abstracts to national meetings in prior publications.^4–6 In addition, lack of a positive-outcome bias may in part reflect the broad topic range of studies analyzed. Positive-outcome bias likely is more pronounced in therapeutic than observational, population, and basic science studies.

This study identified several barriers to full publication of abstracts. The most commonly cited reason for nonpublication of an abstract was lack of time, which probably reflects the pressures of clinical practice and obtaining ongoing competitive funding for research. Other important barriers to publication were the low priority given to publication by authors and a lack of coauthor participation in drafting manuscripts. Only a minority of authors attributed the nonpublication of their abstracts to methodological weaknesses.

This study has limitations. Data were collected only for accepted abstracts and do not reflect those abstracts submitted for presentation and rejected. Although the PUBMED database was thoroughly searched for evidence of publication, publications in nonindexed journals may have been mistakenly coded as unpublished. There were a small number of abstracts for which confirmation of nonpublication could not be elicited from authors and the actual publication rates may be slightly higher than reported. Because searches were performed based on first, second, and senior author names, it is conceivable that data presented in ISC abstracts was published by other coauthors, although we suspect that this happened infrequently. The study did not address the number of submissions needed to achieve final publication for individual abstracts.

In conclusion, although the overall rate of nonpublication of stroke abstracts presented at the 2000 ISC was comparatively low, there is certainly room for improvement because selective publication of abstracts has the potential to lead to bias in the stroke literature.

References ^{7 through 23} are cited in the supplemental Table I.

None.