The editorial process for medical journals: I. Introduction of a series and discussion of the responsibilities of editors, authors, and reviewers☆

Editor responsibility 

The majority of the editors of peer-reviewed clinical journals, including many ophthalmology journal editors, are practicing clinicians with on the job training in editorial matters. While most have served on editorial boards and have a demonstrated grasp of the major tenets of editorship, editors are frequently selected for their clinical, academic or organizational abilities rather than their editorial, ability.1 Publishers might consider requiring and even providing formal editorial training for the editors of their journals.

The editor should hold paramount the interests of the particular journal’s readers, interests not necessarily the same as those of the authors or reviewers. Editors also have to represent patients, especially in matters pertaining to therapy. Publication of clinical research in peer-reviewed journals provides the basis for most treatment decisions. Editors are responsible for monitoring the publication process and for encouraging and validating that the data are gathered and presented in an objective manner. The importance of this responsibility is particularly apparent in the case of clinical trials, which are powerful tools that are extremely important in improving the standard of care. In select circumstances, however, the study design or the results may be manipulated to produce in substantial financial gain or fame for an individual or a sponsor.2 The present journal publication process has evolved to the point where some medical journals may have the capability of controlling medical information and therefore may potentially act to restrict free flow of valid concepts or data. Nonetheless, the editors have the responsibility of providing a reasonable filter along the medical information highway.

The increases in the volume of biomedical research and the tremendous growth of new fields of knowledge have spurred a profusion of authors and journals. Researchers in new areas utilize these specialty journals both as an avenue for exchange of information and as a method of defining their new or revamped discipline. This exponential proliferation of scientific information, resulting in part from the availability of government and pharmaceutical company funding, combined with pressure to publish to secure promotion and tenure, provides ample opportunity to publish redundant or inconsequential research.

Editors rely on peer review to help them choose among the many manuscripts competing for limited space in print journals. The editor usually consults at least two reviewers but still exercises independent judgment on the acceptability of papers. Reviewers make recommendations that carry weight, but these are not always decisive. According to an analysis by Wilkes and Kravitz,3 half of journal editors surveyed rely almost exclusively on reviewer recommendations when making acceptance decisions. Although critical reviews enhance journal quality, it is not possible for the editor to fully know the capabilities or potential biases of each reviewer. Most journals do not train reviewers or assess their background in research methodology or critical literature review.4 Publishers might consider working with editors in developing tools to train reviewers and measure specific reviewer tasks or talents.

The actual peer review process of journals is frequently not clarified for the authors. The process of peer review for each journal should be stated in the Instructions for Authors to expose the process. All the major ophthalmology journals provide this information. The author, who has always spent substantial effort in the preparation of a manuscript, should usually receive a publication decision that includes specific comments about the manuscript. Journals should establish or clearly state their appeal process for authors who are dissatisfied with the decision outcome.

Editors acknowledge that peer review is an advanced form of editing and that peer review is not a scientific process. Even the most careful and competent peer review cannot certify the validity of all of a manuscript’s claims. Not all papers rejected after peer review are of inferior quality, as many authors will attest. Each journal has different priorities (and priorities are continually changing, sometimes based on the backlog of manuscripts). Each manuscript is handled independently, with the result that the editor may decide to reject a technically good manuscript because of the perceived lack of interest to the journals readership, insufficient new information, or redundancy related to other manuscripts in the literature or in the present peer review process of that journal.

Author responsibility 

Although manuscripts are occasionally invited or commissioned before submission, most scientific medical specialty journals primarily rely upon authors to take the initiative in submitting manuscripts for publication. As there are more than 25,000 biomedical journals world wide, most submitted manuscripts may ultimately be published somewhere if authors persist.

There are data to confirm that many published reports, including those describing some randomized clinical trials, have significant flaws.5 Other authoritative reviews conclude that many of the frequently cited reports in the current literature are of inferior scientific quality.6 Approximately 60% of randomized clinical trials that were rejected by the journal Ophthalmology during one survey period were subsequently published in different journals within 3 years (Donald Minckler, unpublished data). Clinicians with insufficient research training frequently perform or direct the studies that result in manuscripts. Many clinicians lack knowledge of epidemiology and biostatistics and do not have access to consultants. Therefore, the quality of statistical analysis is often inadequate. The most specific errors in statistical analysis relate to continued lack of sample size calculations, lack of a description of the method of randomization, and lack of confidence intervals.

Editors recognize that some methodological problems in submitted manuscripts cannot be avoided. They also recognize that some manuscripts provide information that is not conclusive but rather is the best that could be expected under the circumstances; the editor may determine that these manuscripts deserve publication, despite these deficiencies. But it is the responsibility of the author to recognize the limitations of the study, identify them for the reader and reviewer, usually in the discussion, and explain how the limitations affect the interpretation of the data. To ignore these responsibilities is both a scientific and ethical mistake.

Before each submission authors should review the Uniform Requirements Of Manuscripts Submitted To Biomedical Journals, as it covers a broad range of issues that could enhance each manuscript.7 These are available at many journal web sites and additionally several journals have study design worksheets to assist this process. Each journal, however, has its own tradition and style; therefore, the Instructions for Authors will vary among the journals but usually with minor variations from these uniform requirements.

Prospective studies, initiating with a hypothesis and appropriate comparison controls, are essential for more definitive patient care decisions. Fortunately they are becoming more frequent, although the numbers and percentages still remain very low.8 The development of guidelines, such as for randomized control trial studies,9 systematic reviews and meta-analyses of randomized control trials,10 observational studies,11 and studies of diagnostic tests (available at http://www.consort-statement.org/stardstatement.htm; accessibility verified November 28, 2002) are valuable additions for authors, editors, readers, and the public. Authors are being required to observe the guidelines in submissions to some journals. The Consolidated Standards of Reporting Trials (CONSORT) helps authors to improve the quality of reporting since it requires that authors themselves complete a checklist on reporting quality.12

To comprehend the results of a randomized controlled trial, the author (and reader) must understand its design, conduct, analysis, validity and interpretation, and understand the concepts of CONSORT. The Quality Of Reporting of Meta-Analyses (QUOROM) statement is similar to the CONSORT in that it has a statement, an itemized list, and flow chart.10 Although the standards of reporting randomized controlled trials have been well established internationally for almost 25 years,13 key study elements continue to be omitted by authors, which hampers interpretation and systematic review of randomized controlled trials.14 Some journals, including the American Journal of Ophthalmology at http://www.ajo.com/, offer the opportunity (usually on a web site) for authors to provide supplemental material, such as raw data used in medical research, as a means of permitting careful rigorous scrutiny.

There are several available resources in articles, book chapters, and even on the Internet that address the basic strategies necessary for producing a better manuscript. In particular the author must consider the target audience, the mission of the journal, and adhere closely to the journal’s Instruction for Authors. The study described in the manuscript must be a well-designed and executed investigation; failure in this area is the main reason for rejection. If the sample size is too small or the study design if poorly conceived, the author should anticipate rejection. The authors should not ignore other published works pertinent to the conclusions in their manuscript. The clarity and style of writing are extremely important; authors should write to inform the reader rather than to impress the reader, using prose and grammar that is both accurate and concise.

Institutional Review Boards, some of which are concerned more recently about the ethics of studies and patient safety than scientific quality, often fail to assist the author in detecting important flaws in study design.15 Statistical review of all manuscripts during the peer review process is a protracted and expensive task, not usually performed by most journals on every manuscript that uses statistics. The introduction of the structured abstract, including design, analysis, and interpretation of results, makes authors codify the elements of the study but cannot ensure proper methodology and data interpretation. Readers who depend on the abstract alone, however, will certainly miss substantial information.

Faulty methods utilized in previously published studies tend to be perpetuated in the literature, since some researchers may use the same (inappropriate) methods in future comparison studies. Once published, a letter to the editor may prove inadequate to correct misinformation that may be identified and discussed by astute readers. Worse yet, postpublication criticism often goes unanswered by the authors. The availability of adjunctive web publishing, which sometimes may appear simultaneously with the print version, allows for some corrections and linkage to the original faulty manuscript, but the initial print version usually remains the definitive reference. Even if the journal recognizes the web version as the definitive version, only the print version is indexed in MEDLINE. Some journals provide a forum for discussion and further critique of accepted manuscripts (for example, Authors Interactive at ajo.com) but these may not exist in a referable, retrievable, or permanent format. The major concern is that practice guidelines may be generated based on these manuscripts without recognizing their faults.16

A substantial proportion of authors do not fulfill the criteria for authorship as defined by the International Committee of Medical Journal Editors (that is, substantial contributions to the intellectual tasks of research, including the conception, design, analysis, and interpretation of the data, and writing or revision of the manuscript).17 One study18 reported that, in multi-author journal publications, several of the authors may disagree with substantial portions of earlier versions of the manuscripts and that the final published reports ended up being an attempted consensus statement rather than vigorous scientific research. Another report suggests that many authors do not discuss the results of their manuscript in the light of the evidence from other controlled trials on their topic.19 A structured discussion (with essential elements required, similar to the structured abstract) might minimize some of these problems by encouraging authors to agree on each of several points about the manuscript.

Rennie and Flanagin20 have led a campaign for system changes to improve the ethical climate of publication of research by making it more open and more responsible. As the numbers of authors per paper has risen, the credit, accountability, and responsibility of the manuscript have been diluted.21 Rennie proposes that general authorship be abolished in favor of attributing clearly stated contributions and that the work done by the contributors be listed for the readers. Rennie also proposes that journals consider making the “aftercare” (that is, the updating) of published manuscripts the responsibility of authors and a precondition of acceptance, as is done in the Cochrane database. Postpublication peer review (a system that makes every reader a potential reviewer) is another option, since it requires even more accountability on the part of the authors. The electronic format of many journals allows for readers to become reviewers, and they may demand more accountability from authors although these comments may not be retrievable, referable, or permanent.

Authors often fail to avoid a number of other mistakes in their submissions including the failure to verify their references, a task facilitated by electronic reference software. Inconsistencies in data between the abstract and the body of the manuscript and reporting of data and other information solely in the abstract are relatively common mistakes; one study indicated that educational intervention directed to the author has not been effective in reducing these blunders.22

Submission to many publications is motivated by a desire or necessity for faculty promotion in academic institutions. This is a double-edged sword and has the possibility of fostering dissemination of information with insufficient need or value. With many major epidemiologic relationships already identified, there is also a concern that epidemiologists or clinicians are increasingly devoting themselves to the study of more subtle associations between risk factors and disease –associations that are far less certain and that can lead to the parade of conflicting results so familiar to the public.23

Reviewer responsibilities 

The main responsibility of the reviewer is to apply a set of outside criteria and standards to the paper, to provide constructive criticism for the authors, to write a critique and make critical judgments that will aid the editor in making decisions, and to accept a set of ethical responsibilities in relation to these activities. The reviewer should assess whether each component of the manuscript meets the journal standards. Finally, the reviewer makes a final recommendation regarding the suitability of the submission to the readership of the journal and suggests acceptance, rejection, or revision using some type of priority score.

Many journals expect the reviewers to give constructive criticism to authors and to render their opinions (even stern judgments) in a collegial spirit accompanied by specific suggestions for improvement. This process should always require civility in reviewers’ and editors’ comments.

Since the names of the reviewers are unknown to the authors, reviewers enjoy a privilege but also an authority with perhaps insufficient accountability to the authors. Rennie20 believes there are both ethical and practical arguments for either an entirely open peer review or an entirely closed peer review. The fact that reviewers often know the authors or, even when masked, can guess at their identity complicates matters in a small specialty like ophthalmology.

There have been a few studies that have identified the characteristics or profile of a good reviewer.24 The longer time spent on a review (up to 3 hours) improves quality.24 Four other characteristics of reviewers that predicted better quality reviews were these: younger age, affiliation with a top academic institution, familiarity with the journal’s editor, and masking of the name and institutional affiliation of the authors of the manuscript.25, 26 Younger reviewers were also less likely to decline an invitation to review. Editorial board members and senior reviewers should therefore be encouraged to involve and mentor junior colleagues in the process; most journals have procedures for sharing a reviewing assignment. In the same studies, reviews by members of an editorial board were rated of poorer quality, as judged by authors, than editors judged them. Reviewers trained in epidemiology or statistics were more likely to produce good reviews. Training in statistics, epidemiology, study design and interpretation, and in some instances, critical appraisal of the literature is increasingly available in medical school curricula. Reviewers, however, receive little preparation for performing reviews as part of their formal training. Experienced mentors working on a one-on-one basis with junior faculty are the best means of education but are not routinely utilized at present. An organized approach to teaching how to review manuscripts has been offered in a recent series of articles in Academic Medicine.27

Perceived (or real) reviewer or editor bias has always been a nemesis in the editorial process. Bias is a difficult trait to monitor or to surmount whether it exists in terms of intellectual positions or disagreements, a publication preference for positive outcomes of research, or personal social conviction. There will always be complaints about the oft-cited biases having to do with ethnicity, nationality, sex, and status.

Conclusions 

The editorial process is an evolving but as yet imperfect process that needs continual assessment and improvement by journal editors. In this Perspective we have touched on some of the responsibilities of editors, authors, and reviewers in the process. This series is intended to serve as a stimulus for thought and subsequently for discussions, recommendations, and improvements of the process in the ophthalmology journals.