CRITERIA FOR PEER REVIEWING THE ORIGINAL ARTICLES SUBMITTED T O A BIOMEDICAL JOURNAL: INTELLECTUAL HONESTY IS THE BEST POLICY

Address for correspondence : Dr. Bishnu Hari Paudel B. P. Koirala Institute of Health Sciences, Dharan, Nepal. Email: bhpaudel2002@yahoo.com

publication is expected to be scientific, concise, simple, selfexplanatory, and of course, intellectually honest.
There are probably thousands of biomedical periodicals (journals) published worldwide at certain intervals.The journal of the Nepal Medical Association (JNMA) is one of such periodicals, which has about four decades of history of scientific publication.The scientific publications in the journal, nowadays, appear after scrutinizing the manuscripts by peer reviewers.There are few more such journals published in Nepal.

WHAT IS PEER REVIEW?
For publication of a scientific work in a journal, authors submit manuscript of their research work in a printed form and/or in its electronic version.Before the research work is printed as a publication, the manuscript undergoes an editorial processing.The journal editors identify specific peer-scientists to critically evaluate the publication potential of the submitted manuscripts.A peer reviewer is the expert in certain area of biomedical sciences, and the peer review is the evaluation of importance and quality of original research manuscripts.Peer review is an intrinsic part of all scholarly work, including the scientific process.Peer reviewers are the experts who are not a part of the editorial staff.Ideally, the review is unbiased, independent, and critical assessment of the manuscript.Peer review can therefore be viewed as an important extension of the scientific process.Although its actual value has been little studied, and is widely debated, peer review helps editors decide which manuscripts are suitable for their journals, and helps authors and editors in their efforts to improve the quality of research reporting.A journal that has undergone such reviewing is one that submits most of its published research articles for outside review before their publication.The number and the type of the manuscripts sent for review, the number of reviewers, the reviewing procedures, and the use made of the reviewers' opinions may vary according to journal's publication policy.
Peer-reviewed research articles are considered highly trustworthy biomedical research because they have undergone scrutiny by experts who are knowledgeable in the topic.The peer review of original articles, before they are published, has been widely accepted practice and it is a norm in biomedical publications.In the interests of transparency, each journal should publicly disclose its policies in its instructions to authors. 2 The biomedical journals worldwide follow certain guidelines for reviewers.It makes the job of the reviewer more objective.Simultaneously, the contributing authors would find unbiased judgement on their research work.The job of editors would also be facilitated and they may keep a record of performance of individual reviewer so that they can refer to them for subsequent review requests.
The process of reviewing is a privilege and prestige for the reviewers. 3It should also be recognised that the task of reviewing a scientific research work is a great responsibility.It demands honesty, especially the intellectual honesty, on the part of reviewers.In other words, the reviewer should be clear enough that if s/he is not an expert in the field on which a manuscript has been asked to review, s/he should honestly reject reviewing the manuscript or a part of it.Then s/he should convey to the editorial board that s/he is not in a position to review the given paper or a part of it.

AIMS OF THIS ARTICLE
The aim of this article is to propose a set of peer review criteria and highlight its usefulness in the process of quantifying the peer review.It is also aimed to sensitise all the concerned scientists and doctors i.e., reviewers, editors, authors and critical readers of national and international biomedical journals.

RATIONALE OF DISCUSSING PEER REVIEW IN THE NATIONAL CONTEXT
The practice of peer review has been adopted by the journal of Nepal Medical Association (JNMA) because, I believe, the editorial board of the journal, and quality-concerned medical scientists including this author, have understood that there is a pressing need to further improve the publication quality of the journal.The publication quality has also been a matter of concern for the entire NMA family and its active members.The efforts are, albeit slow, moving on positively to make the journal internationally recognisable.Therefore, this paper is an attempt to highlight some major issues to be dealt seriously so as to minimise the (assumed) potential errors of peerreviewing and facilitate the decision making process of editorial board before an original article appears in the JNMA and other biomedical journals published in Nepal.
Unfortunately, we do not have concrete data regarding the performance of peer reviewers and publication quality of the national biomedical journals, especially the JNMA; therefore, this article is primarily based on supposition and occasional remarks done by editors, reviewers, authors, and readers of the JNMA.However, on the basis of reviewing experience of about a dozen of manuscripts submitted to the JNMA in recent past, and review of research grant projects of BP Koirala Institute of Health Sciences; this author is encouraged to speculate that most of the manuscripts submitted to the JNMA required transformational changes before they were acceptable Paudel .Peer review scoring criteria for publication.This assumption might be true for many biomedical journals published worldwide.Thus, minimising the transformational editing and imparting justice to the authors and the health/biomedical science are essential for the journal.An investigation is required to answer the question of imparting justice to science and the authors.However, there is perceivable evidence on JNMA performance and sufficient support from international literature to justify discussion on the topic of peer review.Furthermore, the international literature on importance of peer review is full of concrete evidences to convince this author to raise the issue of quality peer review.Thus, this author's speculation has genuine ground.Peer review of many submissions including the JNMA submissions bear special significance because, as international literature says, peer reviewed research articles are, and should be, highly credible because they have undergone scrutiny by speciality experts.

SOURCE OF THIS ARTICLE
This article is the outcome of review of relevant research publications pertaining to peer review and related issues presented in the First to Fourth (1989, 1993, 1997, and 2001)  International Congresses on Peer Review in Biomedical Publication (ICPRBP).A special emphasis has been given to the papers presented in the Third ICPRBP (held in September 1997 in Prague, Czech Republic), which were subsequently published in the Journal of American Medical Association (JAMA).The entire issue of the journal (1998, Vol.280, issue 3) deals with the papers on peer reviewing, editorial policy, and publication practices regarding authorship, gender biases, assessment of reviewers' performance etc.It is worth noting here that the JAMA regularly monitors editorial performance, which includes review time, editorial decisions, and publication 4 without interfering the review process or editorial decision.Going through the entire issue of the journal this author found that it is extremely useful and would be of great relevance for editors, reviewers and even for the readers in further improving the quality of national biomedical publications.

BRIEF ACCOUNT OF WHAT THE WORLD LITERATURE SAY ABOUT PEER REVIEW
The Second ICPRBP discussed primarily about the management of manuscript by editors.In the Third ICPRBP various aspects of quality of publication were discussed.It was observed that reviews often seemed inconsistent, unreliable, and even incompetent; and the comment of reviewers may be hasty and superficial. 5However, good reviewers, if provided with relevant material along with the abstract and full-length paper, find helpful for peer reviewing.Even masking the review process has given conflicting reports on quality of review and it has added new biases, for example, delay in publication of articles on passive smoking studies with non-significant results as compared to those with significant results. 6Similarly, Callaham et al., have reported that analysis of abstracts submitted to a scientific meeting showed that the abstracts with positive-results were accepted more for presentation and publication. 7 has been found that there are fewer women in editorial position in epidemiology journals in the USA.Its cause is still not known but it sufficiently indicates possibility of selection bias in appointing editors in such positions. 8The US reviewers are found to favour the papers submitted by the US citizens than the non-US, and even the non-US reviewers were inclined to behave like US reviewers. 9It may be called citizen bias.It is not surprising that reviewing a review article has been a very responsible and difficult task and its review depends on reviewer's discipline and nationality. 10Thus, the world literature contains existence of bias in biomedical publication and its system.It is an unsafe practice in science, which might lead society in wrong direction.Therefore, continuous efforts are going on to minimise bias in biomedical publications by bringing out the flaws detected in literature.
During the Fourth ICPRBP some more exploration on the issue was done.An American study assessed journal quality indicators (peer review status, impact factor, citation rate, acceptance rate, circulation, MEDLINE indexing, and Brandon's Library List indexing) as valid predictors of research article methodological quality of randomised and nonrandomised controlled trials of 30 general medical journals.All the journals included in the study were peer reviewed and indexed in MEDLINE.Multivariate analysis showed that citation rate alone was the most reliable predictor of journal article quality.The study concluded that high citation rates, impact factors and circulation rates, low manuscript acceptance rates, and indexing on Brandon's list appear to be predictive of higher methodological quality scores for articles published in the journals. 11e role of reviewers is difficult in the sense that they have to work as an advocate of a journal (role of editor) and as an advocate of authors (role of authors).While working as journal advocate if s/he rejects a manuscript, it makes authors pessimistic about chance of subsequent publication by them. 12hus, considering the current status of manuscript submission to the JNMA and many other national journals publishe in Nepal, such negative impact resulting from rejecting manuscript should be minimised by providing valuable suggestions, if possible with evidence from available literature, to the author(s) in a decent language so that authors are supported and not discouraged.
There has been poor agreement between abstract and body of full paper in international literature, 13 and this author is tempted to speculate that the issue might be extremely serious in the biomedical publications in Nepal.Using Consolidated Standards of Reporting Trials (CONSORT) descriptor criteria have been considered a gold standard in reporting randomised controlled trials.It has been found that there is no difference in text reporting associated structured abstract using CONSORT descriptor criteria to evaluate reporting quality and without using CONSORT descriptor criteria. 14The study was focused on two different American Ophthalmology journals during the first half of 1990s.Furthermore, even the leading biomedical journals of the world (BMJ, JAMA, The Lancet etc) did not adequately follow the CONSORT recommendations. 15Therefore, it is presumed that there is a need to investigate the adherence of authors to prescribed format of biomedical publications.An indication towards non-adherence of authors to prescribed format of the JNMA submissions may be found in various areas and especially in the referencing system of the JNMA.In fact, this author, after reviewing about a dozen of manuscripts recently, did not find even a single author who completely adhered to the Vancouver style of stating references.It is worth mentioning here that the Vancouver System of referencing has already been upgraded and is called Uniform Requirements for Manuscripts Submitted to Biomedical Journals: Writing and Editing for Biomedical Publication. 2

EFFECT OF MASKING ON PEER REVIEW QUALITY
Masking author identity is not associated with success of reviewing but for reviewers with less research or reviewing experience it might increase the success. 16Some other studies on blinding and unmasking authors identity to reviewers or making reviewers to sign their review reports or masking reviewers to authors have failed to show significant effect on the quality of peer review. 17,18,19,20Even a written feedback by editors to peer reviewers has not improved the quality of subsequent reviews. 21Irrespective of size (big or small) of the journal, reviewers' recommendations have been found to be similar influence on editorial decision. 22A study has shown that about 50% peer-reviewers want disclosing their identity in the article if their institute /organization recognizes their contribution to the article they reviewed.About two third of them did not favour the disclosing because they perceived that their institute does not recognize their contribution. 23evertheless, the peer review is being practiced and is considered gold standard before manuscripts are published in biomedical scientific periodicals 24 and thus, it is necessary for the progress of research reporting in biomedical sciences.
The above-mentioned evidence shows that reviewing is an extremely difficult task, even in the countries where research culture is well established, in doing justice to the authors and science.Now, the editorial board along with concerned members of the NMA and the authorities of other biomedical journal of Nepal would agree with the notion that the majority of national biomedical journals have a Herculean task ahead in making their publication international along with imparting justice to their scientific contributors and national biomedical science.We should be optimistic that with the improvement in peer review and submission quality, the publication quality will definitely go up.For example, some improvement in use of statistical tools has been found in Chinese biomedical publications possibly owing to input from statistical reviewer. 25However, peer reviewers are also found to do mistakes making unfounded statistical criticism in difficult areas such as sample size and multiple comparisons.This is usually from subject matter reviewers with limited statistical knowledge. 26Therefore, it may be recommended that the JNMA editorial board identify genuine peer reviewers to ensure quality in its publications.Similarly, other national biomedical journals, since their experience is in early childhood, may need this approach.

WHAT ARE THE ATTRIBUTES OF A GOOD REVIEWER?
The assessment of cognitive process that reviewers use while evaluating a manuscript has been considered time consuming, difficult and expensive. 5The reviewers trained in epidemiology or statistics are known to produce high-quality peer review and the review quality improves if they increase the time for reviewing up to 3 hours for a manuscript.Spending time beyond 3 hours does not increase the review quality. 27This time may be more for us who learn English as a non-native language.They find it helpful if they are provided with relevant material along with the abstract and full-length paper.
It is important to note that the reviewers require some training for quality peer reviewing.Not only the reviewers, even the editors would also like to have training regarding their role or job of review process on time because ideal reviewers or editors are found to be of young age, i.e., below 60 years. 27n addition, it has been believed that editors and reviewers are extremely busy people.
Being an expert is obviously the prerequisite for a reviewer.However, there are some other important qualities, which help the reviewer to be a fair critic.To name few, these qualities are: having critical mind, broad-based knowledge of basic medical and clinical sciences including natural sciences (physics, mathematics, chemistry etc.).Philosophical understanding, in terms of dialectical thinking (both inductive and deductive), helps to put evidences in logical sequence.

Paudel . Peer review scoring criteria
Simultaneously, strong foundation in various aspects of biology such as biology of lower animals, vertebrates and comparative biology, and sociology is an extra benefit.

BIOMEDICAL PUBLICATION IN FUTURE
Presenting as the opening address of the Third ICPRBP in Prague (1997) Dr. Rennie, the deputy editor (West) of JAMA, predicted that in the future authorship will be abolished in favour of contribution and the work done by the contributors will be listed for the readers, peer review will become open, and anonymous review will be obsolete, scientists will take full responsibility for aftercare of their papers, and editors enable and encourage readers to assume the responsibility of reviewers. 28

FOR THE EDITORS AND JOURNAL AUTHORITIES
Based on the review of international literature an attempt has been made in this article to suggest a set of criteria (vide infra Table ) for the process of peer reviewing.These criteria are based on international literature and have tremendous potential for immediate use.It is expected that these criteria will be followed in the process of management of manuscripts submitted to the biomedical journals.This author believes that these criteria have been made suitable for current standard of the journal and would go on improving dynamically as editorial policy, readers' concern etc. go on changing with time, and a strong scientific culture establishes in the country.With the improvement in publication quality and habit of using published reports, we can hope some advancement in national biomedical sciences.
It is worth mentioning here some other issues that may be related to the job of the journal editorial boards.It appears that time has come for the national journals to deal with publication disputes, which might be expected in near future.Thus, the authorities of a journal may constitute an independent body, consisting of its intellectually honest members, to look into the matter pertaining to the complaints of manuscript contributors, authorship, and other related issues. 29Regarding the authorship of a paper, the departments/sections from where the paper originates should establish their detailed, written guidelines on authorship practices. 30If editors wish (may be they need!) to evaluate their reviewers, this author believes that our reviewers will not object this idea.It can be done with minimal additional work on the part editorial board.The editors need to develop a simple rating scale to monitor reviewer quality.Such a rating system could be used for awarding the best reviewers during regular conferences/ meetings to encourage them.Although making reviewers to sign their review reports has not been found to improve quality of peer review, 18 the editors of a journal may decide whether such a measure would be useful for them.Nevertheless, in the initial stage of journal quality development, a commitment by reviewer such as "I have reviewed the research work or part of it with full knowledge of the content and I have been intellectually honest throughout my review process, and my comments or suggested modifications are unbiased and critical" may be useful to obtain from the reviewers.Such a measure might make the peer reviewers more conscious and responsible.This sort of regulation may be practiced for initial few years until a journal's editorial board identifies sufficient number of qualified peer reviewers.

FOR REVIEWERS, EDITORS, AND READERS OF A BIOMEDICAL JOURNAL
It is expected that the suggested set of criteria (vide infra Table ) for reviewers will facilitate their peer-review process so that publication quality is taken care and justice to authors/ contributors is imparted.Majority of the criteria are based on literature and some other points of the criteria (taxonomy list) are incorporated and adjusted by this author arbitrarily judging the present status of JNMA performance.However, the proposed scoring system is entirely this author's creation, it is unlikely that the readers will encounter such a scoring system for peer reviewing in national or international literature.This author believes that these criteria would go on changing to meet national demand and the future international guidelines.Currently, adhering to the proposed guideline would make us objective and this objectivity should primarily help editorial decision-making on the acceptability of original article for publication.In turn, the journal will definitely move faster towards internationalisation. Implementation of the criteria, and timely advancement would prevent us from danger of deinternationalisation of our would-be-international journal. 31he proposed manuscript scoring criteria may be of some use for scoring the manuscript submitted to other national (even international) biomedical journals.

USING THE CRITERIA
These criteria are only a rough and preliminary guideline and they are subject to modification depending on the editorial policy.These criteria are most useful in reviewing original articles; they may be of some use while reviewing review articles, research projects, case reports, letters to editors etc.The maximum possible score that a manuscript can secure is 42 and the minimum is minus 21.Negative marking is required if the items 3 and 6 are violated.Based on the score secured by a manuscript after peer review, its quality and publication potentiality may be categorised as the scoring: 1) equal or more than 28.5 may be accepted for publication after minor changes by reviewer(s) and/or editorial board, 2) between 14.5-28 is acceptable only after clarifying/incorporating all the comments and suggestions of reviewers and/or editorial board, and 3) between below 14 is rejected.In the case of first category, publication of the article primarily depends on editorial board and/or reviewers' input whereas in the second category, the Paudel .Peer review scoring criteria authors are responsible for further revision and resubmission of their manuscript.Based on the score obtained by a manuscript, further management of the editorial board would be either accept the manuscript for publication or send it back to the author(s) for revision or reject it.In controversial situations, the board may get opinion of two or more peer reviewers, club their scores, and act accordingly.
The above criteria have been proposed for the sake of objectivity in assessing a manuscript but one should also keep in mind that being too mathematical may not always be fruitful.Some of the items of the criteria may not be applicable to the articles of policy matters or census type of studies.In such cases, review may be done using the same taxonomy but with individualised and modified scoring.Similarly, it is natural that the criteria are not perfect and thus some reviewers might argue the validity and reliability of the proposed scoring system.

LIMITATIONS AND STRENGTH OF THE CRITERIA SUGGESTED FOR PEER REVIEWING
It was not possible to eliminate potential overlap among some of the items of the criteria.New reviewers may find it difficult to delineate between different items, thus opinion of additional reviewer(s) would be desirable to the editorial board to appropriately score the manuscript quality.Nevertheless, going through the relevant literature would give the reviewers confidence in dissecting out the merits and flaws present in the article they are reviewing.An emphasis has been given to English language because most of the articles submitted to the JNMA and many other journals of Nepal are/would be coming from authors who learn English as second or third language.Since this author believes that the criteria suggested in this article are not absolute, a discussion and consensus among the editorial board members, reviewers, authors, and the readers may be necessary to make the criteria more objective and acceptable for immediate use.Nevertheless, adhering to certain criteria is the only and right path towards better peer reviewing and one of the major ways of uplifting the standard of the national biomedical publications including that of JNMA.In summary, peer review is a process of assessing the quality of manuscript submitted to a journal.This is a standard worldwide practice in biomedical publications.The journals with proper peer review are considered "standard".A set of criteria has been proposed to make peer reviewers, editors, contributors, and readers more conscious, honest, and responsible while reporting, reviewing, or processing a manuscript.In turn, the quality of publications would move towards internationalisation, national biomedical science -towards enrichment and national health care -towards better quality.Thus, let us keep in mind the saying by John Gardner while reviewing the manuscript submissions, "Liberty and duty, freedom and responsibility: that's the deal."