Ian Parberry y. University of North Texas. July 31, 1994

Transcription

1 AForm for Referees in Theoretical Computer Science æ Ian Parberry y Department of Computer Sciences University of North Texas July 31, Introduction The Referee's Form is designed to help referees communicate more clearly with Editors. It is not intended to replace the formal written report, but is intended as a clear, concise synopsis. The LATEX source æle for the form also includes space for the referee to provide his or her formal report, and is available by anonymous ftp. To use it, referees can copy the form and check one response in each of the subsections. The ærst page of the form is for the Editor's eyes only, but the second page is intended to be forwarded to the author. The remainder of this manuscript is divided into three major sections. The ærst, ëinstalling the Files", describes how to obtain the æles needed to use the LATEX source æle for the form. The second, ëusing the Form", describes the mechanics of ælling in the form once you have obtained the necessary æles. The third, ëevaluating the Paper", describes the meaning of each of the questions in the form and their associated responses. 2 Installing the Files You will need to obtain the following æles before you can use the form: form.tex: ipform.sty: art11.sty: fullpage.sty: the LATEX source code for the form, the form style æle, the 11pt article style æle èstandard distributionè, and the fullpage style æle èstandard distribution in the USè. The æles ipform.sty and form.tex are available by anonymous ftp from ftp.unt.edu èip address è in the directory ianèguidesèform. A copy of fullpage.sty is also provided for non- US users. The following æles may also be useful: sample.tex: instructions.*: acopyofform.tex ælled in for a sample paper, and an electronic version of this manuscript. You should ensure that a copy of ipform.sty is placed into either: æ cæ Ian Parberry, This document may be reproduced without fee provided the copies are not distributed for proæt, and this copyright notice is included. y Author's address: Department of Computer Sciences, University of North Texas, P.O. Box 13886, Denton, TX , U.S.A. Electronic mail: ian@ponder.csci.unt.edu. 1

2 æ the directory that contains the style æles for your system, or æ your own personal style æle directory èif you have oneè, or æ the same directory as form.tex. Once the necessary style æles have been installed, you simply need to make a copy of form.tex for the paper being refereed and use a text-editor to æll in the entries. This process in described in more detail in the next section. 3 Using the Form The source æle for the form, form.tex, is divided into three parts, the preamble, the checkbox entries, and the formal report. Each of these is described in a subsection below. 3.1 Preamble The preamble consists of information about the author and editor, presented in a sequence of ënewcommand statements. These should initially be ælled in by the editor, and corrected by the author. These are as follows: ëjournalname: ëpapertitle: ëauthornames: ëpapernumber: ërefereename: ëreferee ëeditorname: ëeditor ëeditortitle: ëissuedate: ëpreparedate: the name of the journal for which the review is being prepared, the title of the paper under review, abbreviated if it is excessively long, the name of the author or authors, the number assigned to the paper by the head oæce of the journal èif anyè, the name of the referee completing the form, the referee's electronic mail address, the name of the editor for whom the review is being prepared, the editor's electronic mail address, the editor's title èsuch as ëeditor", ëassociate Editor", ëmanaging Editor"è the date that the editor sent out the form to the referee, and the date on which the referee prepared the form. A set of blank ënewcommand statements are provided as follows. ënewcommandíëjournalnameííí ënewcommandíëpapertitleííí ënewcommandíëauthornamesííí ënewcommandíëpapernumberííí ënewcommandíërefereenameííí ënewcommandíëreferee ííí ënewcommandíëeditornameííí ënewcommandíëeditor ííí ënewcommandíëeditortitleííí ënewcommandíëissuedateííí ënewcommandíëpreparedateííí The editor and referee simply insert the information in the appropriate places, as follows: ënewcommandíëjournalnameííbohemian Journal of Countingí ënewcommandíëpapertitleíía Treatise on the Binomial Theoremí 2

3 ënewcommandíëauthornamesííjames Moriartyí ënewcommandíëpapernumberííbjc449í ënewcommandíërefereenameíímycroft Holmesí ënewcommandíëeditornameííirene Adlerí ënewcommandíëeditortitleííassociate Editorí ënewcommandíëissuedateííjuly 30, 1884í ënewcommandíëpreparedateííseptember 9, 1884í 3.2 Checkbox Entries The body of the form is divided into a sequence of questions with a checkbox for each of the numbered responses. To select a response, simply identify the newcommand statement for the particular question and change the number from 0 to the number of the desired response. For example, the initial statement for Question èësigniæcance"è is ënewcommandíësignificanceíí0í, which results in no boxes being checked. Changing that to ënewcommandíësignificanceíí3í results in box number 3, ëprogress" being checked. 3.3 The Formal Report The last item in the form is a formalreport environment. The referee should place his or her formal report between the ëbeginíformalreportí and ëendíformalreportí statements. The style æle will automatically generate the heading from the information in the preamble. 4 Evaluating the Paper This section consists of a brief explanation of each of the responses to the form items. 1For the Editor: This page will not be shown to the author. 1.1 Recommendation: This question records your recommendation as to whether the paper should be accepted. Of course, the Editor need not necessarily agree with your opinion. However, it is important that you express one clearly and unambiguously. 1. Accept: The paper should be accepted in its current form without further debate. 2. Accept, advise changes: The paper is acceptable, but there are some changes that would improve the manuscript. Acceptance need not be contingent on these changes. 3. Accept contingent on changes: The paper is acceptable contingent on some changes being made. If the Editor can verify that these changes have indeed been made, a second round of refereeing may not be necessary. 4. Revise and re-referee: The authors should be invited to submit a revised manuscript. A second round of refereeing is strongly recommended. 5. Combine with simultaneous paper: The results were discovered more or less simultaneously by others. It is recommended that all of the researchers involved pool their resources and produce a single paper. 6. Cannot be refereed properly: Full refereeing was not possible due to major technical andèor presentational diæculties. The authors must address these problems and resubmit the paper. 3

4 7. Paper not publishable in this journal: The paper may be suitable for publication, but not in this journal. Reasons may include scope, interest, or quality. 8. Paper not publishable: The paper is not publishable in any form in any journal. 1.2 The Referee: This group of questions requires you to evaluate your eæectiveness as a referee. It is not intended to be used to grade your performance, but rather to enable the Editor to compare the weight ofyour opinion with that of other referees who mayhave diæerent ideas about the paper you are refereeing Competence of Referee: How competent are you in the area of specialization of the paper? This refers the speciæc area èsuch as ëlower bounds for PRAMs"è, rather than the general area èsuch as ëparallel computation"è. It may be assumed that the Editor has chosen you as referee because of your expertise in the general area, but he or she may not be fully aware of your level of expertise in the speciæc area. 1. Expert: I consider myself to be an expert in the æeld. 2. Well-versed: I have made contributions to the literature, and keep up with the latest results on a fairly regular basis, 3. Interested: I usually read the literature, but may not be completely up-to-date. 4. Competent: I am capable of reading the literature, but I am not completely familiar with it Conædence Level: How conædent are you in your opinions? 1. Very conædent: I am almost certain about my evaluation. 2. Conædent: I am sure about my evaluation èbeyond reasonable doubtè. 3. Reasonably conædent: I believe my evaluation to be correct Eæort Spent: How much eæort did you spend refereeing this paper? It is important that you indicate to the Editor how much timeyou spent, so that he or she can tell whether you did a cursory job or a detailed job. It also indicates how diæcult the paper is to read and understand. 1. Great: It took a great deal of time and eæort. 2. Reasonable: It took time and eæort, but not exceptionally so. 3. Minor: It took very little time or eæort. 4. Almost none: I was able to read it almost eæortlessly Comprehension: How well do you understand the paper? 1. Understand perfectly: I understand all of the details. 2. Understand majority: I understand almost all of it. 3. Understand ideas: I understand the main ideas. 4. Slightly confused: I found the paper slightly confusing. 5. Very confused: I don't understand much at all Details Checked: How much of the details did you check? 1. All details: Ichecked all of the details in the paper. 2. Most details: I checked most of the details and believe the rest to be straightforward. 4

5 3. Enough details: I checked enough of the details to be convinced of the correctness or incorrectness of the results. 4. At a high level only: I checked mainly general structure and logic. 2For the Author: This page will be sent to the author after the Editor has used it. 2.1 The Results: This section of the form requires you to evaluate the results independently of the presentation. This may not be easy since bad presentation may obscure good results Signiæcance: How signiæcant do you think the results are? 1. Seminal: A breakthrough that will stimulate a great amount of research. 2. Interesting: A paper that will stimulate great interest. 3. Progress: A paper that has made some progress on reasonably important questions. 4. Incremental progress: Progress has been made, but it is mainly technical in nature. 5. Ancient history: The æeld has moved on from results of this type. 6. Mundane: Anybody working in the æeld can ægure out how to do this without much eæort. 7. Trivial: At the level of a graduate student exercise. 8. Cannot tell: The paper is so badly written that I cannot tell what is in it Originality: How original are the results? 1. Original: The results are new and original. 2. Simultaneous discovery: The results appear to have been discovered more or less at the same time by other researchers. 3. Small twist to known work: The work consists mostly of what is known, with perhaps a small amount of new results. 4. Already published by authors: The authors appear to have already published these results in a refereed journal. 5. Already published by others: The authors appear unaware that the results have been previously published. 6. Folk theorems: Everybody knows these results Proofs: How do you rate the proofs themselves? 1. Beautiful: The author has spent a lot of time on form and technique. 2. Pretty: They are very good, but not works of art. 3. Serviceable: They are satisfactory, but not outstanding. 4. Ugly: They are clumsy and inelegant Proof Techniques: How do you rate the proof techniques used? 1. Deep: The paper invents and applies new mathematical techniques. 2. Innovative: The paper uses ordinary methods in an unheard-of way. 3. Clever: The paper uses ordinary methods in a clever and non-obvious way. 4. Elegant: The paper uses standard techniques to prove interesting new things. 5. Workmanlike: The proofs are not diæcult, but are competently done. 6. Simple: The proofs are straightforward enough for a reasonably well-trained researcher to be able to duplicate them from a statement of the theorem alone. 5

6 7. Trivial: The proofs are obvious Correctness: Do you believe that the results are correct, and how strongly do you feel about your opinion? 1. Correct: I am certain they are correct. 2. Correct beyond reasonable doubt: I think that they are correct, but there might conceivably be subtle æaws that I have missed. 3. Probably correct: They seem right, but I am not willing to certify their correctness. 4. Probably incorrect: They seem wrong, but I haven't located speciæc æaws. 5. Proofs wrong: The proofs contain æaws that can probably be patched to give results identical or similar to those claimed. 6. Results wrong: The proofs contain æaws that I believe may be insurmountable. 7. Cannot tell: The paper is so badly written that I cannot tell whether the results are correct without redoing the authors' research. 2.2 The Paper: This group of questions requires you to evaluate the presentation independently of the results Accessibility: What level of training is necessary to read the paper? 1. Expert: Somebody who is an expert in the speciæc area of the paper. 2. Specialist: Somebody who works in the general area of the paper. 3. Theoretical computer scientist: Most theoreticians, with some eæort. 4. Computer scientist: Even a non-theoretician, with some eæort Presentation: How well is the paper organized and written? 1. Almost æawless: The paper is almost perfect. 2. Polished: Very little needs to be done. 3. Adequate: The paper reads reasonably well, but needs to be polished. 4. Rough: The paper does not read well. 5. Incomprehensible: The paper reads so badly as to render understanding impossible. Density: Is the length of the paper justiæed by itscontents? It should be terse, but not too terse. 1. Too long:the paper is far too long for the weight of material. It must be condensed. 2. Adequate: The length of the paper is about right for the weight of material. 3. Terse: The material is nicely terse. 4. Too terse: It is so terse that most readers will ænd it heavy going. It should be expanded Missing Proof Details: All proofs have gaps. Can the reader reasonably be expected to æll in the missing details? 1. Probably Wrong: The gaps in the proofs probably hide errors. 2. Often incomprehensible: There are many large gaps that render the paper incomprehensible. 3. Requires great eæort: The gaps can be ælled in with great eæort. 4. Requires eæort: The gaps can be ælled in with a reasonable amount of eæort. 5. Requires small eæort: The gaps can be ælled in with a small amount of eæort. 6

7 6. Requires no eæort: There are no gaps, because the material is too simple for that. 7. Too much detail: The paper goes into far too much detail and leaves nothing for the reader Technical Writing: Does the Abstract truly abstract the essential contribution of the paper? Is the author's grammar, syntax, semantics, and spelling correct? Does the Introduction adequately introduce and motivate the topic? Does the author describe the problem succinctly yet completely? Are the references reasonably complete and accurate? Does the title adequately describe the paper? 1. Excellent: Of a very high standard. 2. Good: Of a high standard. 3. Adequate: Acceptable, but could be improved. 4. Substandard: Needs some work to bring it to an acceptable standard. 5. Inadequate: Needs a lot of work to bring it to an acceptable standard. 6. Very bad: Should be completely redone. 7. Missing: Completely absent from the paper. 5 Further Reading Readers who require further advice on the refereeing process in theoretical computer science can consult Parberry ë4, 5, 6ë èand the references contained thereinè, Postscript and dvi versions of which can be found at ftp.unt.edu, in directory ianèguidesèreferee. 6 Acknowledgements The sample form æle referred to in Section 2 was ælled in using data from a previously unknown referee's report. For more information on the author of the paper under consideration, the reader should consult Bowers ë1ë and Gould ë2, 3ë. I am indebted to Dana Richards for several useful suggestions on the sample form, and for providing these references. References ë1ë J. F. Bowers. James Moriarty: A forgotten mathematician. New Scientist, pages 17í19, December ë2ë H. W. Gould. The case of the strange binomial identities of Professor Moriarty. Fibonacci Quarterly, 10è4è:381í392,402, October ë3ë H. W. Gould. The design of the four binomial identities: Moriarty intervenes. Fibonacci Quarterly, 12è2è:300í308, October ë4ë I. Parberry. A guide for new referees in theoretical computer science. SIGACT News, 20è4è:92í 109, ë5ë I. Parberry. A guide for new referees in theoretical computer science. Bulletin of the EATCS, è40è:511í530, ë6ë I. Parberry. A guide for new referees in theoretical computer science. Information and Computation, 112è1è:96í116,