These instructions are for the Subject Area Chairs (SACs) for CMG'08. Any questions should be referred to the Program Chair or Assistant Program Chair at . A quick reference SAC schedule is also available.
Referee Management
SACs will be assigned a group of referees by the Program Chair via the Volunteer Coordinator. Please review the Referee Qualification Guidelines. The SAC must "qualify" all of his/her referees to ensure that they are available during the refereeing period, are qualified for the assigned subject area, have appropriate Web access to perform referee tasks, and are still at the phone/email address provided from CMG's records. All changes in phone, fax, company, or email information should be reported to the Assistant Program Chair.
To qualify the referees, the SAC must call or email each one assigned to that area. It is highly recommended that the SAC call each referee initially, as discussion of the referee's expertise can assist in assignment to an alternate Subject Area, if necessary. If after 3 attempts, the referee has not responded to the SAC's email or phone calls, the name should be deleted from the list of available referees. If a volunteer would be better suited to another Subject Area, please notify the Program Chair or Assistant Program Chair at , who will assign that person to another SAC.
If there are multiple SACs for a subject area, consider dividing the list of referees, perhaps via a track or sub-area, and processing them accordingly.
The referee qualification process must be completed by June 6. Changes to the referee list information (if any) should be returned to the Assistant Program Chair after that so the HQ database can be updated. Notify the Assistant Program Chair of any problems or concerns during the qualification process.
Although papers are due by Friday, June 13, the submission system will not be disabled until Monday, June 16. There will be a de facto extension for paper submissions until the system is closed on June 16: After that, any time extension request for paper submission must be approved by the Program Chair.
Paper Assignment
The SAC will assign papers to designated referees immediately after the paper submission deadline. (As papers are submitted, the SAC may assign them to referees immediately, with a caveat that revisions may occur until the paper deadline. Most papers will arrive just before the deadline, but assigning those that arrive early will allow some referees to get a head start.) Depending on the number of papers in the Subject Area, each referee should be assigned up to 4 papers. Ideally, each paper should have at least four referees assigned to it, with the SAC counting as one referee.
Abstracts
The SAC will edit all abstracts in their Subject Area (or in cases where there are co-chairs, the abstracts assigned to each). The editing is NOT for content; we want to let the author state what he/she wants. Rather, edit for typos, spelling, punctuation, and readability.
Abstracts should NOT be edited until after the paper has been submitted. Abstracts often change between abstract submission and paper submission, and sometimes abstracts are submitted without papers being submitted later. Abstract editing can be done during the days prior to Agenda Scheduling Weekend.
If a paper or abstract looks as if it should be in a different Subject Area, contact the Program Chair or Assistant Program Chair to discuss. Then follow the process on the web site to send it to another area after discussing with the Assistant Program Chair.
Agenda Scheduling Weekend is scheduled for July 26-27. At that time, the conference will be scheduled by the General Chair, Program Chair, and Assistant Program Chair. A few SACs may be invited to participate at the discretion of the Program Chair. During the week before Agenda Scheduling Weekend, conference calls with SACs will be held to determine which papers are accepted for the conference.
Referee Process
The SAC must make sure papers are readable for their referees. Specifically, the SAC must download ALL papers assigned to their Subject Area and check for any conversion problems that may occur. These also double as the SACs' copies, since a SAC must referee all of that area's papers.
Referee instructions can be found HERE.
Note: Most papers typically arrive at the last minute.
Referees will have Web access to their assigned papers as soon as they are assigned. This is easiest done via email. Each referee (including the SAC) should download their assigned papers, read and evaluate them, and fill in the evaluation form via the Abstract and Paper Submission System.
If you're tight on referees, try to assign more papers to all SACs. If you run out of referees, beg your colleagues or friends to assist. Remember, all referees receive gifts for helping CMG. However, do keep in mind they should be qualified in the subject area.
Keep a record, to be forwarded to the Assistant Program Chair, of referees who don't do what they agreed to do. A list is maintained for future reference.
As paper evaluations are completed, use a spreadsheet to keep track of the ratings for each paper from each referee. This is also a good way to keep track of which referees still owe you evaluations.
By July 3, the SAC should contact referees to ensure that they are not having any problems downloading or reading papers and that the process is going smoothly. Referee evaluations are due July 18. Referee reports will be available to the SAC through the paper submission system on-line. This is also a good way to review who has completed assignments. The SACs then have until July 24 to go through results and recommend to the Program Chair which papers to accept. A conference call will be scheduled in the days prior to Agenda Scheduling Weekend to discuss which papers are recommended to be accepted for the conference.
In addition to accepted papers, each SAC can recommend 1 or 2 papers that merit consideration for Best Paper award. If no papers merit that consideration, don't recommend any.
SAC Instructions: Rating, Ranking, & Selecting Papers
These instructions cover CMG'08 policies for rating, ranking, selecting, and rejecting papers in Subject Areas. They will assist the SAC in making some potentially difficult decisions.
The SAC is a referee and must read and evaluate all papers assigned to that Subject Area. If any paper is beyond comprehension, contact the Program Chair or Assistant Program Chair for additional guidance.
Read the Presentation Policy carefully! It has been officially relaxed, and it's permissible to name products in papers as long as there is no promotional or disparaging intent or context. If referees complain in their evaluations, make sure they aren't using old criteria for their opinions. The Presentation Policy is at the following address: http://www.cmg.org/conference/prespol.html.
Rejection Policy: If two or more referees feel a paper should be rejected, it should be considered rejected. Review the reasons for rejection (they MUST be noted on the referee form) and make sure you are comfortable with the reasons. It's very important that the authors receive this feedback, and if the referees do not supply it, you must get it from them.
Rejection Reversal: If you disagree with a referee's decision of rejection, you may override it, but consider contacting the rejecting referee(s) and explaining your reasons. Discussing any concerns will hopefully resolve the decision-making. However, if the SAC and the referees cannot come to a mutually satisfactory decision, contact the Program Chair for help in resolving this situation.
Ranking the Papers: To offer objectivity to the ratings of the referees, use the following system to rank the papers in your Subject Area. Award points as follows:
Compute the average for the evaluations you have and produce a list ordered by the average. That provides an objective assessment of the referees' evaluations. Note that these averages are important, but they are not a straight-jacket. Some judgment calls may be required, especially for papers "on the bubble" for acceptance.
Selecting the Papers: The selection process must be balanced between paper quality and the needs of your Subject Area. For example, if your top seven papers are all on the exact same subject, do you include ALL of them at the expense of good papers on other subjects? No. On the other hand, when deciding on the last few slots available for your area, the numeric average calculated earlier will be an important factor in the process. Of course, the SACs' overall knowledge of all submitted papers in their area will play a key part in final decisions as to which papers end up being accepted for the conference.
All Subject Areas which have multiple SACs: Each co-chair is responsible for an agreed-upon share of the papers. When it's time to schedule, the co-chairs have to put their heads together, check their respective paper rankings, and decide on accepted papers for their Subject Area. THIS MUST BE ACCOMPLISHED IN A FAIRLY SHORT TIME!
Best papers: As always, we will offer Best Paper Awards. Make some notes on what you think your best paper(s) may be, and also let the Program Chair know how many "Award Quality" votes the referees gave each paper in your area. This should be done the week prior to Agenda Scheduling Weekend.
Finally: Prepare and retain good records! They will undoubtedly prove useful at some point. Be sure to hang onto them until the conference.