The ICASSP Paper Review Process
Steve Young
SLTC Newsletter, October 2009
The winter holidays are approaching and that signals one of the busiest times of the year for the Speech and Language Processing Technical Committee. More than 600 papers have been submitted to ICASSP 2010 in the speech and language area and it is the job of the technical committee (TC), and especially the four Area Chairs, to oversee the process by which papers are reviewed, selected and assigned to sessions.
The first stage of the process is to allocate reviewers to papers in a way which ensures as best we can that papers are assigned to reviewers with the appropriate expertise and that there are no obvious conflicts of interest. This is done by firstly allocating papers based on their specified EDICS numbers and then manually checking the assignments.
Our primary goal in managing the review process itself is to ensure that it is as fair and as transparent as possible. To help ensure this, we allocate four reviewers to every paper and we guarantee that all papers will receive at least three reviews. One of the four reviewers will be a TC member and once all the reviews are complete, the TC member conducts a meta-review of each paper assigned to him or her. When all of the reviewers agree, this meta-review will simply record the consensus opinion. However, when there is disagreement or the paper is on the borderline, the TC member will examine each of the reviews carefully and weigh the evidence provided by the reviewers to support the scores they gave. After weighing the evidence, the TC member makes a firm recommendation either one way or the other. By this process we hope to avoid papers being rejected because a reviewer misunderstood the paper or did not read it properly.
The final stage of the process is to make the accept/reject decisions and allocate the accepted papers to sessions. Not surprisingly, this typically requires quite a few iterations to get right. However, eventually the programme is finalised and the results are announced to the authors. At this point, the Area Chairs will breath a huge sigh of relief.
The ICASSP review process is hard work for all of the TCs, but it is especially so for our TC simply because we have the most papers. Over 20% of all papers submitted to ICASSP are in the speech and language area. This year the TC members and our panel of reviewers will together conduct over 2,500 reviews, and TC members will then conduct more than 600 meta-reviews. Guiding all of this will be the Area Chairs: Pascale Fung, TJ Hazen, Thomas Hain and Brian Kingsbury. The Area Chairs are the unsung heroes of ICASSP - they work really hard for many weeks to keep the process on track and on schedule. So I would like to end by thanking in advance all of the volunteers who will contribute their time and expertise to the ICASSP review process and especially the Area Chairs on whom the process so crucially depends.
Steve Young is Chair of the Speech and Language Processing Technical Committee of the IEEE Signal Processing Society.
Add A Comment