Connect:  Facebook  Twitter  LinkedIn

Audio and Acoustic Signal Processing Technical Committee

AASP Challenges


--> Click here for Call for participation for active challenges -->>


The AASP Technical Committee runs a series of ‘Challenges’ in order to encourage research and development with comparable and repeatable results, and to stimulate new ground-breaking approaches to specific problems in the AASP technical scope. This activity is coordinated by the Challenges Subcommittee listed below.

Call for Challenges

Proposals to organize an AASP Challenge are invited. This is an open call with no deadline. Please email a Stage 1 - Statement of Interest (see below) to the Challenges subcommittee chair (p.naylor@imperial.ac.uk).

  1. Stage 1 - Statement of Interest
  2. To propose to organise a Challenge, please send a statement of interest in about 2 pages outlining the aim of the challenge and its value to the community, giving also a preliminary perspective of the practical elements including the planned test data and evaluation methodology.

  3. Stage 2 - Full Proposal
  4. If supported by the Subcommittee, a full proposal would be invited. The full proposal should include the following items:

    • a textual description of the challenge and its context (1 to 2 pages);
    • a clear formulation of the problem to be addressed;
    • a specification of the evaluation methodology leading to an objective figure of merit (FoM) and, where appropriate, a software tool to compute the FoM;
    • a development dataset which represents the challenge and which will be made public (a public training dataset may also be needed in some challenges);
    • a test dataset which also represents the challenge but which will remain private during the challenge;
    • a commitment to provide a website to disseminate the challenge itself and, eventually, the results;
    • a commitment to evaluate the submitted results and publish the comparison on the website and elsewhere as appropriate;
    • a proposed schedule for the challenge (date of publication of the challenge, deadline of results submission, deadline of comparative results publication).

    Please send the Statement of Interest and the Full Proposal to the AASP Challenges Subcommittee chair at the address below. All proposals will be considered by the Challenges Subcommittee. The Subcommittee may request modifications to the challenge as a condition of acceptance.

Participation

Researchers entering the challenge are invited to sign up at the challenge website. Participants will address the challenge specification and employ the evaluation methodology and the development dataset to develop their algorithm. Participation is open to all.

Evaluation

At the end of the challenge, the organizers will coordinate a comparative evaluation employing the defined FoM. Evaluation may be done for example by releasing the test dataset and asking the participants to return their results on the test data within a short period of time, typically two weeks. Participants would be honour-bound not to use the test dataset for tuning.

Dissemination

The evaluation results will then be published by the organizers. The Challenges Subcommittee will work with the challenge organizers towards publication of the challenge and its outcome in the IEEE Transactions and appropriate conferences, ICASSP in particular. In addition, the challenge organizers’ website will be linked from the TC webpage. Participants can choose to remain anonymous in publications.

The 'AASP Challenges' Subcommittee

The current membership of the subcommittee is as follows:

Patrick Naylor Imperial College London, UK (Chair) p.naylor@imperial.ac.uk
Laurent Daudet Paris Diderot University, France
Jean-Marc Jot DTS Inc., USA
Bryan Pardo Northwestern University, USA
Mark Plumbley Queen Mary University of London, UK
Gael Richard TELECOM ParisTech, France
Ivan Tashev Microsoft Research, USA
Emmanuel Vincent INRIA, France

IEEE AASP CHALLENGES 2013

REVERB


Call for Participation:

Recently, substantial progress has been made in the field of reverberant speech signal processing, including both single- and multi-channel de-reverberation techniques, and automatic speech recognition (ASR) techniques robust to reverberation. To evaluate state-of-the-art algorithms and draw new insights regarding potential future research directions, we call for participation in the REVERB (REverberant Voice Enhancement and Recognition Benchmark) challenge that will provide an opportunity to the researchers in the field to carry out comprehensive evaluation of their methods based on a common database and evaluation metrics. The challenge aims at bringing together researchers from a broad range of disciplines to discuss novel and established approaches to handle reverberant speech. This challenge is part of the IEEE SPS AASP challenge series. The following is an outline description of the challenge.

REVERB Challenge Overview

  • Data: Real and simulated 1-, 2-, and 8-channel recordings in reverberant meeting rooms based on the Wall Street Journal Corpus. This data is common for the following 2 tasks*.

  • Task 1: Enhancement of reverberant speech with single-/multi-channel de-reverberation techniques

    (Evaluation metrics: objective and subjective measures)

  • Task 2: Robust recognition of reverberant speech

    (Evaluation metric: word error rate)

    * Participants are invited to take part in either or both of the above tasks.

  • REVERB Challenge Workshop

    The results will be presented by the participants at the REVERB challenge workshop, which will be held in conjunction with ICASSP2014 and HSCMA2014.

    Important Dates

    July

    1st,

    2013

    Release of development dataset and scripts for evaluation

    Nov

    5,

    2013

    Release of evaluation dataset

    Dec

    1,

    2013

    Deadline for submission of results

    Jan

    10,

    2014

    Deadline for submission of papers

    Feb

    28,

    2014

    Notification of acceptance

    May

    10,

    2014

    Workshop in conjunction with ICASSP2014 (Florence, Italy)

    Further details at reverb2014.dereverberation.com

    Organizing Committee
    Marc Delcroix (NTT), Sharon Gannot (Bar-Ilan Univ.), Emannuel Habets (International Audio Labs Erlangen), Reinhold Haeb-Umbach (Paderborn Univ.), Walter Kellermann (Univ. of Erlangen-Nuremberg), Keisuke Kinoshita (NTT), Volker Leutnant (Paderborn Univ.), Roland Maas (Univ. of Erlangen-Nuremberg), Tomohiro Nakatani (NTT), Bhiksha Raj (Carnegie Mellon Univ.), Armin Sehr (Beuth Univ. of Applied Sciences Berlin), Takuya Yoshioka (NTT)

    IEEE AASP CHALLENGES 2012

    Detection and Classification of Acoustic Scenes and Events


    Call for Participation:

    On behalf of the IEEE AASP Technical Committee, I am happy to announce a new challenge entitled: "Detection and Classification of Acoustic Scenes and Events". The challenge has the form of a public contest for the evaluation of the performance of systems for the detection and classification of acoustic events and audio scenes.

    The challenge includes a set of tasks for the detection and classification of acoustic scenes and events and its goal is to provide a focus of attention for the scientific community in developing systems for computational auditory scene analysis (CASA) that will encourage sharing of ideas and improve the state of the art, potentially leading to the development of systems that achieve performance closer to that of humans.

    This challenge will help the research community move forward by providing a focus for better defining the specific tasks, and will also provide incentive for researchers to actively pursue research on this field. Finally, it will offer a reference point for future systems developed to perform similar tasks and it will provide the community with a high quality database for future research.

    There will be a discussion phase where potential participants are invited to contribute their ideas, ending on 30th September 2012. The deadline for code submission is 31st March 2013. Results will be presented at a special session in WASPAA 2013 (http://www.waspaa.com/); participants are invited to present a poster at a special session. Also, authors of novel work are encouraged to submit their work as a regular paper at WASPAA 2013.

    For more details as well as a copy of the full proposal of the challenge please visit:
    http://www.elec.qmul.ac.uk/digitalmusic/sceneseventschallenge/

    The challenge organisers,
    Dimitrios Giannoulis (QMUL), Emmanouil Benetos (QMUL), Dan Stowell (QMUL), Mathieu Lagrange (IRCAM) and Mark Plumbley (QMUL)


    2nd CHiME Speech Separation and Recognition Challenge

    ----------------------------------------------

    Deadline: January 15, 2013
    Workshop: June 1, 2013, Vancouver, Canada

    http://spandh.dcs.shef.ac.uk/chime_challenge/

    ----------------------------------------------

    Following the success of the 1st PASCAL CHiME Speech Separation and
    Recognition Challenge, we are happy to announce a new challenge
    dedicated to speech recognition in real-world reverberant, noisy conditions,
    that will culminate in a dedicated satellite workshop of ICASSP 2013.

    The challenge is supported by several IEEE Technical Committees and by
    an Industrial Board.

    FEATURED TASKS

    The challenge consists of recognising distant-microphone speech mixed in
    two-channel nonstationary noise recorded over a period of several weeks
    in a real family house. Entrants may address either one or both of the
    following tracks:

    Medium vocabulary track: WSJ 5k sentences uttered by a static speaker

    Small vocabulary track: simpler commands but small head movements

    TO ENTER

    You will find everything you need to get started (and even more) on the
    challenge website:
    - a full description of the challenge,
    - clean, reverberated and multi-condition training and development data,
    - baseline training, decoding and scoring software tools based on HTK.

    Submission consists of a 2- to 8-page paper describing your system and
    reporting its performance on the development and the test set. In
    addition, you are welcome to submit an earlier paper to ICASSP 2013,
    which will tentatively be grouped with other papers into a dedicated
    session.

    Any approach is welcome, whether emerging or established.

    If you are interested in participating, please email us so we can
    monitor interest and send you further updates about the challenge.

    BEST CHALLENGE PAPER AWARD

    The best challenge paper will distinguished by an award from the
    Industrial Board.

    IMPORTANT DATES

    July 2012 Launch
    October 2012 Test set release
    January 15, 2013 Challenge & workshop submission deadline
    February 18, 2013 Paper notification & release of the challenge results
    June 1, 2013 ICASSP satellite workshop

    INDUSTRIAL BOARD

    Masami Akamine, Toshiba
    Carlos Avendano, Audience
    Li Deng, Microsoft
    Erik McDermott, Google
    Gautham Mysore, Adobe
    Atsushi Nakamura, NTT
    Peder A. Olsen, IBM
    Trausti Thormundsson, Conexant
    Daniel Willett, Nuance

    WORKSHOP SPONSORS

    Conexant Systems Inc.
    Audience Inc.
    Mitsubishi Electric Research Laboratories

    ORGANISERS

    Emmanuel Vincent, INRIA
    Jon Barker, University of Sheffield
    Shinji Watanabe & Jonathan Le Roux, MERL
    Francesco Nesta & Marco Matassoni, FBK-IRST