| Paper ID | AUD-23.2 |
| Paper Title |
IMPROVING SOUND EVENT DETECTION METRICS: INSIGHTS FROM DCASE 2020 |
| Authors |
Giacomo Ferroni, Audio Analytic, United Kingdom; Nicolas Turpault, INRIA, France; Juan Azcarreta, Francesco Tuveri, Audio Analytic, United Kingdom; Romain Serizel, LORIA, France; Cagdas Bilen, Sacha Krstulovic, Audio Analytic, United Kingdom |
| Session | AUD-23: Detection and Classification of Acoustic Scenes and Events 4: Datasets and metrics |
| Location | Gather.Town |
| Session Time: | Thursday, 10 June, 15:30 - 16:15 |
| Presentation Time: | Thursday, 10 June, 15:30 - 16:15 |
| Presentation |
Poster
|
| Topic |
Audio and Acoustic Signal Processing: [AUD-CLAS] Detection and Classification of Acoustic Scenes and Events |
| IEEE Xplore Open Preview |
Click here to view in IEEE Xplore |
| Virtual Presentation |
Click here to watch in the Virtual Conference |
| Abstract |
The ranking of sound event detection (SED) systems may be biased by assumptions inherent to evaluation criteria and to the choice of an operating point. This paper compares conventional event-based and segment-based criteria against the Polyphonic Sound Detection Score (PSDS)'s intersection-based criterion, over a selection of systems from DCASE 2020 Challenge Task 4. It shows that, by relying on collars, the conventional event-based criterion introduces different strictness levels depending on the length of the sound events, and that the segment-based criterion may lack precision and be application dependent. Alternatively, PSDS's intersection-based criterion overcomes the dependency of the evaluation on sound event duration and provides robustness to labelling subjectivity, by allowing valid detections of interrupted events. Furthermore, PSDS enhances the comparison of SED systems by measuring sound event modelling performance independently from the systems' operating points. |
