Accéder directement au contenu Accéder directement à la navigation
Article dans une revue

Assessing inter-annotator agreement from collaborative annotation campaign in marine bioacoustics

Abstract : It is currently widely recognized that automated methods are crucial to help processing long-term recordings of marine bioacoustics. To evaluate the efficiency of such methods, it is essential to develop large-scale annotated datasets. However, besides being laborious and resource intensive, recent studies have suggested that such a task could also be highly subjective with the generation of annotator specific errors. In this work, we investigate the question of inter-annotator agreement from a multi-annotator annotation campaign performed on a marine bioacoustics dataset. After providing quantitative evidence of inter-annotator variability, we investigate potential sources on both the user annotation practice and the annotation data and task to better understand why and how such variability occurs. Our study reveals that the acoustic event type, the Signal-to-Noise Ratio of the acoustic event and the annotator profile are three examples of critical factors impacting the annotation results of a multi-annotator campaign.
Liste complète des métadonnées

https://hal-ensta-bretagne.archives-ouvertes.fr/hal-03028306
Contributeur : Marie Briec <>
Soumis le : vendredi 5 février 2021 - 08:41:13
Dernière modification le : vendredi 19 février 2021 - 03:27:27

Fichier

 Accès restreint
Fichier visible le : 2021-08-05

Connectez-vous pour demander l'accès au fichier

Identifiants

Citation

Paul Nguyen Hong Duc, Maëlle Torterotot, Flore Samaran, Paul R. White, Odile Gérard, et al.. Assessing inter-annotator agreement from collaborative annotation campaign in marine bioacoustics. Ecological Informatics, Elsevier, 2021, 61, pp.101185-1 - 101185-12. ⟨10.1016/j.ecoinf.2020.101185⟩. ⟨hal-03028306⟩

Partager

Métriques

Consultations de la notice

78