background
logo
ArxivPaperAI

SeMaScore : a new evaluation metric for automatic speech recognition tasks

Author:
Zitha Sasindran, Harsha Yelchuri, T. V. Prabhakar
Keyword:
Electrical Engineering and Systems Science, Audio and Speech Processing, Audio and Speech Processing (eess.AS), Machine Learning (cs.LG), Sound (cs.SD)
journal:
--
date:
2024-01-15 00:00:00
Abstract
In this study, we present SeMaScore, generated using a segment-wise mapping and scoring algorithm that serves as an evaluation metric for automatic speech recognition tasks. SeMaScore leverages both the error rate and a more robust similarity score. We show that our algorithm's score generation improves upon the state-of-the-art BERTscore. Our experimental results show that SeMaScore corresponds well with expert human assessments, signal-to-noise ratio levels, and other natural language metrics. We outperform BERTscore by 41x in metric computation speed. Overall, we demonstrate that SeMaScore serves as a more dependable evaluation metric, particularly in real-world situations involving atypical speech patterns.
PDF: SeMaScore : a new evaluation metric for automatic speech recognition tasks.pdf
Empowered by ChatGPT