WAVLab

selected publications

1. SD CSL

   A review of speaker diarization: Recent advances with deep learning

   Tae Jin Park, Naoyuki Kanda, Dimitrios Dimitriadis, Kyu J Han, Shinji Watanabe, and Shrikanth Narayanan

   Computer Speech & Language 2022
2. ASR&SD&SLU&ER Interspeech

   SUPERB: Speech processing Universal PERformance Benchmark

   Shu-wen Yang, Po-Han Chi, Yung-Sung Chuang, Cheng-I Lai, Kushal Lakhotia, Yist Y., Andy T., Jiatong Shi, Xuankai Chang, Guan-Ting Lin, Tzu-Hsien Huang, Wei-Cheng Tseng, Ko-tik Lee, Da-Rong Liu, Zili Huang, Shuyan Dong, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, and Hung-yi Lee

   In Proceedings of Interspeech 2021

   arXiv HTML PDF
3. TTS ICASSP

   Espnet-TTS: Unified, reproducible, and integratable open source end-to-end text-to-speech toolkit

   Tomoki Hayashi, Ryuichi Yamamoto, Katsuki Inoue, Takenori Yoshimura, Shinji Watanabe, Tomoki Toda, Kazuya Takeda, Yu Zhang, and Xu Tan

   In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2020

   arXiv HTML PDF Code
4. ST ACL

   ESPnet-ST: All-in-One Speech Translation Toolkit

   Hirofumi Inaguma, Shun Kiyono, Kevin Duh, Shigeki Karita, Nelson Yalta, Tomoki Hayashi, and Shinji Watanabe

   In Proceedings of the Annual Meeting of the Association for Computational Linguistics 2020

   Code
5. SD Interspeech

   End-to-end speaker diarization for an unknown number of speakers with encoder-decoder based attractors

   Shota Horiguchi, Yusuke Fujita, Shinji Watanabe, Yawen Xue, and Kenji Nagamatsu

   2020

   arXiv Code
6. SD ASRU

   End-to-end neural speaker diarization with self-attention

   Yusuke Fujita, Naoyuki Kanda, Shota Horiguchi, Yawen Xue, Kenji Nagamatsu, and Shinji Watanabe

   In IEEE Automatic Speech Recogiton and Understanding Workshop (ASRU) 2019

   arXiv HTML PDF
7. SD Interspeech

   End-to-End Neural Speaker Diarization with Permutation-Free Objectives

   Yusuke Fujita, Naoyuki Kanda, Shota Horiguchi, Kenji Nagamatsu, and Shinji Watanabe

   In Proceedings of Interspeech 2019

   HTML PDF
8. ASR Interspeech

   Improving Transformer Based End-to-End Speech Recognition with Connectionist Temporal Classification and Language Model Integration

   Shigeki Karita, Nelson Yalta, Shinji Watanabe, Marc Delcroix, Atsunori Ogawa, and Tomohiro Nakatani

   In Proceedings of Interspeech 2019

   HTML PDF
9. ASR Interspeech

   ESPnet: End-to-End Speech Processing Toolkit

   Shinji Watanabe, Takaaki Hori, Shigeki Karita, Tomoki Hayashi, Jiro Nishitoba, Yuya Unno, Nelson Enrique Yalta Soplin, Jahn Heymann, Matthew Wiesner, Nanxin Chen, Adithya Renduchintala, and Tsubasa Ochiai

   Proceedings of Interspeech 2018

   Abs arXiv HTML PDF Code

   This paper introduces a new open source platform for end-to-end speech processing named ESPnet. ESPnet mainly focuses on end-to-end automatic speech recognition (ASR), and adopts widely-used dynamic neural network toolkits, Chainer and PyTorch, as a main deep learning engine. ESPnet also follows the Kaldi ASR toolkit style for data processing, feature extraction/format, and recipes to provide a complete setup for speech recognition and other speech processing experiments. This paper explains a major architecture of this software platform, several important functionalities, which differentiate ESPnet from other open source ASR toolkits, and experimental results with major ASR benchmarks.

10. SE&ASR Interspeech

    The Fifth ’CHiME’ Speech Separation and Recognition Challenge: Dataset, Task and Baselines

    Jon Barker, Shinji Watanabe, Emmanuel Vincent, and Jan Trmal

    Proceedings of Interspeech 2018

    arXiv HTML PDF
11. SD Interspeech

    Diarization is Hard: Some Experiences and Lessons Learned for the JHU Team in the Inaugural DIHARD Challenge.

    Gregory Sell, David Snyder, Alan McCree, Daniel Garcia-Romero, Jesús Villalba, Matthew Maciejewski, Vimal Manohar, Najim Dehak, Daniel Povey, Shinji Watanabe, and others

    In Interspeech 2018

    Abs HTML PDF

    We describe in this paper the experiences of the Johns Hopkins University team during the inaugural DIHARD diarization evaluation. This new task provided microphone recordings in a variety of difficult conditions and challenged researchers to fully consider all speaker activity, without the currently typical practices of unscored collars or ignored overlapping speaker segments. This paper explores several key aspects of currently state-of-the-art diarization methods, such as training data selection, signal bandwidth for feature extraction, representations of speech segments (i-vector versus x-vector) and domain-adaptive processing. In the end, our best system clustered x-vector embeddings trained on wideband microphone data followed by Variational-Bayesian refinement and a speech activity detector specifically trained for this task with in-domain data was found to be the best performing. After presenting these decisions and their final result, we discuss lessons learned and remaining challenges within the lens of this new approach to diarization performance measurement.