Real-Time Speech-to-Text And Speaker Diarization Using Whisper And Pyannote Embeddings
Author :
Shivani Chauhan, Rimmy, Ashish PrajapatiJourna Name:
International Journal of Science, Engineering and Technology Volume:
14 issue:4 Year:Volume-14-issue-4 Views : 120
Abstract:
Speech-to-text (STT) and multi-speaker diarization have emerged as crucial components in intelligent communication systems, virtual meeting platforms, assistive technologies, and large-scale multimedia analytics. Recent advancements in transformer-based architectures such as Whisper and self-supervised pipelines like Pyannote have significantly improved transcription quality and speaker discrimination, enabling highly accurate multi-speaker processing even on consumer hardware. A system that integrates Whisper for multilingual transcription, Pyannote for diarization, and a new speaker identity recognition module that uses voice embedding is presented in this research as a modular, scalable, and real-time integrated system. Real-time microphone input, audio uploads, and live transcript visualization are all supported by a full-stack web platform that uses React and Flask. The Word Error Rate (WER) of 6.2% and the Diarization Error Rate (DER) of 13% were observed in experiments conducted on 5 hours of multi-speaker audio. 87% identity accuracy is achieved by personalized speaker recognition. Practical usability is demonstrated by the proposed system for meetings, lectures, podcasts, and automated captioning.
APA:Shivani Chauhan, Rimmy, Ashish Prajapati. (Volume-14, Issue-4 -(Year-Volume-14-issue-4)). Real-Time Speech-to-Text And Speaker Diarization Using Whisper And Pyannote Embeddings. Retrieved from https://www.ijset.in/wp-content/uploads/IJSET_V14_issue4_115.pdf
Chicago:Shivani Chauhan, Rimmy, Ashish Prajapati. "Real-Time Speech-to-Text And Speaker Diarization Using Whisper And Pyannote Embeddings" Example, Volume-14-issue-4-Year-Volume-14-issue-4-2348-4098. https://www.ijset.in/wp-content/uploads/IJSET_V14_issue4_115.pdf.