High fidelity speech regeneration with application to speech enhancement

Adam Polyak, Lior Wolf, Yossi Adi, Ori Kabeli, Yaniv Taigman

Research output: Contribution to journalConference articlepeer-review

11 Scopus citations


Speech enhancement has seen great improvement in recent years mainly through contributions in denoising, speaker separation, and dereverberation methods that mostly deal with environmental effects on vocal audio. To enhance speech beyond the limitations of the original signal, we take a regeneration approach, in which we recreate the speech from its essence, including the semi-recognized speech, prosody features, and identity. We propose a wav-to-wav generative model for speech that can generate 24khz speech in a real-time manner and which utilizes a compact speech representation, composed of ASR and identity features, to achieve a higher level of intelligibility. Inspired by voice conversion methods, we train to augment the speech characteristics while preserving the identity of the source using an auxiliary identity network. Perceptual acoustic metrics and subjective tests show that the method obtains valuable improvements over recent baselines.

Original languageAmerican English
Pages (from-to)7143-7147
Number of pages5
JournalProceedings - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing
StatePublished - 2021
Externally publishedYes
Event2021 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2021 - Virtual, Toronto, Canada
Duration: 6 Jun 202111 Jun 2021

Bibliographical note

Publisher Copyright:
© 2021 IEEE


  • Audio generation
  • Speech enhancement


Dive into the research topics of 'High fidelity speech regeneration with application to speech enhancement'. Together they form a unique fingerprint.

Cite this