Learn <\/span><\/span> how <\/span><\/span> modern <\/span><\/span> speech <\/span><\/span> recognition <\/span><\/span> algorithms <\/span><\/span> like <\/span><\/span> wav2vec <\/span><\/span> can <\/span><\/span> potentially <\/span><\/span> recognize <\/span><\/span> all <\/span><\/span> languages <\/span><\/span> in <\/span><\/span> the <\/span><\/span> world. <\/span><\/span> <\/h2><\/div><\/div>[vc_column_text css=”.vc_custom_1653562454284{padding-top: 40px !important;padding-right: 40px !important;padding-bottom: 0px !important;padding-left: 40px !important;}”]Many AI researchers believe that we are decades away from achieving Artificial General Intelligence<\/a> (AGI) or true human-like intelligence. Though far away, recent advancements in computer vision, generative modeling, natural language processing, and speech recognition have brought us closer to realizing AGI in the near future.<\/p>\nEfforts to improve the natural language understanding of computers go back to the 1950s<\/a>. Today, automatic speech recognition<\/a> (ASR) models can recognize many languages\u2013limited only by the availability of more diverse language datasets.<\/p>\nExisting speech recognition techniques require curated language datasets with hours of clean audio accompanied by transcribed text. Speech recognition models are trained on these datasets to learn language-specific representations. But what if a speech recognition model trained for English or French receives audio input of lesser-known languages like Tartar or Swahili?<\/p>\n
They will fail miserably or produce gibberish. So, we need more inclusivity in AI speech recognition systems.<\/strong><\/em><\/p>\nWith more than [7000 languages](https:\/\/www.consumersinternational.org\/news-resources\/blog\/posts\/over-7000-languages-are-spoken-in-the-world-today-but-not-many-are-represented-online\/#:~:text=Over 7000 languages are spoken across the world today.,cultures it is spoken in.)<\/a> spoken worldwide, it is impossible to curate high-quality<\/a> audio datasets and scale speech recognition systems for each of them. Researchers at Meta introduced an unsupervised speech recognition approach called wav2vec-unsupervised<\/a><\/strong><\/em> (wav2vec-U<\/em>) that does not require any labeled audio datasets.<\/p>\nIn this post, we\u2019ll take a deep dive into understanding unsupervised speech recognition and the wav2vec-U<\/em> model and how it compares with existing supervised speech recognition approaches.[\/vc_column_text]<\/div><\/div><\/div>[vc_column_text css=”.vc_custom_1653562481213{padding-top: 40px !important;padding-right: 20px !important;padding-bottom: 40px !important;padding-left: 20px !important;}”]<\/p>\nWhat is Unsupervised Speech Recognition?<\/strong><\/h3>\n <\/p>\n
Unsupervised speech recognition is an AI technique that trains speech recognition systems without any labeled audio data, eliminating the need for transcribed text. Without transcribed text, audio waveforms are directly fed into unsupervised models, where they are processed and converted to text. It is the key to unlocking inclusivity and diversity for speech recognition systems.[\/vc_column_text][vc_column_text css=”.vc_custom_1653562522789{padding-top: 40px !important;padding-right: 20px !important;padding-bottom: 40px !important;padding-left: 20px !important;}”]<\/p>\n
Unsupervised wav2Vec<\/em> Architecture Explained<\/strong><\/h3>\n <\/p>\n
wav2vec-U<\/em> requires no transcribed speech data as it learns purely from unlabeled audio. It is built on top of a self-supervised wav2vec 2.0<\/a><\/em> model and a generative adversarial network (GAN). Let\u2019s discuss these two components in detail:[\/vc_column_text][\/vc_column][\/vc_row][vc_section full_width=”stretch_row” pix_over_visibility=”” css=”.vc_custom_1650444445523{padding-top: 80px !important;padding-bottom: 80px !important;background-color: #f8f9fa !important;}” el_id=”pix_section_program”][vc_row full_width=”stretch_row” pix_particles_check=””][vc_column content_align=”text-center” offset=”vc_col-lg-offset-0 vc_col-lg-12 vc_col-md-offset-1 vc_col-md-10″][vc_column_text css=”.vc_custom_1653562631146{padding-top: 40px !important;padding-right: 20px !important;padding-bottom: 40px !important;padding-left: 20px !important;}”]<\/p>\n1. Self-Supervised wav2vec 2.0<\/span><\/strong><\/h3>\n <\/p>\n
![\"\"](\"https:\/\/blog.datumo.com\/en\/wp-content\/uploads\/2022\/05\/Untitled.png\")
<\/p>\n
<\/p>\n
wav2vec 2.0 architecture for learning contextual audio representations. Image by Meta AI<\/a><\/em><\/p>\nSpeech data is a continuous signal without any clear and discrete audio segments. The goal of wav2vec 2.0<\/em> is to learn discrete contextual speech representations from unlabeled audio. Modern ASR systems can perform better speech recognition for various languages using these contextual speech representations.<\/p>\nFirst, the raw audio waveform for any language is passed through a multilayer Convolutional Neural Network<\/a> (CNN) feature encoder to obtain discrete latent speech units that are 25ms long.<\/p>\nThe speech units are then passed through a quantization module using the product quantization technique<\/a> to map speech units more accurately to their learned audio representations.<\/p>\nFinally, the quantized speech representations are passed to a transformer<\/a> encoder where about 50% of the audio is masked to initiate self-supervision. Using a contrastive learning<\/a> approach, the transformer predicts these masked representations in the audio signal.<\/p>\nOver time, the self-supervised model improves and learns more accurate high-level contextual speech representations from the input audio. The self-supervised output model better understands what human voice sounds like and is ready for speech recognition.<\/p>\n
<\/p>\n
<\/h3>\n2. Generative Adversarial Network (GAN)<\/strong><\/h3>\n <\/p>\n
The output of the pre-trained self-supervised wav2vec2.0<\/em> model is passed to a GAN that has a generator and a discriminator neural network.<\/p>\nAt first, the generator outputs non-sensical transcription from the self-supervised input speech representations. The output transcriptions of the generator are phonemes which are distinct units of sound corresponding to the sound in the input language.<\/p>\n
The output phonemes are fed into the discriminator to improve transcriptions, which decides whether the generated text is good or makes sense at all.<\/p>\n
The discriminator compares these phonemes with real language text and gives feedback to the generator to improve transcriptions. This process is repeated numerous times until the generator transcriptions resemble real language text.<\/p>\n
Adding a GAN system enables unsupervised speech recognition, eliminating the need for any annotated speech data at all.<\/p>\n
[\/vc_column_text][\/vc_column][\/vc_row][\/vc_section][vc_row pix_particles_check=””][vc_column]
<\/div>
Efforts to improve the natural language understanding of computers go back to the 1950s<\/a>. Today, automatic speech recognition<\/a> (ASR) models can recognize many languages\u2013limited only by the availability of more diverse language datasets.<\/p>\n Existing speech recognition techniques require curated language datasets with hours of clean audio accompanied by transcribed text. Speech recognition models are trained on these datasets to learn language-specific representations. But what if a speech recognition model trained for English or French receives audio input of lesser-known languages like Tartar or Swahili?<\/p>\n They will fail miserably or produce gibberish. So, we need more inclusivity in AI speech recognition systems.<\/strong><\/em><\/p>\n With more than [7000 languages](https:\/\/www.consumersinternational.org\/news-resources\/blog\/posts\/over-7000-languages-are-spoken-in-the-world-today-but-not-many-are-represented-online\/#:~:text=Over 7000 languages are spoken across the world today.,cultures it is spoken in.)<\/a> spoken worldwide, it is impossible to curate high-quality<\/a> audio datasets and scale speech recognition systems for each of them. Researchers at Meta introduced an unsupervised speech recognition approach called wav2vec-unsupervised<\/a><\/strong><\/em> (wav2vec-U<\/em>) that does not require any labeled audio datasets.<\/p>\n In this post, we\u2019ll take a deep dive into understanding unsupervised speech recognition and the wav2vec-U<\/em> model and how it compares with existing supervised speech recognition approaches.[\/vc_column_text]<\/div><\/div> <\/p>\n Unsupervised speech recognition is an AI technique that trains speech recognition systems without any labeled audio data, eliminating the need for transcribed text. Without transcribed text, audio waveforms are directly fed into unsupervised models, where they are processed and converted to text. It is the key to unlocking inclusivity and diversity for speech recognition systems.[\/vc_column_text][vc_column_text css=”.vc_custom_1653562522789{padding-top: 40px !important;padding-right: 20px !important;padding-bottom: 40px !important;padding-left: 20px !important;}”]<\/p>\n <\/p>\n wav2vec-U<\/em> requires no transcribed speech data as it learns purely from unlabeled audio. It is built on top of a self-supervised wav2vec 2.0<\/a><\/em> model and a generative adversarial network (GAN). Let\u2019s discuss these two components in detail:[\/vc_column_text][\/vc_column][\/vc_row][vc_section full_width=”stretch_row” pix_over_visibility=”” css=”.vc_custom_1650444445523{padding-top: 80px !important;padding-bottom: 80px !important;background-color: #f8f9fa !important;}” el_id=”pix_section_program”][vc_row full_width=”stretch_row” pix_particles_check=””][vc_column content_align=”text-center” offset=”vc_col-lg-offset-0 vc_col-lg-12 vc_col-md-offset-1 vc_col-md-10″][vc_column_text css=”.vc_custom_1653562631146{padding-top: 40px !important;padding-right: 20px !important;padding-bottom: 40px !important;padding-left: 20px !important;}”]<\/p>\n <\/p>\n <\/p>\n wav2vec 2.0 architecture for learning contextual audio representations. Image by Meta AI<\/a><\/em><\/p>\n Speech data is a continuous signal without any clear and discrete audio segments. The goal of wav2vec 2.0<\/em> is to learn discrete contextual speech representations from unlabeled audio. Modern ASR systems can perform better speech recognition for various languages using these contextual speech representations.<\/p>\n First, the raw audio waveform for any language is passed through a multilayer Convolutional Neural Network<\/a> (CNN) feature encoder to obtain discrete latent speech units that are 25ms long.<\/p>\n The speech units are then passed through a quantization module using the product quantization technique<\/a> to map speech units more accurately to their learned audio representations.<\/p>\n Finally, the quantized speech representations are passed to a transformer<\/a> encoder where about 50% of the audio is masked to initiate self-supervision. Using a contrastive learning<\/a> approach, the transformer predicts these masked representations in the audio signal.<\/p>\n Over time, the self-supervised model improves and learns more accurate high-level contextual speech representations from the input audio. The self-supervised output model better understands what human voice sounds like and is ready for speech recognition.<\/p>\n <\/p>\n <\/p>\n The output of the pre-trained self-supervised wav2vec2.0<\/em> model is passed to a GAN that has a generator and a discriminator neural network.<\/p>\n At first, the generator outputs non-sensical transcription from the self-supervised input speech representations. The output transcriptions of the generator are phonemes which are distinct units of sound corresponding to the sound in the input language.<\/p>\n The output phonemes are fed into the discriminator to improve transcriptions, which decides whether the generated text is good or makes sense at all.<\/p>\n The discriminator compares these phonemes with real language text and gives feedback to the generator to improve transcriptions. This process is repeated numerous times until the generator transcriptions resemble real language text.<\/p>\n Adding a GAN system enables unsupervised speech recognition, eliminating the need for any annotated speech data at all.<\/p>\n [\/vc_column_text][\/vc_column][\/vc_row][\/vc_section][vc_row pix_particles_check=””][vc_column]What is Unsupervised Speech Recognition?<\/strong><\/h3>\n
Unsupervised wav2Vec<\/em> Architecture Explained<\/strong><\/h3>\n
1. Self-Supervised wav2vec 2.0<\/span><\/strong><\/h3>\n
<\/p>\n<\/h3>\n
2. Generative Adversarial Network (GAN)<\/strong><\/h3>\n