Case Study: Automatic Speech Recognition

Solution

With our deep understanding of conversational AI, we helped the client collect, transcribe the audio data with a team of expert collectors, linguists and annotators to build large corpus of audio data from remote parts of India.

The scope of work for Shaip included but was not limited to acquiring large volumes of audio training data, transcribing the data and delivering corresponding JSON files containing the metadata [for both speakers and transcribers. For each speaker, the metadata includes an anonymized Speaker ID, device details, demographic information like gender, age, and education, along with their pincode, socio-economic status, languages spoken, and a record of their life’s stay duration. For every transcriber, the data incorporates an anonymized Transcriber ID, demographic details similar to the speakers’, their transcription experience duration, and a thorough breakdown of languages they can read, write, and speak.

Shaip collected 8000 hours of audio data / Spontaneous speech at scale and transcribed 800 hours while maintaining desired levels of quality required to train speech technology for complex projects. Explicit Consent Form was taken from each of the participants. The / Spontaneous speech collected was based on University-provided images. Of 3500 images, 1000 are generic and 2500 relate to district-specific culture, festivals, etc. Images depict various domains like train stations, markets, weather, and more.

Data Collection

General Guidelines

Format

• • Audio at 16 kHz, 16 bits/sample.
  • Single channel.
  • Raw audio without transcoding.

Style

• • Spontaneous speech.
  • Sentences based on University-provided images. Of 3500 images, 1000 are generic and 2500 relate to district-specific culture, festivals, etc. Images depict various domains like train stations, markets, weather, and more.

Recording Background

• • Recorded in a quiet, echo-free environment.
  • No smartphone disturbances (vibration or notifications) during recording.
  • No distortions like clipping or far-field effects.
  • Vibrations from phone unacceptable; external vibrations are tolerable if audio is clear.

Speaker Specification

• • Age range from 20-70 years with balanced gender distribution per district.
  • Minimum of 400 native speakers in each district.
  • Speakers should use their home language/dialect.
  • Consent forms mandatory for all participants.

Quality Check & Critical Quality Assurance

The QA process prioritizes quality assurance for audio recordings and transcriptions. Audio standards focus on precise silences, segment duration, single-speaker clarity, and detailed metadata including age and socio-economic status. Transcription criteria emphasize tag accuracy, word veracity, and correct segment details. The acceptance benchmark dictates that if more than 20% of an audio batch fails these standards, it’s rejected. For less than 20% discrepancies, replacement recordings with similar profiles are required.

Data Transcription

Transcription guidelines emphasize accuracy and verbatim transcription only when words are clear and understandable; unclear words are marked as [unintelligible] or [inaudible] based on the issue. Sentence boundaries in long audio are marked with <SEGMENT>, and no paraphrasing or correction of grammatical errors is allowed. Verbatim transcription covers errors, slangs, and repetitions but omits false starts, filler sounds, and stutters. Background and foreground noises are transcribed with descriptive tags, while proper names, titles, and numbers follow specific transcription rules. Speaker labels are used for every sentence, and incomplete sentences are indicated with.

Project Workflow

The workflow describes the audio transcription process. It starts with onboarding and training participants. They record audio using an app, which is uploaded to a QA platform. This audio undergoes quality checks and automatic segmentation. The tech team then prepares segments for transcription. After manual transcription, there’s a quality assurance step. Transcriptions are delivered to the client, and if accepted, the delivery is deemed complete. If not, revisions are made based on client feedback.