IITM BS Speech Technology (BSEE4001): Syllabus and Tips

Speech Technology (BSEE4001) is a 4 credit degree level elective on how machines recognise, produce and analyse human speech. It starts with signals and classic statistical models, then moves to neural and end-to-end systems for speech recognition (ASR) and text to speech (TTS). The course page and the handbook list no prerequisites. The instructor is Prof. S. Umesh of the Department of Electrical Engineering, IIT Madras.

The DS handbook's degree level table tags it BD. The course page calls it an elective. The two sources agree on credits and prerequisites. The ES handbook lists the course as EE4001 "Speech Technologies", an open elective for Electronic Systems students, also with no prerequisite.

What you learn

The course page lists 12 weeks. In plain words:

• Weeks 1 to 3: signals and classic models. A review of signals and systems, Fourier transforms and correlation. Then features taken from speech audio, Gaussian mixture models, SVD and hidden Markov models (HMMs), with examples of HMMs in ASR, TTS and speaker diarization (working out who spoke when).
• Weeks 4 and 5: classic ASR and TTS. The history of both, and the parts of an ASR system: the acoustic model, the lexicon and the n-gram language model. Then monophone and triphone HMMs, and two older ways to make speech: unit selection and statistical parametric synthesis.
• Weeks 6 to 9: neural and end-to-end systems. Neural networks for acoustic modelling (DNN, CNN and TDNN hybrids), CTC, and encoder-decoder models built first with RNNs and then with transformers, for both ASR and TTS.
• Weeks 10 to 12: speakers and newer problems. Speaker recognition and verification with i-vectors and x-vectors, diarization, speaker adaptation with s-vectors, speech that mixes languages (code-switched speech), speech translation, singing voice synthesis and voice conversion.

The page also says you will get hands-on experience with speech processing toolkits.

How it is assessed

The course page lists 12 weeks of coursework, weekly online assignments, 2 in-person invigilated quizzes and 1 in-person invigilated end term exam. It does not mention an OPPE or a project.

Where it counts

• Minor in Multimodal AI Systems. Speech Technology is one of its three courses, with Deep Learning for Computer Vision and Large Language Models. The minor also needs Deep Learning Practice as a prerequisite. See the Multimodal AI Systems minor.
• BS level data science stream. The BS level needs 2 data science stream courses at level 4 or higher. This course is L4 and tagged BD, which appears to mean that stream. The handbook does not define BD, so confirm with support. See the BS degree level.
• Term plan. The March 2026 table marks it as not offered in May 2026, and offered in September 2026 and January 2027.

Who finds it hard and how to prepare

Different students struggle in different weeks. If you never studied signals, weeks 1 and 2 move fast, because Fourier transforms are reviewed, not taught from zero. If you know deep learning but not probability models, the HMM and GMM weeks feel heavy.

• Before the term, learn what a Fourier transform does in plain terms. Load a short recording of your own voice in Python and plot its spectrogram. Seeing speech as a picture makes the feature weeks easier.
• Revise Gaussian distributions and conditional probability from Statistics 2. GMMs and HMMs are built on them.
• Revise SVD from Machine Learning Foundations. It shows up in week 2.
• Before week 6, make sure RNNs, encoder-decoder models, attention and transformers from Deep Learning are clear. Its last three weeks cover them.

What to take before and after

• Before: Deep Learning (BSCS3004), a mandatory BSc level course. It is not a listed prerequisite, but weeks 6 to 9 lean on it.
• With or after: Deep Learning Practice, also taught by Prof. Umesh. Its weeks 5 to 8 cover spoken language identification, speaker diarisation, speech to text, text to speech and wake word detection. The two courses pair well: one gives the theory, the other the practice.
• Other options: the full elective list is in IITM BS Data Science electives.

Common questions

Can a Data Science student take a BSEE course?

Yes. The code starts with BSEE, but the DS handbook lists it as a normal degree level elective tagged BD. You register for it like any other elective.

Do I need a signal processing background?

It is not a prerequisite, and week 1 reviews signals and systems. If you never studied them, spend a few hours on the basics before the term starts.

Which books does the course suggest?

Theory and Application of Digital Speech Processing by Rabiner and Schafer, Fundamentals of Speech Recognition by Rabiner, Juang and Yegnanarayana, and Spoken Language Processing by Huang, Acero and Hon. The page also points to recent research papers.