IITM BS Sequential Decision Making (BSDA6004): Syllabus and Tips

Sequential Decision Making (BSDA6004) is a 4 credit degree level elective about algorithms that learn while they make one decision after another. It has two parts. Online learning, where you see full feedback after each step. Multi-armed bandits, where you only see the result of the choice you made. The course page lists no prerequisites.

The instructor is Arun Rajkumar, Assistant Professor in the Department of Data Science and AI at IIT Madras. He also teaches Machine Learning Foundations, Machine Learning Techniques and Reinforcement Learning.

What the handbook says, and does not say

The DS handbook names this course only in its list of minors, under Algorithmic Decision Making. It is not in the degree level course table (updated 18 March 2026). So the handbook gives no credits, level, tag or term plan for it. The 4 credits and "no prerequisites" above come from the course page.

The handbook says a 2xxx, 3xxx or 4xxx code shows the level. It says nothing about a 6xxx code. If you want this course to count as a level 4 or higher course at the BS level, ask support first.

What you learn

The course page splits the content into two modules instead of a week by week list.

• Module 1: online learning. It starts with simple ideas like the halving algorithm, then online classifiers such as Perceptron and Winnow. You meet regret, the main way to measure how far an online learner falls behind the best fixed choice. Then learning from expert advice with the Hedge algorithm, and online linear and convex optimisation.
• Module 2: multi-armed bandits. First the adversarial setting with the EXP3 algorithm, and contextual bandits with EXP4. Then stochastic bandits: epsilon greedy, explore then commit, UCB and Thompson Sampling. It ends with linear bandits and LinUCB.

A simple picture of a bandit: you have a few slot machines and a fixed number of pulls. Each pull teaches you something, but it also uses up a chance to play the best machine you know so far. Each algorithm in Module 2 handles that trade-off in a different way.

How it is assessed

The course page lists 12 weeks of coursework, weekly online assignments, 2 in-person invigilated quizzes and 1 in-person invigilated end term exam. It does not mention an OPPE or a project.

Where it counts

• Minor in Algorithmic Decision Making. It is one of three courses, with Reinforcement Learning (BSDA5007) and Algorithms for Data Science (BSDA5003). None of the three is in another minor. See the Algorithmic Decision Making minor.
• General elective. It adds 4 credits at the degree level. Check how it counts with support before you use it for the level rules in the BS degree level.

Who finds it hard and how to prepare

This is a theory course. Much of the work is about bounding regret, so you handle sums, expectations and inequalities. If you liked the derivations in MLF and MLT, you will likely enjoy it. If you only enjoy coding, the proofs may feel dry.

• Revise expectation, variance and inequalities from Statistics 2 before the term.
• Revise the Perceptron from Machine Learning Techniques. Module 1 comes back to it in an online setting.
• Code each bandit algorithm yourself in a few lines of Python against a simulated set of slot machines. Plot the regret over time. The proofs make more sense once you have seen the curves.
• For every algorithm, write one line: what feedback it gets, and what it does with that feedback.

The suggested reading is Bubeck's lecture notes on online optimisation, and a survey on regret analysis of bandit problems by Bubeck and Cesa-Bianchi.

What to take before and after

• Before: the three diploma ML courses. See MLF, MLT and MLP explained.
• With or after: Reinforcement Learning. Its weeks 2 and 3 cover the explore and exploit problem and bandits, then move to full RL. The handbook lists Deep Learning as its corequisite.
• To finish the minor: Algorithms for Data Science. Its course page lists Machine Learning Techniques as a prerequisite.

Common questions

Is Sequential Decision Making the same as Reinforcement Learning?

No. The SDM syllabus stays with online learning and bandits. It does not list Markov decision processes, which are at the centre of the Reinforcement Learning course. The clear overlap between the two syllabuses is bandits.

When is Sequential Decision Making offered?

The handbook's term plan does not include it. Look for it in the course list when registration opens, or ask the support team.

Do I need Deep Learning before this course?

No prerequisite is listed. The course is about decision algorithms and their guarantees, not neural networks.