PG Level Advanced Certification Programme on Artificial Intelligence and
Autonomous Systems

Online Module

• Introduction to Machine Learning and Workflow and Applications
• Linear and Logistic Regression: Optimization view
• Data modeling with probabilistic methods
• Perceptron and Hyperplane classifiers
• Support vector Machines and Kernel Methods

Lab Module

Focus: Introductory concepts and algorithms in machine learning.
Topics: Python, Naive Bayes, Linear Regression, Logistic Regression, Optimization, SVM, Perceptron, Evaluation metrics.
Environments/Tools: Jupyter Notebook, Google Colab, sklearn

Online Module

• Introduction to Neural Networks and Deep Learning
• Multilayer perceptrons and training Neural networks
• Back propagation algorithm
• Convolutional Neural Networks
• Recurrent Neural Networks and LSTMs

Lab Module

Focus: Deeper understanding of neural networks and their applications.
Topics: Multi-layer Perceptron, Convolutional Neural Networks, Transfer Learning (AlexNet).
Environments: Jupyter Notebook, Google Colab, TensorFlow, PyTorch, Keras (with GPU acceleration for larger models)

Online Module

Pre-requisites: Probability and Linear Algebra

• Introduction and Real world examples
• Sequential decision making under Uncertainity: Multi-armed Bandits
• Markov Decision Process, Value and Q-functions, Finite and infinite horizon problems
• Value and Policy iteration, Dynamic ProgramingMarkov Decision Process, Value and Q-functions, Finite and infinite horizon problems (Instructor: AD)
• Policy evaluation with Monte-Carlo and Stochastic approximation (Instructor: AD)

Lab Module

Focus: Principles of reinforcement learning and decision-making.
Topics: Multi-armed bandits, Markov Decision Processes, Policy Iteration, Value Iteration, Monte Carlo, OpenAI Gym environments.
Environments: Jupyter Notebook, OpenAI Gym (CartPole, FrozenLake, Blackjack, Grid World)

Online Module

Pre-requisites: Modules-1,2,3, and 4

• Data-Driven Optimization
• Model-free algorithms for policy evaluation – incremental update temporal difference learning, n-step temporal difference learning
• Model-free least squares temporal difference learning, least squares policy evaluation, incremental least squares temporal difference learning
• Model-free value-based algorithms for control – SARSA, Q-learning, Expected SARSA, Double Q-learning
• Algorithms for on-policy prediction with value function approximation – linear approximation architectures (Fourier/Polynomial bases, tile coding, radial basis functions), non-linear approximation architectures (artificial neural networks)
• Off-policy value-based algorithms with approximation – semi-gradient methods, gradient descent in Bellman error, gradient temporal difference learning, temporal difference with correction
• Policy gradient algorithms – policy parameterization, REINFORCE: Monte-Carlo policy gradient, REINFORCE with baseline, Actor-Critic algorithms
• Recent algorithms - Deep Q-network, Trust Region Policy Optimization, Proximal Policy Optimization, A2C, A3C

Lab Module

Focus: Advanced reinforcement learning algorithms.
Topics: Monte Carlo Simulation, N-step TD, SARSA, Expected SARSA, SARSA Max, Q-learning, DQN, DDQN.
Environments: OpenAI Gym (CartPole, Super Mario, Pac-Man, Taxi, Mountain Car, Bipedal Walker, Hopper Robot, Four-legged Ant, Lunar Lander, Robot-arm), Acme, Gymnasium

Online Module – 9 Hours

Pre-requisites: Modules-1,2,3, and 4

• Deep RL demonstrations on simulated environments
• Introduction to Robotics

Focus: Reinforcement learning algorithms for discrete action spaces.
Topics: PPO, DDPG.
Environments: OpenAI Gym (CartPole, Super Mario, Pac-Man, Taxi, Mountain Car), Stable baseline - 3