APL405
APL405: Machine Learning for Mechanics (Winter semester 2023)
Course Info
Credit: 3 units (2-0-2)
Pre-requisites: APL101/MTL106/MTL108, COL106
Instructors: Rajdip Nayek (rajdipn@am.iitd.ac.in)
Class timings: Tue, Thu & Fri (11:00 to 11:50 AM) at LHC517
Practical Session: Fri (3:15 to 5:15 PM) at two labs LH503 and LH502
Attendance and Marks: Check here
Office hours (TA): By email appointment
Office hours (Instructor): By email appointment (Room B24, Block 4)
Intended audience: BTech students in Applied Mechanics, Materials, Mechanical and Civil Engineering disciplines.
NOTE-For all course related emails, please put APL405 in the subject line
Table of Contents
- Course Content
- Course Structure
- Lecture Schedule
- Practical Schedule
- Homework Schedule
- Course References
- Grading
- Project
Course Content
This is an introductory course to statistical machine learning for students with some background in calculus, linear algebra and statistics. The course is focusing on supervised learning, i.e, classification and regression. The course will cover a range of methods used in machine learning and data science, including:
- Linear regression
- Classification via logistic regression and k-nearest neighbour
- Regression and classification trees
- Bagging and boosting
- Neural networks and deep learning
- Principal component analysis
- k-means clustering
These methods will be studied from various applications throughout the course. The course also covers important practical considerations such as cross-validation, model selection and the bias-variance trade-off. The course includes theory (e.g., derivations and proofs) as well as practice (notably the lab and the course project). The practical part will be implemented using Python.
Course Structure
- Introduction
- Supervised Learning
- k-NN and Decision Trees
- Linear Regression and Logistic regression
- Polynomial regression
- Bias-Variance decomposition
- Loss function vs likelihood based models
- Regularization
- Parameter optimization
- Three sets: Train, test, validation
- Neural Networks (NN)
- Support vector machine (SVM)
- Ensemble algorithms
- Unsupervised Learning (Representation Learning and Dimensionality Reduction)
Lecture Schedule
| Module# | Main Topic | Sub Topics | Lecture Notes |
|---|---|---|---|
| Module 00 | Introduction | Lecture 1 | |
| Module 01 | A Preliminary Approach to Supervised Learning |
Background k-Nearest Neighbours Decision Trees |
Lecture 2 Lecture 3 Lecture 4 |
| Module 02 | Basic Parametric Models | Linear regression Logistic Regression Regularization |
Lecture 5 Lecture 6 Lecture 7 |
| Module 03 | Evaluating Performance | Cross-validation Training error-generalization gap Bias-variance decomposition |
Lecture 8 Lecture 9 Lecture 10 |
| Module 04 | Learning Parametric Models | Loss functions Parameter Optimization |
Lecture 11a Lecture 11b Lecture 12 |
| Module 05 | Neural Networks | Feedforward neural network Backpropagation Convolutional Neural Network |
Lecture 13 Lecture 14 Lecture 15 |
| Module 06 | Kernel Methods | Kernel Ridge Regression Theory of kernels Support Vector Classification |
Lecture 16 Lecture 17 Lecture 18 |
| Module 07 | Ensemble Methods | Bagging Random Forests Boosting |
Lecture 19 Lecture 20 Lecture 21 |
| Module 09 | Generative Models & Unsupervised Learning |
Gaussian mixture models Gaussian mixture models (with EM) k-means clustering PCA |
Lecture 22 Lecture 23 Lecture 24 Lecture 25 |
Practical Schedule
| Week# | Topics | Practical Questions | Notes |
|---|---|---|---|
| Week 1 | Probability refresher | Practical 1 | Notes |
| Week 2 | k-Nearest Neighbours | Practical 2 | |
| Week 3 | Decision Trees | Practical 3 | Dataset |
| Week 4 | Linear Regression | Practical 4 | |
| Week 5 | Logistic Regression | Practical 5 | Dataset |
| Week 6 | Cross-validation & Bias-variance trade-off | Practical 6 | Dataset |
| Week 7 | Introduction to PyTorch for Neural Nets | Practical 7 | |
| Week 8 | Support Vector Machine | Practical 8 | |
| Week 9 | Boosting | Practical 9 | Dataset |
Homework Schedule
A total of three homeworks would be given
| HW# | Questions | Dataset | Solutions |
|---|---|---|---|
| HW1 | Homework 1 | Datafiles | Solution |
| HW2 | Homework 2 | Solution | |
| HW3 | Homework 3 | Datafiles Template |
Solution |
Course References
- Christopher Bishop, “Pattern Recognition and Machine Learning”, Springer, 2006.
- Andriy Burkov, “The Hundred Page Machine Learning Book”, 2019 [free pdf].
- Andreas Lindholm et. al., “Machine Learning: A First Course for Engineers and Scientists”, Cambridge University Press, 2022 [free pdf]
- Brunton, Steven L., and J. Nathan Kutz. “Data-driven science and engineering: Machine learning, dynamical systems, and control”. Cambridge University Press, 2022.
Grading
| Component | Scores |
|---|---|
| Practical Exam | 10 |
| Class Attendance | 5 |
| Homework | 10 |
| Project | 20 |
| Minor #1 | 15 |
| Minor #2 | 15 |
| Major | 25 |
| Total | 100 |
- Students are highly encouraged to attend all classes, but there are no marks on lecture attendance. You still have to mark your attendance on Timble. If the combined lecture-plus-practical attendance is below 75% of the total, one grade will be lowered.
- A penalty of 50% of the mark of the respective homework would be deducted if submitted after the deadline. Any submissions two days after the deadline will not be accepted.
Exams
| Component | Solution |
|---|---|
| Minor #1 | Solution |
| Minor #2 | Solution |
| Major | Solution |
Project
A maximum of two students per project is allowed
Project proposal must be submitted by Feb 12th (11:59 pm) (10% of total project marks)
- Mid-semester project report must be submitted by Apr 9th (11:59 pm) (25% of total project marks)
- Final project report plus a link to a recorded video presentation must be submitted by Apr 27th (11:59 pm) ((30% +35%) of total project marks)
- Any delay in submission of the above will be penalized by 1 mark per hour after the deadline has passed!!
Please follow this link for more details on the project.