Statistical Methods for Machine Learning
A.Y. 2024/2025
Learning objectives
The course describes, in a rigorous statistical framework, some fundamental ideas and techniques behind the design and analysis of machine learning algorithms.
Expected learning outcomes
Upon completion of the course, students will be able to: understand the notion of overfitting and its role in controlling the statistical risk, describe some of the most fundamental machine learning algorithms explaining how they avoid overfitting, run machine learning experiments using the correct statistical methodology. The project report and the oral discussion measure the achievement of these objectives. The grade for the project report and the grade for the written exam are combined to compute the final grade for the course.
Lesson period: Second semester
Assessment methods: Esame
Assessment result: voto verbalizzato in trentesimi
Single course
This course can be attended as a single course.
Course syllabus and organization
Single session
Responsible
Lesson period
Second semester
Course syllabus
Introduction
The Nearest Neighbour algorithm
Tree predictors
Statistical learning
Hyperparameter tuning and risk estimates
Risk analysis of tree predictors
Consistency and nonparametric algorithms
Linear predictors
Online gradient descent
Kernel functions
Support Vector Machines
Stability bounds and risk control for SVM
Boosting and ensemble methods
Neural networks and deep learning
Logistic regression and surrogate loss functions
The Nearest Neighbour algorithm
Tree predictors
Statistical learning
Hyperparameter tuning and risk estimates
Risk analysis of tree predictors
Consistency and nonparametric algorithms
Linear predictors
Online gradient descent
Kernel functions
Support Vector Machines
Stability bounds and risk control for SVM
Boosting and ensemble methods
Neural networks and deep learning
Logistic regression and surrogate loss functions
Prerequisites for admission
The course requires basic knowledge in calculus, linear algebra, probability and statistics.
Before attending this course, students are strongly advised to pass the following exams: Calculus, Discrete mathematics, Statistics and data analysis.
Before attending this course, students are strongly advised to pass the following exams: Calculus, Discrete mathematics, Statistics and data analysis.
Teaching methods
Lecture-style instruction.
Teaching Resources
The main reference material are the lecture notes available through the link ncesa-bianchismml.ariel.ctu.unimi.it/
Reference textbook: Shai Shalev-Shwartz e Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, 2014.
Reference textbook: Shai Shalev-Shwartz e Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, 2014.
Assessment methods and Criteria
The exam consists in writing a paper of about 10-15 pages containing either a report describing experimental results (experimental project) or an in-depth analysis of a theoretical topic (theory project). The final grade (with a 1-30 grading range) is computed by combining the project evaluation with the result of a written test on the syllabus covered in class.
INF/01 - INFORMATICS - University credits: 6
Lessons: 48 hours
Professor:
Cesa Bianchi Nicolo' Antonio
Shifts:
Turno
Professor:
Cesa Bianchi Nicolo' AntonioProfessor(s)