Natural Interaction
A.Y. 2024/2025
Learning objectives
The course describes and analyses theory and techniques for the design of artificial agents that are able to:
- perceive the user's behavioural signals, via contactless sensors (Kinect,Leap motion, video camere eye-trackers, etc) and extract their most relevant cues
- infer and predict, from such cues in a given context, user's behavioural state
- provide the appropriate feedback to the user
To such end, the course concerns: a rigorous introduction to the neurobiological and psychological models of natural interaction through gestures, postures and gaze; statistical machine learning and inference for modelling the dynamics of interaction
- perceive the user's behavioural signals, via contactless sensors (Kinect,Leap motion, video camere eye-trackers, etc) and extract their most relevant cues
- infer and predict, from such cues in a given context, user's behavioural state
- provide the appropriate feedback to the user
To such end, the course concerns: a rigorous introduction to the neurobiological and psychological models of natural interaction through gestures, postures and gaze; statistical machine learning and inference for modelling the dynamics of interaction
Expected learning outcomes
Upon completion of the course students will be able to:
1. Define the methodology and the most appropriate techniques for modelling interaction agents dealing with uncertainty
2. Measure and analyse behavioural signals, and extract their most informative cues
3. Design and implement simple interaction agents to be exploited in different applications such as smart living ambients, behavioural biometrics, entertainment, video surveillance.
These objectives are measured via a combination of three components: the project realisation, the project technical report and the oral discussion. The final grade is formed by assessing the software developed, the project report, and then using the oral discussion for fine tuning.
1. Define the methodology and the most appropriate techniques for modelling interaction agents dealing with uncertainty
2. Measure and analyse behavioural signals, and extract their most informative cues
3. Design and implement simple interaction agents to be exploited in different applications such as smart living ambients, behavioural biometrics, entertainment, video surveillance.
These objectives are measured via a combination of three components: the project realisation, the project technical report and the oral discussion. The final grade is formed by assessing the software developed, the project report, and then using the oral discussion for fine tuning.
Lesson period: First semester
Assessment methods: Esame
Assessment result: voto verbalizzato in trentesimi
Single course
This course can be attended as a single course.
Course syllabus and organization
Single session
Responsible
Lesson period
First semester
Course syllabus
The course aims at providing the methodological background to address the modelling problem for detecting and understanding human behavioural signals as captured by a variety of sensors (cameras, microphones, Kinect, Leap motion, etc.). Such background involves both the psychological/neurobiological bases of non verbal behaviours and probabilistic machine-learning methods to set up computational models of behavioural/social signal understanding.
The course articulates in two parts:
Part 1 Neurobiological and psychological models of nonverbal behaviour
- Gaze interaction
- Gesture interaction
- Posture interaction
Part 2 Probabilistic models and machine-learning techniques for the analysis of nonverbal behaviour
- Probabilistic models: inference and learning
- Supervised modelling: classic techniques and deep learning
- Unsupervised modelling: classic techniques and deep learning
Practical examples and applications
The course articulates in two parts:
Part 1 Neurobiological and psychological models of nonverbal behaviour
- Gaze interaction
- Gesture interaction
- Posture interaction
Part 2 Probabilistic models and machine-learning techniques for the analysis of nonverbal behaviour
- Probabilistic models: inference and learning
- Supervised modelling: classic techniques and deep learning
- Unsupervised modelling: classic techniques and deep learning
Practical examples and applications
Prerequisites for admission
Students are strongly encouraged to preliminarily attend the courses of Statistical methods for machine learning, Artificial Intelligence and Computer Vision. Python programming capabilities are welcomed for project work.
Teaching methods
Lectures on general aspects of natural interaction and, while developing the projects, hands-on activities within the PHUSE Lab
Teaching Resources
Lectures notes, live notes and slides, books, chapters, papers, videos and code are available on the course web site hosted by Ariel (https://gboccignoneni.ariel.ctu.unimi.it/v5/Home/)
Assessment methods and Criteria
The examination concerns the development of a project and an oral discussion. The project is assigned based on ongoing natural interaction research within the Perceptual computing and HUman SEnsing Laboratory (PHUSE Lab).
INF/01 - INFORMATICS - University credits: 6
Lessons: 48 hours
Professor:
Boccignone Giuseppe
Shifts:
Turno
Professor:
Boccignone GiuseppeProfessor(s)