Affective Computing:

a Multidisciplinary Challenge

12 November 2015, Ecole Centrale de Lyon, France


14:10 - 14:40 : Björn Schuller, University of Passau/Germany & Imperial College London/UK
Sound Affects.

Abstract: Sound is vibration – mostly an audible mechanical wave of pressure and displacement such as through air. But it also “affects”: it sets us into and bears our moods – be it in form of music, speech, or any other type of sound. In this talk, I will present recent work at audEERING, Imperial College, and the University of Passau towards making computers perceive this sensation, or “be affected” by sound. This includes feature and data free acquisition of affect by deep, transfer, and cooperative learning with gamified crowd sourcing. It further touches upon efficiency and applicability. A range of results from research competitions and demo-videos round off this view on the state-of-the-art in Computer Affect Audition.

Bio: Björn received his diploma, doctoral degree, habilitation, and Adjunct Teaching Professor from TUM in Munich/Germany. He is Full Professor heading the Chair of Complex and Intelligent Systems at the University of Passau/Germany, a Reader (Associate Professor) in Machine Learning at Imperial College London/UK, and the co-founding CEO of audEERING UG. Previously, he headed the Machine Intelligence and Signal Processing Group at TUM, was invited as a (permanent) Visiting Professor at HIT/P.R. China and the Université de Genève/Switzerland, was with Joanneum Research in Graz/Austria, and the CNRS-LIMSI in Orsay/France. He is elected member of the IEEE SLTC, Senior Member of the IEEE, and President-Emeritus of the AAAC. He (co-)authored 500+ publications (9400+ citations, h-index = 45), and is Editor in Chief of the IEEE Transactions on Affective Computing. In 2015, he was awarded one of 40 extraordinary scientists under the age of 40 by the World Economic Forum. Among others, he is recipient of an ERC Starting Grant for the iHEARu project, beneficiary of ERC Advanced and Proof-of-Concept grants, was a Coordinator and PI in FP7, and now is PI in four H2020 projects.

14:40 - 15:10 : Thierry Pun, Computer Vision and Multimedia Laboratory (, Computer Science Department, and Swiss Center for Affective Sciences ( University of Geneva, Switzerland
Affective computing: overview of activities

Abstract: Affective computing is the understanding, the recognition and the inclusion of human emotions in the design of computational systems. Therefore, affective computing aims at developing devices and whole frameworks accounting for users’ affective reactions or states, so as to ease human-computer and human-human interaction. In this talk we will first briefly overview the main research projects in this area that are conducted at the Computer Vision and Multimedia Laboratory, University of Geneva: emotions assessment for affective computing; affective tagging based on physiological signals, on crowdsourcing, and on fusion with behavioral signals; flow and engagement for affective gaming; situated ecological affect estimation. We will present the EATMINT project investigating the impact of mutual emotion awareness in mediated interaction and remote cooperation, as well as the Aesthetics in Movies research whose main objective is to define and detect aesthetic emotions in the context of movie watching on the basis of spectators' reactions synchronization.

Bio: Thierry Pun is full professor at the University of Geneva, Switzerland, head of the Computer Vision and Multimedia Laboratory (CVML) and of the Multimodal Interaction Group (MMI). His current research interests are in affective computing and emotions analysis, social signal processing and multimodal interaction. He is involved in projects concerning affective computing and gaming, physiological signals analysis for emotion assessment and social interactions analysis, multimodal interfaces for blind and elderly users, brain-computer interaction. Thierry Pun has authored or co-authored over 350 full papers as well as eight patents ( He was one of the general chairs and organizers of ACII 2013 (Affective Computing and Intelligent Interaction) in Geneva, Switzerland, September 2-5, 2013 ( Prof. Pun has been instrumental in the creation of two spin-offs of the University of Geneva, and has participated in and/or lead a number of research projects, Swiss and European, financed by public and private entities.

15:30 - 16:00 : Mateus Joffily, Groupe d’Analyse et de Théorie Economique Lyon-St Etienne, CNRS
Affect, emotion and active inference

Abstract: I will present a new perspective on affect and emotion that rests on the free energy principle (Joffily and Coricelli, 2013). The free energy principle has been recently proposed in neurosciences as a unifying (Bayesian) account for perception, learning and action (Friston, 2010). In active inference, agents do not just infer hidden states (perceptual inference and learning) but actively sample outcomes that minimize the free energy (an upper bound on surprise) of sensory inputs. In this framework, valence, i.e., the positive or negative charge that accompanies affect, emotions and feelings, is formally defined as the negative rate of change of free energy over time. When free energy is increasing (decreasing) over time, negative (positive) affective states are visited. Moreover, when higher-order dynamics of free energy, e.g., the second time-derivative, are also taken into account, a family of discrete emotions can be identified, such as happiness, unhappiness, hope, fear, disappointment and relief. According to this perspective, an important function of valence turns out to regulate the uncertainty of posterior beliefs about hidden states and behavior. This dynamic interaction between valence and uncertainty highlights the crucial adaptive role of emotion.

Bio: Mateus Joffily is a research engineer in CNRS working at the Groupe d’Analyse et de Théorie Economique Lyon-St Etienne (GATE). His research interests are in emotion, decision-making and Bayesian hierarchical modeling from the perspective of neurosciences.