My work focuses on the nonverbal expression of emotions. I am particularly interested in how the emotion process drives the temporal unfolding of expression in different modalities, I adopt an appraisal perspective to study emotion expressions, and I consider expressive cues as potential efferent effects of emotion appraisals.
I am the head Applied Affective Sciences research unit and I am responsible for several basic and applied research projects done in collaboration with external partners.
I completed a Ph.D. course in Psychology of Communication and Linguistic Processes at the Catholic University of Milan in 2007. My work focused on emotion expression and ANS correlates of emotion episodes. My thesis was entitled "Multicomponential analysis of emotional experience. Study of expressive and physiological components and their implications for affective computing". The goal was to study how facial and vocal expressions are related to modifications in the ANS activity in consequence of the experience of an emotion.
The same year, I joined the Swiss Center for Affective Sciences as a Post-doctoral researcher with the goal of studying emotion expression from a multimodal point of view.
In 2012 I was nominated Senior Scientist and I became the Head of the Applied Affective Sciences research unit
After the development of the Geneva Emotional Competence Test (Schlegel & Mortillaro, 2019), I am now studying the role of Emotional Intelligence in different domains, including sports and health care. This line of research includes studies about interventions to improve emotional competencies, longitudinal studies in the context of hospitality services, and the development of new assessment tools.
Multimodal expression of emotion:
I am interested in how different expressive modalities are jointly used during an emotional episode and how perception is influenced by unimodal and multimodal information. A particular feature of multimodal emotion expression is the way in which the different expressive modalities recruited during an emotion episode are timely used. With this respect, I am interested in how the degree of temporal synchronization influences the perception of emotional authenticity.
Nonverbal correlates of emotional appraisals
Planned studies include the investigation of the links between specific expressive features (vocal, facial, and gestures) and emotion appraisals.
We are developing instruments that can be used to assess emotional responses in several application settings and collaborating with researchers in the affective computing domain.
Current and Past Projects
"Measuring emotional intelligence in the workplace: Development of a new test"
The aim of this project is to develop and validate a new test to measure emotional intelligence in the context of job-related tasks. Emotional intelligence tests that are currently available have been criticized for their methodological approaches and for being generic and based on self-report measures. Conversely, this test will be a performance-based test that will include scenarios taken from real work situations.
"Automatic detection of subtle positive emotions from facial expressions"
This is a collaborative research project between GfK, Fraunhofer IIS and the University of Geneva (UNIGE) aimed at developing a semi-automatic tool for detecting subtle emotions in response to audio-visual stimuli.
"I can cope with that!: The effect of stress on emotion components: appraisal, expression, and physiology."
In this project, we investigated the appraisals that are typically involved in the experience of a series of emotions. Furthermore, the influence of stress on appraisals and other emotional components was investigated. The hypothesis is that stress can bias individuals’ evaluations by reducing his or her estimation of the sense of control on the situation (i.e., the coping potential).
"Emotional factors in ethical choices".
This project addressed the relationship between different appraisals in the experience of different positive affective states and the role of emotion appraisals in ethical decisions. In particular, we investigated how moral emotions are experienced in the context of socially responsible behavior. More generally, the project investigated how emotions can influence decisions about ethical issues.
- Scherer, K.R., Dieckmann, A., Unfried, M., Ellgring, H.,& Mortillaro, M. (2021). Investigating Appraisal-Driven Facial Expression and Inference in Emotion Communication. Emotion, 21(1), 73-95.
- Simonet, D.V., Miller, K.E., Askew, K.L. Summer, K.E., Mortillaro, M., Schlegel, K. (2021). How Multidimensional is Emotional Intelligence? Bifactor Modeling of Global and Broad Emotional abilities of the Geneva Emotional Competence Test. Journal of Intelligence, 9, 14
- Shaham, G., Mortillaro, M., & Aviezer, H. (2020). Automatic facial reactions to facial, body and vocal expressions: A stimulus-response compatibility study. Psychophysiology, 57, 12.
- Tisserand, Y., Aylett, R., Mortillaro, M., & Rudrauf, D. (2020). Real-time simulation of virtual humans' emotional facial expressions, harnessing autonomic physiological and musculoskeletal control. Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents, IVA 2020.
- Schlegel*, K., & Mortillaro*, M. (2019). The Geneva Emotional Competence Test (GECo): An ability measure of workplace emotional intelligence. Journal of Applied Psychology, 104(4), 559-580.
- Scherer, K. R., Ellgring, H., Dieckmann, A., Unfried, M. &, Mortillaro, M. (2019). Dynamic Facial Expression of Emotion and Observer Inference. Frontiers in Psychology, 10, 508.
- Schuller, B., Weninger,, F., Zhang, Y., Ringeval, F., Batliner, A., Steidl, S., Eyben, F., Marchi, E., Vinciarelli, A., Scherer, K., Chetouani, M., & Mortillaro, M. (2019). Affective and Behavioural Computing: Lessons Learnt from the First Computational Paralinguistic Challenge. Computer Speech and Language, 52, 156-180.
- Seuss, D., Dieckmann, A., Hassan, T., Garbas, J., Ellgring, J.H., Mortillaro, M., & Scherer, K. (2019). Emotion expression from different angles: A video database for facial expressions of actors shot by a camera array. Proceedings 8th International Conference on Affective Computing and Intelligent Interaction (ACII), 35–41.
- Soleymani, M., & Mortillaro, M. (2018). Behavioral and Physiological Responses to Visual Interest and Appraisals: Multimodal Analysis and Automatic Recognition. Frontiers in ICT – Human-Computer Interaction, 5, 17.
- Mortillaro, M., & Dukes, D. (2018). Jumping for joy: the importance of the body and of dynamics in the expression and recognition of positive emotions. Frontiers in Psychology, 9, 763.
- Scherer, K.R, Mortillaro, M., Rotondi, I., Sergi, I, & Trznadel, S. (2018). Appraisal-driven facial actions as building blocks for emotion inference. Journal of Personality and Social Psychology, 114(3), 358-379.
- Dukes, D., Clément, F., Audrin, C., & Mortillaro, M. (2017). Looking beyond the static face in emotion recognition: The informative case of interest. Visual Cognition. 25(4-6), 575-588.
- Scherer, K., Mortillaro, M., & Mehu, M. (2017). Facial expression is driven by appraisal and generates appraisal inference. In J. M. Fernandez-Dols and J. A. Russell (Eds.), The science of Facial Expression (353-373). New York: Oxford University Press
- Mortillaro, M., & Mehu, M. (2015). Emotions: Methods of Assessment. In J. D. Wright (Ed.), International Encyclopedia of the Social & Behavioral Sciences, 2nd edition, Vol 7 (pp. 519-525). Oxford: Elsevier.
- Mortillaro, M., Meuleman, B., & Scherer, K. R. (2015). Automated recognition of emotion appraisals. In J. Vallverdú (ed.), Handbook of Research on Synthesizing Human Emotion in Intelligent Systems and Robotics (pp.338-351). Hershey, PA: Information Science Reference, IGI Global.
- Glowinski, D., Mortillaro, M., Scherer, K., Dael, N., Volpe, G., & Camurri, A. (2015). Towards a minimal representation of affective gestures (Extended abstract). International Conference on Affective Computing and Intelligent Interaction (ACII), Xi'an, 2015, pp. 498-504.
- Visch, V., Goudbeek, M.B., & Mortillaro, M. (2014). Robust anger: Recognition of deteriorated dynamic bodily emotion expressions. Cognition & Emotion, 28(5), 936-946.
- Mortillaro, M., & Scherer, K. R. (2014). Stressed out: How stress develops and how to cope with it. GfK Marketing Intelligence Review, 6(1), 17-21.
- Scherer, K. R., Mortillaro, M., & Mehu, M. (2013). Understanding the Mechanisms Underlying the Production of Facial Expression of Emotion: A Componential Perspective. Emotion Review, 5(1), 47-53.
- Weninger, F., Eyben F., Schuller B.. W., Mortillaro M., & Scherer K. R. (2013). On the Acoustics of Emotion in Audio: What Speech, Music and Sound have in Common. Frontiers in Psychology. 4(292).
- Mortillaro, M., Mehu, M., & Scherer, K.R. (2013). The evolutionary origin of multimodal synchronization in emotional expression. In E. Altenmüller, S. Schmidt, and E. Zimmerman (Eds.), Evolution of Emotional Communication: from Sounds in Nonhuman Mammals to Speech and Music in Man (pp.3-25). New York: Oxford University Press.
- Mortillaro, M., Ricci-Bitti, P.E., Bellelli, G., Galati, D., & Scherer, K.R. (2013). Pride is not created equal: variations between Northern and Southern Italy. In K.R. Scherer and J.R. Fontaine (Eds.), Components of Emotional Meaning: A sourcebook (pp. 366-376). New York: Oxford University Press.
- Mehu, M., Mortillaro, M., Bänziger, T, & Scherer, K. R. (2012). Reliable facial muscles activation enhances recognisability and credibility of emotional expression. Emotion,12(4), 701-715.
- Mortillaro, M., Meuleman, B., & Scherer, K. R. (2012). Advocating a componential appraisal model to guide emotion recognition. International Journal of Synthetic Emotions, 3(1), 18-32.
- Dael, N., Mortillaro, M., & Scherer, K.R. (2012). The Body Action and Posture coding system (BAP): Development and reliability. Journal of Nonverbal Behavior. 36, 97-121.
- Dael, N., Mortillaro, M., & Scherer, K.R. (2012). Emotion expression in body action and posture. Emotion. 12(5), 1085-1101.
- Bänziger, T., Mortillaro, M., & Scherer, K.R. (2012). Introducing the Geneva Multimodal Expression corpus for experimental research on emotion perception. Emotion, 12(5), 1161-1179.
- Mortillaro, M., Mehu, M., & Scherer, K. R. (2011). Subtly different positive emotions can be distinguished by their facial expressions. Social Psychological and Personality Science, 2, 262-271.
- Scherer, K.R., Clark- Polner, E., & Mortillaro, M. (2011). In the eye of the beholder? Universality and cultural specificity in the expression and perception of emotion. International Journal of Psychology, 46, 401-435.
- Glowinski, D., Dael, N., Camurri, A., Volpe, G., Mortillaro, M., & Scherer, K. R. (2011). Towards a minimal representation of affective gestures. IEEE Transactions on Affective Computing, 2, 106-118.
- Mantovani, F., Confalonieri, L., Mortillaro, M., Realdon, O., Zurloni, V., & Anolli, L. (2010). The potential of affective computing in e-learning: the journey from theory to practice in the “Myself” project. In A. Tzanavari and N. Tapatsoulis (Eds.), Affective, Interactive and Cognitive Methods for E- Learning Design: Creating an Optimal Education Experience (pp. 260-274). Hershey, PA: IGI Global.
- Mohammadi, G.,. Vinciarelli, & Mortillaro, M. (2010). The voice of personality: Mapping vocal nonverbal behavior into trait attributions. Proceedings of the International Workshop on Social Signal Processing, pp. 17-20, Florence, 2010.
- Mortillaro, M., & Scherer, K. R. (2009). Multimodal expression of emotion. In D. Sander & K. R.Scherer (Eds.), The Oxford Companion to Emotion and the Affective Sciences (pp.268-269). New York: Oxford University Press.
- Mortillaro, M., & Scherer, K. R. (2009). Bodily expression of emotion. In D. Sander & K. R. Scherer (Eds.), The Oxford Companion to Emotion and the Affective Sciences (pp.78-79). New York: Oxford University Press.
- Castellano, G., Mortillaro, M., Camurri, A., Volpe, G., & Scherer, K. R. (2008). Automated analysis of body movement in emotionally expressive piano performances. Music Perception, 26, 103-120.
- Zurloni, V., Mortillaro, M., Vescovo, A., & Anolli, L., Mantovani, F. (2008). Addressing emotions within e-learning systems. In T. T. Kidd & H. Song (Eds.), Handbook of Research on Instructional Systems and Technology (pp. 803-816). Hershey, PA: IGI Global.
- Anolli, L., Mantovani, F., Balestra M., Kommers, P., Robotti, O., Salamin, A. D., Agliati, A., Realdon, O., Zurloni, V., Confalonieri, L., Mortillaro, M., Vescovo, A., Venturini, P., Wetterling, & Tarocchi, A. (2008). Experiential learning and affective computing for training soft-skills: Myself project. In T. T. Kidd & H. Song (Eds.), Handbook of Research on Instructional Systems and Technology (pp. 360- 372). Hershey, PA: Information Science Reference.
- Anolli, L., Mortillaro, M., Zurloni, V., Confalonieri, L., Mantovani, F., (2008)."Affective computing" e apprendimento: analisi di dispositivi per il riconoscimento automatico delle emozioni all’interno di piattaforme di e-learning. In E. Magno Caldognetto and F. Cavicchio (Eds.), Aspetti Emotivi e Relazionali nell’E-Learning (pp.171-199). Firenze, IT: Firenze University Press .
- Mortillaro, M. (2007). Multicomponential analysis of emotional experience. Study of expressive and physiological components and their implications for affective computing. Unpublished doctoral dissertation. Catholic University of Milan, Milan, Italy.
- Mantovani, F., Agliati, A., Mortillaro, M., Vescovo, A., & Zurloni V. (2006). Communication – Presence roundtrip: Travelling along theoretical, methodological and applicative connections. In G. Riva, M. T. Anguera, B. K. Wiederhold, & F. Mantovani, (Eds.), From Communication to Presence: Cognition, Emotion and Culture towards the Ultimate Communicative Experiences (pp.3-22). Amsterdam, The Netherlands; IOS Press.
- Realdon, O., Zurloni V., Confalonieri, L., Mortillaro, M., & Mantovani, F., (2006). Learning Communication Skills through computer-based Interactive Simulations. In G. Riva, M. T. Anguera, B. K. Wiederhold, & F. Mantovani, (Eds.), From Communication to Presence: Cognition, Emotion and Culture towards the Ultimate Communicative Experiences (pp.281-303). Amsterdam, The Netherlands; IOS Press.
- Anolli, L., Mantovani, F., Mortillaro, M., Vescovo, A., Agliati, A., Confalonieri, L., Realdon, O., Zurloni, V., & Sacchi, A. (2005). A Multimodal Database as a Background for Emotional Synthesis, Recognition and Training in E-Learning Systems. In J. Tao, T. Tan, & R. W. Picard (Eds.), Affective Computing and Intelligent Interaction: First International Conference, ACII 2005, Beijing, China, October 22-24, 2005. Proceedings. Lecture Notes in Computer Science, 3784, (pp. 566 – 573).