Research involves reading, thinking, programming, experimenting, writing, traveling, and presenting. Here are the fruits of that labour. ★ denotes recommended reading.

Doctoral Thesis

My doctoral thesis is entitled ★ MEI: Multimodal Emotional Intelligence (12 Mb). See demo video and TEDx talk.

Abstract: In this thesis, we design and implement a multimodal emotion system for robots. The overreaching goal is to advance the fundamentals of robot primary emotions by using clues from infant development. Our approach has three prime characteristics, which set it apart from current robot emotion systems. Firstly, it is multimodal. Humans express and recognize emotions through a variety of dynamic channels, such as voice, movement and music. Our paradigm uses speed, intensity, irregularity and extent (SIRE) to colour a robot’s voice, gesture and gait with emotion, using a simple 4-dimensional representation. Secondly, it models emotion statistically. Many emotion models are hand-defined based on a posteriori rules, yet humans are known to be statistical learning machines. Our MEI (multimodal emotional intelligence) module once trained, can recognize emotion in a context it has never encountered, and generate statistically probable emotion expressions. Finally, it is developed through a social process found in caregiver-infant interactions. Emotions are thought to be innate, but according to evidence in developmental psychology, much development happens between the ages of zero and one. In this thesis, we model this first year of life where emotional intelligence grows rapidly, possibly due to a universal phenomenon called motherese.

Here's a quick preview from the introduction.

Master's Thesis

My Master's thesis was entitled Design and Implementation of Emotions for Humanoid Robots based on the Modality-independent DESIRE Model (31 Mb). It discusses how we can generate and analyze emotions in the same way, whether it's in voice, music or movement. My model is based on four parameters: speed, intensity, regularity and extent (SIRE). You can use this approach to add emotional colour to humanoids such as HRP-2, NAO, and potentially even robots without any arms or legs!


Book Chapters

International Conferences

International Poster Presentations

Editorial Activities

Japanese Conferences