I am a Ph.D. student in Machine Learning for Social Robotics at KTH Royal Institute of Technology in Stockholm. My main supervisor is Hedvig Kjellström and co-supervisors are Gustav Eje Henter and Iolanda Leite.
My research is on generative models of non-verbal behavior, such as hand gestures and facial expressions. You can watch me explaining it here. My latest project can be found on this page.
I am working in the HealthTech project EACare, where we aim to develop a robot system to detect early signs of Dementia from the communicative behavior. My part is non-verbal behavior generation.
News
- Oct 2020: Let’s face it received the IVA 2020 Best Paper Award!
- Oct 2020: I will give a talk about my work on gesture generation for the Talking Robotics seminar series on October 30th 4pm UTC(GMT). More details under this link.
- Oct 2020: Gesticulator was nominated for ICMI 2020 Best Paper Award!
- Sept 2020: I have an open position for a master thesis on benchmarking gesture generation models in an interaction. See the proposal for more details.
- Sept 2020: One more paper accepted to IVA 2020: Let’s face it: Probabilistic multi-modal interlocutor-aware generation of facial gestures in dyadic settings.
- Aug 2020: Our paper Gesticulator: A framework for semantically-aware speech-driven gesture generation got accepted to ICMI 2020!
- July 2020: Two papers accepted for IVA 2020. More details in the publications tap.
- May 2020: We received honourable mention award at Eurographics 2020 for our paper Style-Controllable Speech-Driven Gesture Synthesis Using Normalising Flows.
- April 2020: The CFP for our IVA’20 Workshop on Generation and Evaluation of Non-verbal Behaviour for Embodied Agents is out. As part of this workshop we organize Gesture Generation Challenge!
- Feb 2020: Our paper Style-Controllable Speech-Driven Gesture Synthesis Using Normalising Flows was accepted to Eurographics 2020.
- Jan 2020: Sifan Jiang started his master thesis with me. He will be extending my gesture generation model to humanoid robot NAO.
- November 2019: Talk on general direction of my research.
- October 2019: Master students wanted for a master thesis project on gesture generation for a humanoid robot.
- September 2019 Talk on How to make your agent gesture in a natural way at Max Planck Institute for Intelligent Systems: Perceiving Systems Department in Tuebingen.
- August 2019: Our gesture generation model was applied to a new dataset. Now it can gesticulate in both Japanese and English. Check out a short demo video and our code with a pre-trained model.
- June 2019: Our two papers were accepted for ICDL-EPIROB 2019 Workshop on Naturalistic Non-Verbal and Affective Human-Robot Interactions.
- May 2019: Talk on my research at the Pint Of Science in Stockholm.
- April 2019: Our paper Analyzing input and output representations for speech-driven gesture generation was accepter to IVA 2019 for oral presentation. (24% acceptance rate)
- Jan 2019: Our paper On the importance of representations for speech-driven gesture generation was accepted at AAMAS 2019 for poster presentation.
- Oct 2018: My project proposal was published ICMI Doctoral Consortium 2018.
- April 2018: Joined Social Robotics Sweden (SoRoS) community.
- June 2017: We had a poster at The First Swedish Symposium on Deep Learning (SSDL).
Academic Service
- Co-organizing
- IVA’20 Workshop on Generation and Evaluation of Non-verbal Behaviour for Embodied Agents
- GESPIN 2020 Conference
- SoRoS 2020 Workshop
- NIPS 2018 Workshop on AI for Social Good
- Reviewer for
- ECAI 2020, IJCAI 2020, SIGRAPH 2020, IVA 2020, ICMI 2020 LBR
- ACII 2019, ICSR 2019
- NIPS 2018 Workshop on AI for Social Good