I am a Ph.D. student in Machine Learning for Social Robotics at KTH Royal Institute of Technology in Stockholm. My main supervisor is Hedvig Kjellström and co-supervisors are Gustav Eje Henter and Iolanda Leite.

My research is on machine learning models for non-verbal behavior generation, such as hand gestures and facial expressions. I recently gave a talk about it, which you find in this video. My latest project, Gesticulator, can be found on this project page.

I am working in the HealthTech project EACare, where we aim to develop a robot system to detect early signs of Dementia from the communicative behavior. My part is non-verbal behavior generation.

News

• Mar 2021: Our GENEA (Generation and Evaluation of Non-verbal Behavior for Embodied Agents) Workshop 2021 has been accepted as an official workshop at ICMI 2021. Call for papers is out!
• Feb 2021: Our demonstrator A Framework for Integrating Gesture Generation Models into Interactive Conversational Agents was accepted to the 20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2021).
• Feb 2021: Synced wrote an article about Gesticulator, which you can access under this link
• Jan 2021: Our paper Moving fast and slow: Analysis of representations and post-processing in speech-driven automatic gesture generation got accepted to the International Journal of Human-Computer Interaction.
• Dec 2020: Our paper A large, crowdsourced evaluation of gesture generation systems on common data: The GENEA Challenge 2020 was accepted to IUI 2021 conference.
• Oct 2020: Gesticulator have got the ICMI 2020 Best Paper Award!
• Oct 2020: Let’s face it received the IVA 2020 Best Paper Award!
• Oct 2020: I will give a talk about my work on gesture generation for the Talking Robotics seminar series on October 30th 4pm UTC(GMT). More details under this link.
• Sept 2020: I have an open position for a master thesis on benchmarking gesture generation models in an interaction. See the proposal for more details.
• Sept 2020: One more paper accepted to IVA 2020: Let’s face it: Probabilistic multi-modal interlocutor-aware generation of facial gestures in dyadic settings.
• Aug 2020: Our paper Gesticulator: A framework for semantically-aware speech-driven gesture generation got accepted to ICMI 2020!
• July 2020: Two papers accepted for IVA 2020. More details in the publications tap.
• May 2020: We received honourable mention award at Eurographics 2020 for our paper Style-Controllable Speech-Driven Gesture Synthesis Using Normalising Flows.
• April 2020: The CFP for our IVA’20 Workshop on Generation and Evaluation of Non-verbal Behaviour for Embodied Agents is out. As part of this workshop we organize Gesture Generation Challenge!
• Feb 2020: Our paper Style-Controllable Speech-Driven Gesture Synthesis Using Normalising Flows was accepted to Eurographics 2020.
• Jan 2020: Sifan Jiang started his master thesis with me. He will be extending my gesture generation model to humanoid robot NAO.
• November 2019: Talk on general direction of my research.
• October 2019: Master students wanted for a master thesis project on gesture generation for a humanoid robot.
• September 2019 Talk on How to make your agent gesture in a natural way at Max Planck Institute for Intelligent Systems: Perceiving Systems Department in Tuebingen.
• August 2019: Our gesture generation model was applied to a new dataset. Now it can gesticulate in both Japanese and English. Check out a short demo video and our code with a pre-trained model.
• June 2019: Our two papers were accepted for ICDL-EPIROB 2019 Workshop on Naturalistic Non-Verbal and Affective Human-Robot Interactions.
• May 2019: Talk on my research at the Pint Of Science in Stockholm.
• April 2019: Our paper Analyzing input and output representations for speech-driven gesture generation was accepter to IVA 2019 for oral presentation. (24% acceptance rate)
• Jan 2019: Our paper On the importance of representations for speech-driven gesture generation was accepted at AAMAS 2019 for poster presentation.
• Oct 2018: My project proposal was published ICMI Doctoral Consortium 2018.
• April 2018: Joined Social Robotics Sweden (SoRoS) community.
• June 2017: We had a poster at The First Swedish Symposium on Deep Learning (SSDL).

   

Academic Service

• Co-organizing
  • ICMI’21 Workshop on Generation and Evaluation of Non-verbal Behaviour for Embodied Agents
  • IVA’20 Workshop on Generation and Evaluation of Non-verbal Behaviour for Embodied Agents
  • GESPIN 2020 Conference
  • SoRoS 2020 Workshop
  • NIPS 2018 Workshop on AI for Social Good
• Reviewer for
  • ECAI 2020, IJCAI 2020, SIGRAPH 2020, IVA 2020, ICMI 2020 LBR
  • ACII 2019, ICSR 2019
  • NIPS 2018 Workshop on AI for Social Good