Speech2Properties2Gestures: Gesture-Property Prediction as a Tool for Generating Representational Gestures from Speech

Taras Kucherenko, Rajmund Nagy, Patrik Jonell, Michael Neff, Hedvig Kjellström, Gustav Eje Henter

International Conference on Intelligent Virtual Agents (IVA '21)

Honorable Mention Award

[Project proposal]
[Detailed analysis of gesture property predictability]

 

 


 

ABSTRACT

We propose a new framework for gesture generation, aiming to allow data-driven approaches to produce more semantically rich gestures. Our approach first predicts whether to gesture, followed by a prediction of the gesture properties. Those properties are then used as conditioning for a modern probabilistic gesture-generation model capable of high-quality output. This empowers the approach to generate gestures that are both diverse and representational.

portrait

 


 

Citation format:

@inproceedings{kucherenko2021speech2properties2gestures,
      title={Speech2{P}roperties2{G}estures: {G}esture-Property Prediction as a Tool for Generating Representational Gestures from Speech}, 
      author={Kucherenko, Taras and Nagy, Rajmund and Jonell, Patrik and Neff, Michael and Kjellstr{\"o}m, Hedvig and Henter, Gustav Eje},
      year={2021},
      isbn = {9781450366724},
      publisher = {Association for Computing Machinery},
      address = {New York, NY, USA},
      url = {https://doi.org/10.1145/3472306.347833},
      doi = {10.1145/3472306.347833},
      booktitle = {Proceedings of the  21th ACM International Conference on Intelligent Virtual Agents},
      location = {Virtual Event, Japan},
      series = {IVA '21}
}