In the summer of 2024, I worked with choreographer and particle physicist Dr. Mariel Pettee on an exciting development using A.I. to generate partnered dance choreography. This innovation came from training AI models on videos captured from studio practice, allowing the machine to learn and generate new dance moves, a process well illustrated in our paper Beyond Imitation: Generative and Variational Choreography via Machine Learning.

 

Building on this work, our project aimed to take things a step further by focusing on duets. We were interested in exploring how AI can choreograph dances that involve complex interactions between the dancers, reflecting a more dynamic expression of human connection and creativity. The idea was to expand the aforementioned existing work in AI-generated choreography by introducing modern machine learning techniques to create dance sequences for duo performances. Initially using a generative model with LSTM units to learn from motion-captured dance movements, this project proposed a new approach by incorporating self-attention mechanisms and Graph Neural Networks.

 

These technologies can allow for a deeper understanding of the relationships within and between dancers' movements over time. More specifically, the project explored the integration of self-attention mechanisms to replace traditional LSTM sequences, enabling the model to more effectively emphasize significant connections across frames of dance movements. Additionally, it sought to benefit from capturing the spatial relationships between dancers' joints by using GNNs for pre and post-processing. The deliverables of the project consisted of the developed dataset and its visualizations, the implemented agents with their respective weights, and a report together with a repository containing the full documentation and code for each step. 

​

Through a partnership with HumanAI and Google, Mariel and I supervised two talented computer scientists as they research and develop this model. More information can be found in this medium post by one of our  collaborating computer scientists, Luis Zerkowski.