Learning hand-motion to music by example

Abstract

The process of creating character animation requires hours of work and years of experience by skilled animators. Assisting this process by providing tools to automate some of the procedures during the animation creation is, therefore, of interest to many computer graphics researchers.

This thesis presents a system that generates arm dancing motion to new music tracks, based on sample motion captured data of dancing to other pieces of music. Rather than a common approach of creating character animation, which is to synthesize motions from an existing motion database, our system is novel in that it analyzes both the supplied motion and music data for certain characteristics such as distance-from-body, amplitude, and centres-of-motion (for motion data), and melodic contour, loudness, and note-density (for music data). It then learns relationships between the music and motion. When new music is provided, the characteristics are analyzed as before, and used to predict characteristics of the motion. A generative process then creates motion according to these constraints.

The system is evaluated with the music data that are used to trained the model, as well as new musical data including existing music tracks that are not specifically created for the purpose of animation creation.