Dance AI Recognizes The Movements of Breakdance by Deep Learning
About DanceAI Project
Update: May 23, 2021
Hello, I’m FollowTheDarkside, breakdancer, engineer and videographer. I will introduce Dance AI that researched and developed in collaboration with Daichi Shimizu (Graduate School of Education, The University of Tokyo), Junya Kobayashi (The College of Liberal Arts and Sciences, Chiba University) and Almina.
First of all, please watch the following video.
We are dancers with different backgrounds, and researching and developing AI for break dance, which is an element of hiphop culture.
We use sensor devices like Smart Footwear ORPHE, smartphones, Arduino and MMR to recognize break dance movements.
Simple description of the mechanism
By using sensor devices and specially developed applications, sensor data including 3-axis acceleration received by Bluetooth can be recorded as a CSV file. The acceleration is extracted as a matrix from this CSV, and deep learning is performed by the CNN model (Convolutional Neural Network). The model is built by Keras (TensorFlow backend). The training data is labeled by movement and uses classification techniques.
As a result of cross validation, the accuracy rate was as high as about 96%.
Since the previous application is made in Swift, CoreML is used to incorporate the learned model into it. This makes it possible to recognize the movement of the shoe wearer in real time.
As you might guess, data collection is very very very hard! It is literally a crystal of blood, sweat and tears!!!
Future
Breakdance has become popular all over the world. Recently, it has been provisionally adopted for the Paris Olympics 2024.
Dance AI will help the public people to understand breakdance. In the future, I would like to develop an evaluation AI of important elements in HIPHOP culture, such as the originality of dancers. It may be interesting even if AI becomes a judge of a dance event. However, I do not expect to replace everything with AI. As a BBOY, I want to use AI to find new value and to further develop the culture.
Peace, Unity, Love & Having Fun!
