Dance AI Recognizes The Movements of Breakdance by Deep Learning

About DanceAI Project

Naoyuki Hirasawa
3 min readJun 29, 2019

Update: May 23, 2021

Hello, I’m FollowTheDarkside, breakdancer, engineer and videographer. I will introduce Dance AI that researched and developed in collaboration with Daichi Shimizu (Graduate School of Education, The University of Tokyo), Junya Kobayashi (The College of Liberal Arts and Sciences, Chiba University) and Almina.

First of all, please watch the following video.

YouTube playlist of DanceAI

We are dancers with different backgrounds, and researching and developing AI for break dance, which is an element of hiphop culture.

DanceAI with sensor devices

We use sensor devices like Smart Footwear ORPHE, smartphones, Arduino and MMR to recognize break dance movements.

Simple description of the mechanism

Learning of sensor data

By using sensor devices and specially developed applications, sensor data including 3-axis acceleration received by Bluetooth can be recorded as a CSV file. The acceleration is extracted as a matrix from this CSV, and deep learning is performed by the CNN model (Convolutional Neural Network). The model is built by Keras (TensorFlow backend). The training data is labeled by movement and uses classification techniques.

As a result of cross validation, the accuracy rate was as high as about 96%.

Motion recognition is done in real time

Since the previous application is made in Swift, CoreML is used to incorporate the learned model into it. This makes it possible to recognize the movement of the shoe wearer in real time.

As you might guess, data collection is very very very hard! It is literally a crystal of blood, sweat and tears!!!


Breakdance has become popular all over the world. Recently, it has been provisionally adopted for the Paris Olympics 2024.

Dance AI will help the public people to understand breakdance. In the future, I would like to develop an evaluation AI of important elements in HIPHOP culture, such as the originality of dancers. It may be interesting even if AI becomes a judge of a dance event. However, I do not expect to replace everything with AI. As a BBOY, I want to use AI to find new value and to further develop the culture.

Peace, Unity, Love & Having Fun!

DanceAI recognizes three step, one of the basic movements of breakdance
DanceAI recognizes emotional state of dancer



Naoyuki Hirasawa

FollowTheDarkside (FTD) - creative coder / visual artist / writer / bboy - Portfolio: