Daniel M. Lofaro, David Grunberg, Paul Oh, Youngmoo Kim, and Jun Ho Oh
Humanoid Robot, Human Robot Interaction, Musical Beat Tracking, Visual Beat Tracking
An interactive participant in a live musical performance requires a multitude of senses in order to perform. These senses include hearing, sight, and touch. Our long-term goal is to have a full-sized humanoid (Jaemi Hubo) become an interactive participant in a live music ensemble. This paper focuses on Jaemi Hubo's sense of sight and hearing as it pertains to a musical environment. The humanoid's musical awareness is increased through a hybrid approach that combines real-time auditory and visual beat tracking. Experiments contrast audio-only, vision-only and hybrid tracking. These results demonstrate the efficacy of hybrid tracking as a means to realizing a robot's ability to become interactive musical participants.
Important Links:
Go Back
