On the Identification of Musical Notes via Multimodal Interaction

Lance July, Edward Cana, and Mehrdad H. Zadeh


component, haptic, piano, musical notes, pitch identification


Learning something new is never easy and musical notes are no exception. Many people have a difficulty identifying musical notes when simply hearing different sounds. Therefore, an interface that adds the additional sense of touch to ‘feel’ notes may increase the user’s ability to more accurately identify notes. In this preliminary study, a multimodal interface is developed to learn musical notes. Users can listen, feel, and see the notes using the interface. The interface includes a sound library and a virtual piano that graphically and haptically displays musical notes. In addition, a human factors study is conducted to investigate the effects of haptic feedback on the correct identification of several musical notes. User testing results show that adding haptics to musical notes increases the user’s ability to identify the notes that they hear.

Important Links:

Go Back