Game of Tones: Learning to Play Songs on a Piano Using Projected Instructions and Games

Linsey Raymaekers, Jo Vermeulen, Kris Luyten and Karin Coninx
Presented by
Linsey Raymaekers

In CHI 2014 Extended Abstracts — Interactivity, Toronto, Canada, April 26th - May 1st, 2014, 4 pages.


  • Shared 3rd place in the Nokia Ubimedia Mindtrek Awards (NUMA 2013).


Learning to play a musical instrument such as the piano requires a substantial amount of practice and perseverance in learning to read and play from sheet music. Our interactivity demo allows people to learn to play songs without requiring sheet music reading skills. We project a graphical notation on top of a piano that indicates what key(s) need to be pressed and create a feedback loop that monitors the player’s performance. We implemented The Augmented Piano (TAP), which is a straightforward combination of a physical piano with our alternative notation projected on top. Piano Attack (PAT) extends TAP with a shooting game that continuously provides game-based incentives for learning to play the piano.



Press Coverage