Maryland Graduate Student Develops AI Violin-Teaching Program

The program, which can map students' movements in 3D, is designed to work as a practice tool to supplement in-person lessons

Snehesh Shrestha, a Ph.D. candidate in the computer science department at the University of Maryland (UMD), is in the process of developing a program that will use artificial intelligence to give live feedback to violin students.

The work is taking place as part of a collaboration between Irina Muresanu (an Associate Professor of Violin) and Cornelia Fermüller (an Associate Research Scientist at UMD's Institute for Advanced Computer Studies). The idea was born out of the COVID pandemic, during which in-person teaching was impossible and the two researchers began to explore ways to make remote teaching more effective.

In the program, students use a laptop webcam, or the front-facing camera on their phone, to capture video of themselves playing the instrument. The software can then turn the video into a 3D image, in order to evaluate factors like the student's posture, their grip on the instrument and bow, and the way they move.

The program is not designed to replace traditional teaching, but rather to support students' individual practice — and to make lessons accessible for those who could not otherwise afford them.

The point of the program is "to understand the whole space, not just blindly building a technology, but understanding how can we fill the gaps that are currently there in the entire music learning process," Shrestha said.

"And by identifying gaps where we can empower the teacher and the students, we could really build something a lot more powerful than just building a single technology," he continued. "And that really was like the starting point of exploring into what the technology can provide towards the future direction of music education."

"While I believe that traditional methods are still the best way to pass on to our students the legacy and heritage of the classical music world, I am excited to explore ways in which artificial intelligence can be integrated as a feedback mechanism into daily practice — the central experience of any musician’s life," Muresanu said.

"It will democratize music education," Fermüller added. "I’m saying democratize — it will make it cheaper for students. They can practice in between lessons. If they have a one-on-one teacher, they can use it to accelerate. But in schools, many students don’t have the benefit of private education, there’s often one teacher teaching many. So they can really use this tool."