A Georgia Tech student has surrounded himself with a team of dancing robots and an improvising, marimba-playing bot to collaborate on an original, Miles Davis-inspired composition. Mason Bretan, a Ph.D. candidate in music technology, plays the drums, guitar and keyboard. A robot named Shimon listens to the sounds, then generates music on a marimba using its computational knowledge of jazz theory and improvisation. At the same time, a trio of Shimi robots autonomously generates dance choreographies based on a joint analysis of the music and a self-awareness of their physical constraints and abilities. The Shimis also play their own complementing music, based on a combination of Bretan’s original compositions and improvisational algorithms. The six-minute, high-energy funk piece is called “What You Say” and is based on Davis’ “What I Say.” It’s the latest project from the lab of Gil Weinberg, Bretan’s advisor and director of Georgia Tech’s Center for Music Technology.

This is a performance showcasing part of my PhD research in robotic musicianship at Georgia Tech including machine improvisation, path planning, and embodied cognition. The smaller Shimi robots figure out how to move based on an analysis of the music and Shimon generates an improvisation given a precomposed chord progression using a generative algorithm that jointly optimizes for higher level musical parameters and its physical constraints. – Mason Bretan