Synaesynth is an experimental instrument that translates the colours of your life into music. It achieves this through a unique blend of software and hardware, creating a simulated synaesthetic experience. This project sits at the intersection of Product Design, Interactive Art Practices, and Human Computer Interaction (HCI), exploring a tangible interface between the digital and physical worlds.
Synaesynth operates by capturing colour data from a video feed and converting it into musical tones. The software presents a grid-divided video feed, matching the hardware's button grid. Pressing a pad captures the colour of the corresponding video segment, which is then algorithmically translated into a MIDI note, which can be sent to a synthesiser.
Initial software prototypes were developed in Python, GML, and Processing, with Processing ultimately chosen for its user-friendliness. Research also included exploring mobile Raspberry Pi setups and web-based controls. User surveys conducted on Reddit helped refine the color-sound conversion rules, with a focus on expectations from musicians and laypersons alike.
User testing with a hardware prototype was undertaken with bedroom DJs, beat-makers, and producers played a crucial role in refining the Synaesynth. Filmed sessions, screen captures and audio recordings, along with interviews and participant sketches, informed key feature developments.
Synaesynth contributes to generative art and electronic music, enhancing discussions on augmented reality and emerging media. It allows artists to create context-responsive works, and in live performances, it can create an audio-visual feedback loop. The project aims to inspire reflections on cross-sensory perception and the modern challenge of sensory overload.
The project invites users to explore and reimagine their interaction with the world.
This project was made possible with help from Elliot Henkel, Philip Pille and Sonny W.
