The Challenge

Real-Time Visuals for Live Performance

At Istanbul AKM, this project for Türk Telekom was designed as a real-time visual system in which the live music performance directly shaped the entire visual language of the stage. The core idea was to treat the music not as an accompanying element, but as the primary source driving the visual narrative.

The audio signal captured from the live piano performance was fed directly into the system and analyzed in real time using TouchDesigner. Rhythmic structures, dynamic changes, and especially low-frequency data were extracted and converted into control parameters for visual generation. This allowed the tempo, intensity, and emotional shifts of the music to directly influence the behavior of the visual system.

These data streams were then routed into Unreal Engine, where the visuals were built using the Niagara particle system. Particle density, movement, and overall scene flow were shaped in real time based on the incoming audio data. As a result, every visual motion on screen was directly tied to the music being performed live.

The generated visuals were rendered and displayed on the LED wall through Unreal Engine’s nDisplay infrastructure with minimal latency. Synchronization between sound and image was treated as a critical component of the system and maintained consistently throughout the performance. Rather than following the music, the visuals were produced simultaneously with it, forming a single, unified stage language.

Through this approach, the on-stage visuals moved beyond pre-defined content and became a living system that continuously evolved with the performance itself, allowing the audience to experience the music both audibly and visually in real time.