Trialogue

Brief

Trialogue is a multimedia performance and a musical interface exploring the interactions between nature, human and machine through the law of physics, musical improvisation and artificial intelligence in real time. As a sequence of precipitation is presented among us, What do we hear as humans? What does a machine hear? How do we choose to respond? How does a machine choose to respond? Through analogue acoustic sounds, computer generated rhythms and visuals and lighting effects, Trialogue aims to highlight our similarities and differences, and curate a meaningful conversation between the three parties. This is a project co-created by me and Nuntinee (Nun) Tansrisakul

Concept Video

Program Description

Score 1: Genesis (2’)

“The origin. The beginning. The perpetual law of physics that gives birth to everything.”

A human performer initiates the trialogue by dipping her hand in the water and lifting it up to create a water drop sequence. A custom-made installation with a webcam and haptic sensors then captures the rhythm and dynamics of the water drops, and gradually translates them into a midi sequence.

Score 2: Mockingbird (2')

“We ape, we mimic, we mock. We act.”

Two deep generative networks join the conversation. A variational autoencoder (VAE) is trained to listen to the translated midi sequence and generate accompanying sequences that either mimics or complements the original sequence. Another LSTM will follow by generating subsequent notes that carry patterns and characteristics inside the previous sequences.

Score 3: Mediator (2')

“I kind of just float around and do my own thing.”

A second human performer chimes in and mediates between the drop sequence and the AI generated sequence by tapping, lifting, scratching, rolling an umbrella on top of a metal surface, which creates another layer of sound in relation to the other two participants in the trialogue. Contact microphones are installed on the metal surface to pick up the sound and trajectory of the umbrella’s movements for real-time audio playback.

Score 4: Trialogue (3' 30")

“From three to infinity.”

A dynamic session takes place among the three parties. The water drop creator, the AI, and the umbrella tapper will alternatively listen to each other, engage and disengage, tangle and untangle, and carry on until a mutual termination. Generative visuals based on the trialogue will be projected to illustrates the relationship among them.

Back to Experiments