Fold by Ted Moore

Fold for piano, sax, bass clarinet and video

by Ted Moore, Browser Festival commission

Composer’s Documentation: https://www.tedmooremusic.com/works/fold/

Full dataset: https://rdmc.nottingham.ac.uk/handle/internal/10796

Fold by Ted Moore reflects the theme of “location sharing” by using audio feedback to “sound out” the resonant frequencies of the locations of the remote collaborators like Curt Miller in Rhode Island, US and Jana de Troyer in Ghent, Belgium. Using the resonant tubes of their instruments, the performers created and manipulated audio feedback tones in their remote locations. These sounds were broadcast over the internet to the performance space where the audience was present in Stuttgart, Germany. The feedback was generated in the remote locations through a feedback control patch in Max/MSP. Additionally, each remote performer’s sound was analyzed for pitch-class content and frequency content and sent to the performance space (over the internet using a Max package called Colab-Hub) as data, directing the pianist’s playing. The pianist’s gestures were pre-composed as rhythms and registrar shapes, but the pitches were mostly supplied by the remote analyses of audio feedback. The hope was that the physical properties of the remote performers’ instruments and spaces would manifest in the performance space as acoustic sound filtered through the piano part. Overall the piece is slow and meditative, inviting everyone to enjoy the sonic relationships between the feedback and the piano. The form of the piece contains different sections where Curt and Jana are doing different things (and we can or cannot see them visually on the projection). Each has a different score indicating when to create feedback vs. performing tones. Also, a pre-composed piano part appears about two-thirds of the way through the performance, grounding the piece and the listeners in the presence of Alex’s expressive piano playing.

Objective

Originally, the composer wanted to experiment with the computer vision algorithm where performers would use a webcam to share a scene in nature from their remote location. The computer would detect motion and objects in the video and translate them to musical parameters. These parameters would be displayed for each performer to realize the acoustic performance with the live feed of both nature scenes and remote performers being streamed into the concert hall. However, halfway through the realization of the project composer decided to change the original plan for the execution of the work to embrace the theme of ‘location sharing’ of the festival sharing the ‘sonic’ space of the remote performers with the sonic space of the (in-person) performer and audience.

Process

To realize his idea of sharing resonant sonic spaces of each performer, the composer had to find a way to send frequency analysis data over the internet. It was helpful to find Collab Hub, a Max package that did that for him. To control audio feedback, the composer decided to use FFT analysis in Max instead of Supercollider which he was initially thinking of using. Having a working feedback strategy allowed him to trust that the performers would build comfort with the system. With time he started drafting the outline of the piano part that the feedback analysis pitches would fill as well as performance instructions for the performers on how to navigate and connect to the various elements of the digital score instruction, Max patch, feedback system, etc.), thinking about the user experience and how all the parts would work together. The composer used GitHub to update the patch regularly for the performers to have the latest working version and to see how the various versions of the patch worked with Collab-Hub. When he started testing the piece with the performers through Zoom, he noticed unexpected high-frequency feedback from one of the performers and decided to add more high-pass/low-pass filters to attenuate this. Some more tweaking had to be done in the performance space in Stuttgart to accommodate for undesirable frequency feedback such as adding a ‘reset button’ to clear FFT bins to keep the frequencies in the desired range as the performance goes on.

Critical Insights

Materials

Feedback

feedback was an important material through which performers made connections to the digital score
feedback sections were accompanied by text score instructions which performers found easy to follow and they thought that they shaped their playing
performers followed text score instructions in different ways (memorizing, summarising, abbreviating to fit the screen, etc.)
machine listening through a max/MSP patch facilitated the analysis of remote performers’ feedback pitches so that it could become material for interpretation between them
feedback pitch analysis was provided at different points in the piece for the performers to react to: at first the saxophonist reacted to the bass clarinettist’s feedback pitch analysis, then at the end of the piece they had to switch
the pianist’s notation was connected to the pitch analyses of the remote performers’ feedback pitches
the remote performers had to develop a close relationship with their playing and the electronic layers of the piece to perform feedback

Dislocated performance

there were different degrees of dislocation that the performers felt in the performance: Curt and Jana could only hear their individual sounds due to the feedback system setup outwards from Zoom, and the pianist could not see the video projection of the remote performers
in this way, the performers had to create a relationship with the DS through their individual disconnectedness by trusting in each other’s virtual presence
the pianist felt that the pitch analysis material started to take on a personality of its own because of the fast-changing pace of the material
pitch analysis did not always pick up on the pitches that the pianist heard from the remote performers, it acted unpredictably and more spontaneously like a real chamber music partner
since there was so little actual contact between the players, the virtual playing of the remote performers could take place as live electronics and tape
however, just knowing that one is playing with other people with very little actual contact still radically changed the experience of playing together
musicians think that this has to do with the responsibility and trust for the relationships with the people that you know are physically there even though you cannot sense their presence
it was meaningful for the performers to see the evidence of their liveness through the Max score and Zoom interfaces

Video

although the composer designed the video material of the digital score not to be instructive towards the performers and more directed at the audience, performers still made a connection with it
performers felt some direction in the piece through their interaction with video detection and its distortions
one of the performers used subtle movements to encourage the video to react in a ‘pop art’ distorted way as the piece was progressing
video designs that performers were sending to each other over Zoom prompted them to find a connection to each other even if they felt sonically isolated

Flow

the composer found flow in his process of working on the DS once he decided to use the feedback of performers’ resonances from remote locations as material
performers felt flow individually as their parts were structured to do different things in different sections
once the piano performer worked out how long each section lasted, he was able to let go and let the digital elements of the scoring act as stimuli to shape his playing
the text score was meant as a guide for the performers to know how the feedback unfolds over time
interacting with feedback in their own spaces was a rich experience for the performers as it was very engaging even though they could not have full control over it or hear what the effect was in the physical space in Stuttgart
there was also surprise in navigation through feedback as some notes in the pitch analysis would reflect the feedback that the pianist was hearing, and others were not
unbeknownst to the composer the video detection was guiding remote performers through the work and providing connection in their isolation
however, because of the technical setup difficulties of the piece and the overall demands on the attention of the performers, it was difficult to get into a long feeling of flow
performers felt like they needed to switch between different materials of the score and not being able to see or hear each other sometimes hindered their sense of flow

Digital musicianship

it was a new approach for the composer to use audio feedback through Max/MSP as previously he has used Supercollider; this approach builds on his knowledge of feedback in Supercollider and extends it to Max/MSP
the composer learned new skills in sending data over the internet by using the Collab Hub Max package that he hopes to use in the future projects
the two remote performers using feedback learned to control audio feedback and have a new relationship with their instruments
it was an experience of this remote triangular communication that one of the performers will keep reflecting on for future online/remote projects
this experiment taught the performers how far Zoom could be stretched as a medium via sound sharing and routing through Max
because of the isolation of the remote performers’ one of the performers felt like she had to let go of what is normally considered chamber music to embrace the dislocated and disconnected performance setup

Transformations

the composer’s main concern in the composition of this piece was to create a transformative experience for the audience as well as to join remote performers through feedback in their remote locations
the composer used a video in the piece as it can be an orienting medium for the audience if one is unfamiliar with experimental sound
the composer feels that the experiences he had in composing the piece opened some doors for him conceptually and technically
one of the performers would like to keep exploring the use of digital scores in which the machine interprets something from human performers and other performers are reinterpreting the analysis from the machine in different forms
it was transformative for the interaction with the material of the piece just knowing that other performers were there and playing this piece together even without the ability to see or hear them

Summary

Fold by Ted Moore is a composition that explores Browser Festival’s theme of “location sharing” by using audio feedback to connect remote collaborators. The performers, Curt in Rhode Island and Jana in Belgium manipulated audio feedback tones using the resonant tubes of their instruments. These tones are broadcasted over the internet to a performance space in Stuttgart, Germany, where a pianist interacts with them. The feedback is generated through a Max/MSP feedback control patch, and each remote performer’s sound is analyzed for pitch class and frequency content to guide the pianist’s playing. The composition is slow and meditative, inviting listeners to experience the relationships between the feedback and the piano. The piece is divided into different sections, with each performer engaging with feedback at different times. The original concept involved computer vision translating nature scenes into musical parameters, but the focus shifted to emphasize sharing sonic spaces among remote and in-person performers and the audience. The composer used the Collab Hub package in Max to send frequency analysis data over the internet. Feedback was a central element, fostering connections among performers and forming the digital score. Performers adjusted to feedback, sometimes reacting to each other’s analyses. Dislocated performances created a sense of trust and connection between remote performers and the in-person pianist, despite their limited visual and auditory contact. The piece’s video content starts with distorted visuals of performers and evolves as the composition progresses. Performers found a connection with the video, even though it was primarily directed at the audience. During the composition process, flow emerged for the composer when using remote performers’ resonant feedback as material. The text score guided performers through feedback sections, while both remote performers found engagement in experimenting with their instruments’ feedback. The use of feedback created surprise and added complexity to navigation. The experience led to new digital musicianship skills for all involved. Using Max and Collab Hub expanded the technical capabilities of the composer as well as provided a means of generating feedback pitch analysis for the remote performers. The remote triangular communication left a lasting impression on the performers, informing future online projects. The composition’s transformative nature and interaction with the digital score resonated with performers, opening conceptual and technical doors. Despite challenges, the digital score highlighted the meaningful connection between performers through evidence of their liveness in the Max score and Zoom.