The Australian Composers’ Residency in HfMT, Hamburg, Germany

Complete data set: https://rdmc.nottingham.ac.uk/handle/internal/10506

Performances:

502 Days of self by Iran Sanadzadeh: https://www.youtube.com/watch?v=kPy1iQK8NrM

Immanence by Chloe Sobek: https://drive.google.com/file/d/1TWYYqLaGEe_gDnrZWFUOVac–Y1_kRgD/view

The project by DigiScore partner, Cat Hope was aimed at commissioning four Monash University early career composers/performers/researchers to perform in an ensemble concert in Hamburg. The project enabled the participants to hire technologists to assist them in the realisation of their digital score idea which could include Artificial Intelligence, Machine Learning, VR/AR/XR, etc. The participants did not have extensive experience in making digital scores with new technologies before therefore the project also looked into what this could bring to them as composers. In addition, DigiScore supported the final preparation and the performance which took place in Hamburg’s School for Music and Theatre (HfMT). In realisation of their digital scores there, participants benefitted from their peers and mentors’ feedback as well as from HfMT’s multimedia faculty and community.

Four individual projects were realized:

Jaslyn Robertson’s Shadow Aria is a spatialized work using multichannel audio and lighting as a digital score. Conceptually, it is about the silenced voices of marginalized people in new music. When a spotlight is directed at a performer, they are instructed to stop playing and stop moving. In this context, this represents the silenced soloist as a metaphor for erased marginalised voices.

Helen Svoboda’s Wormwood is an animated visual score made in Max/MSP for solo double bass and video. It is inspired by the patterns on the tree bark which are translated into different shape categories to represent overtone groupings and their integer multiples. The shape categories act as launching pads for improvisatory moments, with the inclusion of some extended techniques to vary the timbre and palette of sound. The form is different across each performance; a different order of events is presented to the performer each time.

Iran Sanadzadeh’s 502 Days of self is a digital score which uses AI and Machine learning categorization of pre-recorded 502 days of the composer’s practice data on a pressure-sensitive floors instrument. Its examination of the recordings used speech analysis algorithms. The composer wanted to understand her formed habits on the instrument by sonifying them to other instruments with existing sonic gestural vocabulary. The score translates these pre-recorded habits into the movement, shape and colour of polygons on each part, which the performers interpret based on instructions from the composer.

Chloë Sobek’s Immanence is a VR work which uses paintings of Julian Aubrey Smith. The visual environments and movements of Smith’s paintings served as sonic prompts in the digital score composed by Chloë Sobek. Additionally, the work takes the audience on a journey through ‘alien’ worlds that are strange and yet familiar; depicting the remnants of human presence in a world absent of any perceivable human life. Immanence is about exploring ideas of VR’s ubiquitous use that could become part of our everyday life with companies like Metaverse. The work contemplates the positive and negative aspects of VR, with a particular focus on ways to represent post-anthropocentric values in art.

Critical Insights

502 Days of Self

AI - translation of sonic features through an AI algorithm to an animated digital score can fail to make a connection with the musicians without a rehearsal process

Managing parameter scale - animated shapes had to be reduced in the number of parameters to facilitate players’ connection to the digital score

Performance strategies:

players connected to each other’s sound as they were paired up in interpreting the animated score; in this way, they could react to the rhythmic interpretation of each other
intentionality in players’ sonic gestures was more important than following the shapes closely

Further reflection on the behaviour of the animated notation as guided by a Machine Learning algorithm could lead to the development of a new sonic aesthetic in Iran’s work

Wormwood

Randomization - the algorithm in the Max/MSP patch of Wormwood propelled the performer to make different creative decisions to her habitual improvisation habits on the double bass

Emergent behaviour in the performance - performer had to be reactive and step out of her comfort zone when interpreting the work

Surprising elements:

pushed the performer to explore different overtone groupings on her instrument
purple flashes intended to break the performer away from the previously chosen bowed overtone texture into a more percussive pizzicato
did not make her move too drastically away from the previous texture but triggered a change rather than a disruption to her flow

Shadow Aria

Lighting Interruptions:

players did not feel a disruption with composed interruptions when a spotlight had to stop their playing making them freeze momentarily
were experienced more as a trigger for a change rather than an interruption in one’s flow
affected the form of the piece and players’ decision-making

Managing the digital score:

the decision to go back to playing what they were before or to continue with something else helped to maintain their sense of flow and engagement in the piece

On Form: sudden interruptions also affected the form of the piece and players’ decision-making Connection to the spatialized soundtrack and reaction to each other’s playing helped to maintain a sense of flow during the interruptions

Immanence

Composer’s intentionality:

the sound world of the score was aimed to create an atmosphere rather than a direct relationship to the objects
composer aimed to make the interactive form feel alienating, there is some logic there, but it is not directly perceivable

Flow through navigation was achieved with each revelation of the new painting, moving the viewer closer and closer towards a ‘portal’ and through to the other side Navigation through the sound world of the digital score also contributed to the viewer’s immersion

On Form: audience’s expectations of the form through the visual prompts were constantly propelled forward without resolution

Viewing the work through a headset provided further immersion that allowed one to move with the work where the visuals and audio became one

Summary

The digital scores presented in Hamburg HfMT had many similarities between them in the materials used for their construction as well as how the musicians envisioned the performers to connect to them. For instance, Wormwood and 502 Days of self used animated interfaces to communicate composers’ ideas to the performer(s); they also used multiple parameters to communicate different choices performers could make in interpreting the works. Also, in both cases, Max/MSP was used in generating a digital score as animated shapes and categories for the performers to interpret. In addition, Max/MSP randomization of spotlights during the performance was a crucial element in Jaslyn Robertson’s Shadow Aria. All of the works had a strong conceptual structure behind them; shaping the creative process and the performance. In Shadow Aria, Jaslyn Robertson attempts to recreate an environment of censorship in a physical space. The work was built on the composer’s previous experience making a piece using a webcam. In this piece, when a musician plays into the webcam it distorts a video image to which improvisers could react. Here, she wanted to put the improvisers in a physical space with an environment for improvisation using spatialized audio and spotlights that would turn on randomly on different players. In Immanence, the visual environments were dictated by the paintings, serving as an impetus for objects and animation. These visual elements served as a form in recording sonic parts, representing an audible world which was ‘eerie’ and ‘stark’. The work is meant to be experienced in an immersive VR space in which the sonic atmosphere contributes to the immersion. All the digital scores had a striking visual presence for the performers and some for the audience. Shadow Aria functioned as an audiovisual work with spotlights turned on a performer to serve as a cue for that performer to freeze and stop playing. Wormwood was a projected animated score both for the performer and the audience. While Immanence was a VR work played both as an audiovisual work in the performance and experienced later in a VR headset by the audience.

During their residency in Hamburg, the participants learned many new digital skills. In many cases, these skills were building on the already learned skills, such as Reaper DAW skills which were extended to working in VR using Blender; Max/MSP and spatialization skills which were adapted to the new performance space in Hamburg. In some cases, new digital skills were learned before the residency such as which types of algorithms work effectively for data categorization and their transference to visualisation in Max/MSP; learning to use Adobe After Effects and Max/MSP. Wormwood and 502 Days of self were developed with technical collaborators in Australia who were not present in the residency in Hamburg. This meant that the composers had to become more independent when they were troubleshooting or making changes to their scores in Hamburg. This enabled them to reflect on the nature of collaboration where initially some control is given up on making the work. However, the positive outcome for the composers was learning new skills and becoming more independent with the technological aspect of their digital scores.

Some reflections made on the nature of digital technology was that an inherent part of digital technology is that it is not going to work 100 % as one intended. Using new technologies in new performance spaces with new setups could be challenging. One needs contingency plans, patience and the ability to rely on oneself. Overall, all the works were designed to be accessible to the performers and the audience. For instance, none of them required any technical digital music skills from the performers other than some familiarity with interpreting animated digital scores and the ability to improvise. The performers in this project had a lot of previous experience with different forms of animated digital notation. The participants also benefited from the presence of the Decibel Music Ensemble experts in animated notation. However, some performers of the digital scores had to conceptualize their way of thinking about the digital scores presented to them to interpret them. In Iran’s 502 Days of self, the performers had to follow the intentionality of their sonic gestures more than following with changes to the shapes. For example, players could stay silent through 2 or 3 changes if it helped them in finishing or starting their gestures more smoothly.

Conclusion

All the participants had innovative new experiences through working with their digital scores or they hoped to create such an experience for the performers of their works.

For Iran Sanadzadeh, making 502 Days of self, using Machine Learning analysis from the recordings of her performance practice on the pressure-sensitive floors instrument allowed her to have a perspective on what is possible across many different ways in animating scores while using digital components. It was also an opportunity to deeply explore through the use of AI and Machine Learning the relationship between movement and sound, encouraging the performers in her piece to explore their gestural vocabulary.

For Chloë Sobek, the creation of the score was a transformative experience as it showed her how a digital score could operate in a more interactive and immersive environment. Making a VR digital score contributed significantly to her skill set and allowed her to build on her previous skills in Reaper. In the future, she hopes this could be a transformative experience for the audience once the digital score becomes more interactive. Perhaps, also extending the possibility for them to create sounds through the game engine while being immersed in the digital score, creating their unique version.

Helen Svoboda’s visualization for the double bass in Wormwood encouraged spontaneity and challenged her embodied habits as an improviser through different pacing, structure and dynamics. By relying on the digital score, she found new ways to navigate sound on her instrument while discovering new sound combinations. She finds this to be a transformative component in this digital score. She hopes Wormwood will continue to inspire herself and other performers who will interpret the work.

While Jaslyn Robertson created a performance environment where players could respond sensitively with care while listening to the subtle movements of the spatialized audio. To do this, she had to overcome the challenges of working in a new space in Hamburg with a different speaker configuration and lighting setup. She also had to become comfortable in giving up some control over the musical material by including musicians’ improvisation in response to the audio score as well as a randomized spotlight element.

Working on the digital scores, prompted the composer/performers to reflect on how using different new technologies could be transforming their creativity. For instance, Iran finds that based on her experiences with 502 Days of self, innovative new ways of using digital notation could create a new sonic aesthetic in her work. In making the digital score, the process of translation of her audio recordings to the animated notation through Machine learning analysis challenged her sonic aesthetic as a composer. In this process, she had to constantly evaluate how the behaviour of the animated notation reflected her original intentions and what sound world it was creating. Iran works primarily with text scores, thus the process that she chose for making 502 Days of self, showed her a new way of composing.

While Chloë Sobek reflects that the experience of making Immanence made her feel like she is expanding her sonic practice towards more visuality. She questions how much of it relates to being a musician and a composer. She also questions how deeply she could go into this field before engaging in a collaboration with people who work in the visual medium. In this discussion, it was highlighted that her expansion towards more transdisciplinary might not be embraced in some circles of the music field which still praises expertise in one’s field above other forms of creativity. While this reflects on other conversations DigiScore has had with practitioners who venture into VR and other technologies, we are also noticing that many musicians choose to engage with transdisciplinary practices despite the disciplinary pressures.

Personnel