A. Agents - The Score as an Interactive Agent

In this part we focus on the Agents dimension, which reconceives elements of the music system from a static set of instructions to a behaving partner that has properties that can listen, learn and respond. In traditional musicking the “score” can often be an algorithmic scaffold that executes pre-written instructions. The Agents dimension asks us to consider that scaffold as a social actor in which operational elements can be proposed, or felt, as an entity that exhibits a measurable (though simulated) intentionality.

Core Analytical Insights

Four interrelated concerns structure the Agents dimension:

1. Perceived agency asks how performers sense the score as an autonomous, responsive partner, emphasizing negotiation with an algorithmic participant. In the corpus, musicians described the score as an organism that “listens”, “adapts” and “transforms” their gestures. For example, flashing icons demanded a new direction and offered an emergent dramaturgy where memory and disruption coexisted, creating continual negotiation.
2. Tangible presence looks at how visual, haptic or auditory feedback creates the sense of a “being” in the performance space, linking sensor design to a felt non-human collaborator. For example, sensors (mic, motion trackers, pressure pads) provided haptic or visual feedback, fostering a phenomenological presence even when static graphics reacted to musical activity and devices such as robotic arms or wearables blur the performer-system boundary.
3. Liveness of pre-recorded material defines how fixed media become active, responsive components, showing that meaning can arise when “dead” recordings were re-interpreted in real time. For example, fixed loops, bird calls or textures were modulated by dynamics and phrasing, turning the media into responsive partners.
4. Behavioural meaning of animation outlined the semiotic weight of moving graphics that cue, mirror or generate music, revealing a multisensory loop that binds sight, sound and agency. For example, animated icons (growing lines, flickering shapes, wandering mycelium blobs) encode intent and generated sound directly, forming perceptions that cue and material production are in a reciprocal visual-aural dialogue.

Sub-field analysis

Below each sub-field is introduced with a brief description, followed by the three (or four) themes that emerged through the analysis of the DigiScore corpus. Themes are labelled A1-A12 for easy reference in later discussion.

A) Perceived Agency

Musicians repeatedly described the score as a behaving organism that “listens,” “adapts,” and “transforms” their input. When a gesture was rejected, for example, signalled by a flashing red icon urging a different direction, the score shifted perceptually from static notation to an interactive partner. At these moments the system appeared to refine or reject a gesture urging the performer to “try a different direction”. The experience of the score shifted from static notation to an interactive partner.

Key themes. • A1 - Emergent dramaturgy. The system combined fixed anchors (composer-set timings, visual triggers) with probabilistic functions (e.g. “echo,” “obliterate”). Memory (recurring fragments) and disruption (randomised alteration) coexisted, creating a dramaturgical tension that forced performers into continual negotiation. • A2 - Negotiated listening. Performers developed a listening relationship toward the algorithm, anticipating its responses and adjusting their gestures accordingly. This created a reflexive loop in which agency was reportedly co-produced. • A3 - Trust & risk. The degree to which performers trusted the system’s decisions determines how boldly they explore the sonic space. When trust was low, the performer may “play it safe”; when trust rose, more adventurous gestures emerged.

B) Tangible Presence or the Feeling That Something Is There

In a sub-section of the corpus, it was noted that even static graphics acquired agency when they grew, shrank, or disappeared in response to musical activity. Musicians often report receiving a clear “yes/no” from the system, akin to a conversational partner. Sensors (microphones, motion trackers, pressure pads) delivered real-time physical feedback (e.g. vibrations, colour changes, kinetic movement of on-screen graphics) and collectively these provided real-time physical feedback and fostered a palpable sense with many that an entity co-existed with the performer. A knock-on effect here was that musicians started to “test” or “game” the system and received clear responses. Robotic arms, avatars and flag-like wearable devices further extended the score’s embodiment, blurring the divide between performer and system.

Key themes. • A4 - Phenomenological presence. The system was experienced as an Other that inhabited the performance space, not merely invisible code. • A5 - Embodied interfaces. Haptic or kinetic devices (e.g. robotic arms, wearable devices) extended the score’s body, blurring the boundary between performer and system. • A6 - Multimodal feedback loops. Visual, tactile and auditory signals converged, creating a sensorimotor loop that reinforced the feeling of a co-existent agent.

C) The Liveness of Pre-Recorded Sound

In many examples, fixed samples such as bass-flute loops, bird recordings and environmental textures, were modulated in real time by the performer’s dynamics, density or phrasing. Musicians described these layers as “active partners” that reacted to their playing, sometimes overriding visual cues or returning altered fragments (such as echoes). The necessity to familiarise or anticipate the behaviour of this static material pushed performers into a dialogic stance with the recordings themselves.

Key themes. • A7 - Re-animation of dead media. Algorithmic “listening” turned inert recordings into responsive agents that could echo, invert, or otherwise transform their content. • A8 - Dialogic anticipation. Performing with pre-recorded material forced musicians to memorise or anticipate the material’s behaviour, establishing a dialogue with the recording itself. • A9 - Temporal dissonance. The tension between the fixed temporal signature of a sample and its real-time manipulation created a temporal elasticity that enriches the musical narrative.

D) The Behavioural Meaning of Animation

Animated icons, such as growing lines, flickering shapes, wandering “mycelium” blobs, were encoded with musical intent where, say, size indicated intensity and speed signalled rhythmic acceleration. Some animations generated music directly (e.g. 3-D drawing interfaces that spawned notes), while other animations listened to the sound and adjusted their behaviour, altering density or colour in response. Frequently, these visual agents embodied ecological metaphors, such as swarms that mimic biological networks, prompting performers to inhabit a visual ecology rather than merely read a score.

Key themes. • A10 - Performative visual agents. Animation itself became a behaving actor that cued and produced musical material, establishing a reciprocal visual-aural dialogue. • A11 - Ecological metaphor. Visual agents often embodied ecological metaphors (swarms, growth, decay) that invited performers to inhabit a visual ecology. • A12 - Semiotic layering. Visual parameters (colour, shape, motion) served as semiotic layers that reinforced or subverted the sonic material, deepening the multimodal meaning.

Take-away points

This analysis shows that agency is not confined to a single element but is distributed across code, visualisation, sensors and the performer. Given the importance placed on these properties by the DigiScore musicians, it is recommended that designers should treat the whole system as a co-creator rather than a static container. Because perceived agency reshaped musicianship, pedagogy could embed “listening to the algorithm” as a core skill. Meanwhile, tangible feedback born from haptic, visual, kinetic sources, can create a phenomenological presence. Understanding this through practical exploration of how embodied interfaces (such as gloves, pressure pads or robotic arms) can be harnessed to foster a sense of partnership seems to be a rewarding and useful skill to develop. Additionally, approaching the re-animation of dead media can broaden the expressive palette, meaning that fixed samples can become living agents when equipped with real-time modulation or probabilistic “listening” modules. Finally, animated graphics can carry semiotic weight, so visual designers could treat colour, shape and motion as musical parameters instead of mere decoration.