M. MUSIC - The Sonic-Narrative Core of the PACMMAN Framework

The Music dimension is the musico-physical space that a digital system can conjures in the mind of a musician. It is a living geography that musicians navigate, map and co-create together with technology. The Music dimension asks us to look beyond notes and timbres and to treat the sound world itself as a narrative landscape that the performer can inhabit, explore and co-create.

Core Phenomena

Four phenomena recur across the DigiScore corpus:

1. First, an immersive, metaphorical environment created a felt sense of “being somewhere” (e.g. feelings of being underwater, floating, or upside-down) that grounded the musician while granting expressive freedom.
2. Second, the score functioned as a mutable visual world: animated objects driven by physics appear, drift and vanish unpredictably, compelling moment-to-moment decisions rather than obeying a prescriptive script.
3. Third, a meta-physical fusion of sound, gesture and technology collapsed traditional boundaries of time, distance and agency. As such, breath, motion, brain-computer-interface data and visual cues can become part of a single feedback loop that shapes the sonic output.
4. Fourth, the felt quality of the music and the imagined landscape hinges on, say, microtonality, extended techniques and fluid timbres, giving performers a mental meshwork that oscillated between control and surrender.

Applying these four as lenses across diverse works (e.g. ambient, Indian raga, game-like, VR-driven, for piano, flute, voice or electronics) reveals a consistent grammar: • The immersive, journey-like soundscape positions meaning in spatial navigation rather than static notation. • The dynamic, “living score” turned the performer into a cartographer who maps ever-changing sonic terrain. • The meta-physical convergence expanded musical material to include physiological and visual signals, making breath, motion and brainwaves audible agents. • The fluid, micro-tonal texture shifted analytical focus from pitch to timbre, density and spectral evolution. • Throughout, performers negotiated a tension between control and surrender, deciding whether to follow or ignore system cues and the overall structure resembles a choose-your-own-adventure narrative, where waypoints could be followed, delayed or bypassed, allowing meaning to emerge from each performer’s path.

Sub-field analysis

Amongst the DigiScore corpus, a set of nine key motifs emerged (labelled MU1-MU9). These motifs constitute the Music dimension’s conceptual grammar, capturing the territories of immersion, mutability, embodied techno-feedback, fluid timbral texture, agency tension and open, branching narratives. By viewing any single harmonic event through the four phenomena (above) as lenses of the Music dimensions the event can becomes an environment, a cartographic marker, a sensorimotor interface, or a storytelling node.

Key themes. • MU1 – Spatial metaphor (Immersive, journey-like soundscapes). Performers repeatedly described the sonic world as “underwater,” “floating,” or “a safe place to explore,” framing sound as a terrain that can be traversed. This spatial metaphor foregrounded embodied movement, as such meaning accrued through navigation of layered acoustic landscapes and the listener’s perception was shaped by the feeling of being inside a mutable environment. In this way a single harmonic gesture unfolded simultaneously within a sonic horizon, operating as both a navigational cue and a phenomenological experience of immersion. • MU2 – Safe exploration zone. The acoustic environments were perceived as a protected space that explicitly invited risk-taking and discovery. Musicians reported feeling “secure enough to experiment” while still being challenged by the surrounding perception of a terrain. The safety of the zone enabled a willingness to venture into unfamiliar textures, to follow unexpected cues and to treat the performance as an exploratory expedition rather than a rehearsed recital. • MU3 – Physics-driven sprites (dynamic, living scores). Visual and aural objects within the work obeyed simulated physical laws such as gravity, momentum, collisions and could appear, disappear, or shift unpredictably. The score thus became an animated, multi-layered environment, with, for example, “sprites” that behaves like physical agents. Performers felt that they could act as cartographers, mapping and negotiating the ever-changing landscape, which, for them transformed the score from a prescriptive script into a participatory terrain that responds to kinetic and algorithmic forces. • MU4 – Unpredictable emergence. New symbols and sonic events can materialise or vanish in real time, compelling the performer to make instant decisions such as “chase,” “ignore,” or “re-route.” This on-the-spot emergence reinforced the score’s organic quality, turning each performance into a live-generated ecosystem where the composer’s intentions were continuously renegotiated by emergent affordances. • MU5 – Sensor-sound coupling (Meta-physical convergence of sound, gesture and technology). In some case studies, breath, bodily motion, EEG/BCI data and other physiological signals directly modulated timbre, spatialisation, or visual feedback, turning the body itself into an audible agent. This embodiment of technology created a “meta-physical space” that fused sound, colour, symbols and improvisation, expanding the definition of musical material beyond conventional sound sources to include the performer’s embodied techno-feedback loop. • MU6 – Temporal fluidity. In several works, temporal elements such as latency, “toporhythm” (simultaneous but offset timing) and other asynchronous cues dissolved linear temporal hierarchies, collapsing past, present and future into a single fluid moment. Temporal fluidity destabilised the conventional metric grid and invited performers to navigate a temporal landscape where events could be anticipatory, concurrent or retroactive, reshaping the affective narrative of the piece. • MU7 – Spectral richness (Fluid, micro-tonal sound worlds). For some of the scores, the performance was characterised by micro-tonal detuning, extended techniques and dense overtones. Descriptions such as “fluid, floating, atmospheric” and the “weirdness” of a detuned overtone series highlighted a focus on texture, density and timbral evolution, shifting analytical emphasis from melodic line to spectral colour and timbral morphogenesis. • MU8 – Branching pathways (Narrative as “choose-your-own-adventure”). A good number of case studies explored branching pathways were each decision presented to the performer (e.g. whether to follow, delay, or bypass a cue) spawned a new sonic branch, co-constructing the work’s story in real time. These scores therefore functioned as an interactive narrative where the performer’s interpretive path determined the unfolding storyline. • MU9 – Way-point landmarks. In several works, specific harmonic events, acted as checkpoints that anchored the performer’s mental map of the piece. These waypoints served as reference landmarks in a fluid, non-linear environment, allowing musicians to re-orient themselves, mark progress and decide whether to linger, accelerate, or diverge, thereby structuring the overall narrative flow.

Key Takeaways The Music dimension reminds us that digital musicking can be fundamentally an experience of place. When we foreground the imagined geography of sound, with its terrains, landmarks, physics and embodied feedback, we give educators, composers, performers, technologists and scholars a shared language for describing where the music lives, how it moves and who (human or machine) inhabits it. A number of high-level takeaways emerged from this analysis, they are:

• Pedagogy Teaching can move beyond traditional pitch-reading to nurture spatial listening and embodied mapping. Learners would need exercises that, for example, pair breath, gesture or whole-body motion with sound localisation so they can inhabit the “meta-physical space” that a digital system proposes. By foregrounding the spatial metaphor (MU1) and the safe-exploration zone (MU2), instructors can help students feel that they are navigating an imagined landscape, such as underwater, floating, or weightless, rather than decoding a static notation. This embodied approach turns the classroom into a rehearsal ground for the immersive, journey-like soundscapes that define the Music dimension. • Composition Composers are invited to treat the score as a living terrain rather than a fixed script. Deploying physics engines, procedural animation and real-time sensor streams they can create the physics-driven sprites and dynamic, mutable maps of MU3. Unpredictable emergence (MU4) and sensor-sound coupling (MU5) can inject breath, motion or EEG data directly into timbre, spatialisation and visual feedback, collapsing traditional notions of time, distance and agency. Temporal fluidity (MU6) and spectral richness (MU7) add toporhythmic offsets and micro-tonal textures that make pitch just one colour in a dense timbral palette. By embedding way-point landmarks (MU9) and branching pathways (MU8) into the score, composers give performers reference points and decision trees that sustain an open, “choose-your-own-adventure” narrative while preserving a coherent cartographic structure. • Performance Analysis • Analytical practice can expand beyond pitch-centric metrics and notation to capture the four core phenomena that define the Music dimension:

1. Environment (immersive metaphor),
2. Map (mutable visual world),
3. Meta-physical space (fusion of sound, gesture and technology) and
4. Affective narrative (texture, micro-tonality, control-surrender tension). Spectral analysis, gesture-tracking, interaction logs and physiological recordings could become essential tools for tracing how performers negotiate agency, respond to emergent symbols and construct mental maps. These multimodal data make the MU-motifs (MU1-MU9) observable in situ, revealing how a single harmonic event (such as the sound of a G maj7 chord) simultaneously serves as an environment, a mutable map object, an embodied techno-feedback loop and an affective narrative node. • Human-Computer Interaction BCI, motion-capture and haptic devices could be treated as musical parameters rather than peripheral controllers. Designing feedback loops that render breath, movement or brainwaves audible (MU5) and that affect spatialisation, timbre or visual representation brings the performer’s body into the meta-physical space. Such designs actualise the physics-driven sprites (MU3) and temporal fluidity (MU6), ensuring that the interface is not merely reactive but co-creative, letting the performer and the system continuously reshape the living score. • Aesthetic Research Researchers can adopt the nine MU-motifs and the four phenomena as a unified analytic vocabulary that bridges musicology, media studies and cognitive science. Immersion versus mutability, embodied techno-feedback, fluid timbral texture, agency tension and narrative branching becomes lenses through which digital music works are interrogated. By mapping these lenses onto specific works scholars can systematically trace how the imagined geography of sound drives the other six PACMMAN dimensions.