Key Insights

• XR (extended reality) is a combination of augmented reality, virtual reality, and mixed reality as well as immersive technology like projection mapping, motion capture, and software, it is interactive and interacts with new media and technology
• Ine became engaged with XR because of the pandemic, wanting to engage with new media formats for remote performances
• her normal playing has always been very physical and embodied thus she finds the transition to XR space a natural extension
• the extra bodily parameter in working with XR makes Ine more aware of her physical playing
• the audience becomes part of the VR spaces much more than in a traditional concert performance; this work transcends the conventions of musical performance
• it is good to be aware of different software and hardware, and how they communicate and function when working with partners
• finds the conservatories are still very conservative, not embracing innovation or being curious about extending their programs to incorporate more technology
• the music is not the whole concept anymore in VR, it is adding to the whole experience
• the key difference about performing in VR spaces is the embodiment of this process, the whole body will be shaping the performance not just the sound
• playing and making work in VR, makes you more conscious of every step in this process
• we have to research these new formats that have nothing to do with our current physical concert tradition to put the audience into the driving seat of these experiences
• there should be more support for this and also a place in education to address the need for all these new experiences
• music education has to be much more inclusive of people with limited physical and mental abilities, and music education has to be more open for neurodivergent folks

Summary

Ine Vanoeveren is a classically trained flautist and specialist in contemporary music. She became engaged with XR during the pandemic, wanting to engage with new media formats for remote performances. Most of her performances now are virtual digital environments made in game engines. Her acoustic playing has always been very physical and embodied; thus, she finds her transition to XR spaces to be a natural extension of her playing.  The movements for body tracking in VR need to be exaggerated when an avatar is created, and musical expression is adopted towards this movement. These extra bodily parameters make Ine more aware of her physical playing.

The audience becomes part of the VR performance space much more than in a traditional concert, transcending the conventions of a musical performance. We are at the beginning of what can be possible in virtual reality, and there is still a lot to explore. By collaborating on AI and virtual reality projects, you can put questions to the hierarchies implicit in music education and performance that are still very elitist.

Teamwork and collaboration are very important in Ine’s projects. In addition, she finds it is important for performers to have some digital skills and to be aware of different software and hardware, to know how they communicate and function. Performing music in VR spaces is much more interdisciplinary and involves using your whole body and not just the instrument. It is difficult to communicate this to conservatories. Based on her experience, the conservatories are still very conservative, not embracing innovation or being curious about extending their programs to incorporate more technology. The music is not the whole concept anymore in VR, it is adding to the experience. Besides programming existing repertoire in VR like Ferneyhough’s Unity Capsule, Ine is also working in collaboration with composers to make pieces specifically aimed at being experienced in VR. She is currently working with a team of programmers and creative coders where they start exploring an idea together from scratch, moving to visual representation and prototyping of the idea, and later to the development to test individual parameters. 

She thinks that playing and making work in VR, makes you more conscious of every step of being inside this process. Also, seeing and experiencing the work once it is done adds to your perception of the work and allows you to take new knowledge of it. The musicians become dedicated to searching with their instruments, similar to contemporary music but now also with the whole body while simultaneously looking for the boundaries of their playing. It is well beyond the technical skills that one learns in the conservatories.

Currently, Ine is working on a project that uses an AI-based app called Figment – it is a software custom developed for artists for time-based disciplines, it uses smaller banks of training data, specific to the project; she is experimenting with this app for a project with flute multiphonics and haptic feedback. The musicians experimenting with it will get a sensation back through the haptic sensors which multiphonic fingerings to play next, it is also using the whole body, so they are thinking currently which other sensory feedback to add to suggest to musicians how their body posture should be in the moment of playing a particular multiphonic.

Ine finds researching new experiences for new remote formats and hybrid art formats much more interesting than the idea of creating an event that can be consumed virtually without the physical experience. Ine believes that we have to research these new formats that have nothing to do with our current physical concert tradition to put the audience into the driving seat of these experiences. It is a similar principle from games for people to decide on their trajectory and how they want to perceive “the game” or the performance experienced in VR. She believes there should be more support for working with technology and a place in education to address the need for new experiences in VR. Also, music education has to be much more inclusive of people with limited physical and mental needs in this research.