A collection of digital scores by Andrew Hugill in collaboration with Anya Ustaszewski, Elisabeth Winklander and Simon Allen
Digital Syzygies score/brainwave app code
https://github.com/DigiScore/digital_syzygies
Complete Dataset
https://figshare.com/articles/dataset/Digital_Syzygies/21429138
Tracks produced
http://www.digitalsyzygies.org.uk/listen.html
About Digital Syzygies
Digital Syzygiesis a collection of digital scores facilitated by the Emotiv Insight EEG Headset Brain Interface to connect four musicians who were remotely located (in Sweden, Sri Lanka/London, Brighton and Leicester). This technology was defined by Andrew Hugill and his team as a novel solution with which to bind the quartet who are either autistic or are d/Deaf (Hugill is both), and as an inclusive mechanism with which to support a shared creative pursuit. Over 6 months of experimentation, ongoing concepts and theoretical frameworks developed by the DigiScore project were used to develop a set of neural-controlled musical compositions.
To facilitate the translation of raw brainwaves from the EEG headset into the Digital Syzygies compositions DigiScore’s principal investigator Prof Craig Vear developed a software environment that provided a creative platform for the Digital Syzygies team to explore. It linked the brainwave output from the EEG headset to their defined aesthetic design and supported the further development of their creative ideas. This was Beta-tested by Hugill and Vear and eventually packaged as a Python library and as a standalone app (see above).
This Brainwave-to-Music app allowed the brains of the musicians to dictate the course of musical events, subject to responses measured by six EEG performance metrics: engagement, excitement, focus, interest, relaxation, and stress. It also allowed individual musicians to ‘plug’ into a composition using the brain reader and in real-time control libraries of self-created sound files and visual material into the music. This was recorded and shared amongst the team and released as an album. Future planning is in place for a presentation in a live performance environment with an instrumentalist reacting to the soundscape and visual material.
The project avoids the traditional distinctions between “composer” and “performer”. Each participant is simply a musician who is involved in the creation of the score and its realisation. However, the project was initiated and led by Andrew Hugill, who was responsible for overseeing and coordinating all the musical and technical activities.
Digital Syzygies is about unexpected alignments that occur as the result of autistic and/or d/Deaf musicians connected to explore musical relationships, commonalities and differences in a state of ever-changing flow. The Digital Syzygies digital score required musicians to navigate a collection of sound material using their brains. The way they responded to sounds indicated the classification of those sounds. Those classifications were later navigated by Andrew Hugill. The sounding result was the combination of these navigations and has been recorded as:
01. Andrew’s world
02. Anya’s world
03. Elisabeth’s world
04. Simon’s world
05. Syzygy 1: Andrew and Anya
06. Syzygy 2: Anya and Elisabeth
07. Syzygy 3: Elisabeth and Simon
08. Syzygy 4: Simon and Andrew
09. Syzygy 5: Andrew and Elisabeth
10. Syzygy 6: Anya and Simon
11. Syzygy 7: Andrew, Anya and Elisabeth
12. Syzygy 8: Anya, Elisabeth and Simon
13. Syzygy 9: Elisabeth, Simon and Andrew
14. Syzygy 10: Simon, Andrew and Anya
15. Syzygy 11: Full quartet
16. Syzygy 12: Full quartet
Critical questions
Some questions that guided this research:
- can a Digital Score provide a platform in which a quartet of d/Deaf and autistic musicians could work creativity and challenge the traditional composer-performer model towards a way of working that was more inclusive?
- can a Digital Score be used to find musical connections and differences for musicians with invisible disabilities?
The Process
The creative process ran as follows:
- Andrew Hugill provided musicians with an initial digital score to begin the process of realisation.
- Each musician worked with the headsets and the instructions to realise the score, bringing their sound world to bear and providing the resulting data stream to the project.
- A series of sessions were initiated focused on refining and developing the realisation, concentrating on establishing connections between the musicians.
- Andrew Hugill used his performance metrics in the EEG player combining different sound worlds of each musician, as duos, trios, and quartets into a series of pieces.
Critical Insights from Data Set Analysis
Connectivity to the Digital Score
- There was a real dialogue taking place between the musicians supported by an Emotiv brainwave reader acting as a digital score by facilitating a sharing and mutual understanding between the musicians
- The digital score as a brainwave reader could still be seen as a blunt tool designed with neurotypical brains in mind as many of the states it registered were sometimes different from what the musicians were feeling
- Despite this and the fact that all of the musicians found the digital score challenging to use, they also found it useful and liberating
- Andrew learned to embrace the arbitrariness of some of the digital score’s behaviour as a simple way of letting his brain select different classification parameters independent of his choosing
- There is a wealth of possibilities for how the streaming data from the digital scores (EEG readers could be represented) as real-time graphs, 3D brain models or interactive recordings made while playing
- In the future, the digital score can be realised by one or more musicians in a live performance
- Performance metrics were used as a guide to sound classification that triggered 6 of the parameters of the musicians’ brains: focus, engagement, interest, excitement, stress, relaxation
- These performance metrics could be seen as constraints, however, musicians considered them creative because they led to productive results
- It remains to be investigated how much memory of a certain sound could also be used to trigger performance metrics, however, for the hearing-deficient folks, memories of certain sounds as well as their partial hearing of them also triggered performance metrics of the digital scores
- There was personal significance as to the sounds chosen by each musician to be representative of their sound worlds; Andrew learned a lot about each participant from their chosen sounds and their classifications when he was in the process of combining them as duos, trios, and quartets
- The visual side of the images used with the digital score could still be explored, in the semi-structured interview, we learned that Elisabeth discovered how strongly she reacts to colour in interpreting image-based scores made by Simon
Flow
- Going with the flow and letting the brain do the job of selecting sounds was one of the strategies for navigating the digital score for Andrew
- In this way, Andrew is an involuntary agent in the score who is letting the navigation of the digital score happen by letting go of the control of the digital score and letting his brain do the work
- He finds that the flow state in being autistic is normal, that is autistic people already are in an overstimulated state and adding to this state usually results in creating more difficulty for them, whereas using the brainwave reader score assumes their normal state to navigate the digital score to achieve interesting musical results
- Thus, once the flow state is engaged with the brainwave reader digital score, the immersion becomes total
- Everyone had a different strategy for navigating the digital score, Anya used mindfulness and focused on a specific parameter of the digital score to trigger that part of her brain
- Others found zooming in on one task, closing their eyes or even making swimming motions with their body as an effective way of navigating different performance parameters
Digital musicianship
- Each performer was simply a musician involved in the creation of the score and its realisation, there were no hierarchies of performer/composer in this collaboration
- All the roles of performer/composer and also audience became one and the distinction between them was blurred in this project
- There was no top-down approach, everyone was involved on a similar level
- There is a lot of possibility of expansion of the digital score mediated by the brainwave reader to include different brain types beyond neurotypical
- All the participants gained new musical skills in this collaboration which built on their previous musical skills, Anya found the digital score to be useful as a subconscious sound arranger tool whereas for Andrew it was a completely new skill that allowed his brain to select sounds even without him hearing them
Transformations
- The digital score allowed autistic and d/Deaf musicians to work directly with this technology, to express themselves in ways that bypass conventional barriers
- In terms of accessibility, making music with this type of digital score removes bars to participation because one does not have to necessarily be a trained musician or have the ability to read the Western type of notation
- People with limited mobility can also benefit from making music with the brainwave reader digital score
- For Anya, she was able to see what future research developments could be had with this digital score. For example, how would people with different neurodivergent conditions react and express themselves to music with this type of digital score?
- Some of the musicians in the group have synaesthesia, thus some of the future developments could include visual scores and even videos as a way of working with different performance parameters
- For Elisabeth, she had a significant new experience in the digital score collaboration which enable her to interpret image-based scores made by Simon as a compliment to the digital score sounds made with the brainwave reader
- Andrew was able to perform for the first time in many years with this digital score because of his previous hearing problems which did not allow him to listen to sounds for prolonged periods
- For Andrew, the effect of working with the digital score was transformational and highly impactful; he is now engaged in unlearning everything that he has previously learned about making music because making music with this type of digital score accommodates his current autistic and deaf condition
Impact
For more insight into the impact of this project please see:
Personnel