How to use the cards
There are two main ways to use the cards: Free Play, or Idea Game. Of course, they can be used in any way that you feel is right.
Once you have an idea for a digital score, or are in the process of developing one, you can choose any card at random and use that for an open conversation. You can pick any number of cards from anywhere in the pack or read them all.
The Idea Game has been designed to guide you through the design, development and testing of a digital score. There are several phases that lead you and your team through digital score creation. The most important thing is that you arrive at a digital score concept and test it out as soon as possible. Digital scores package musical ideas, but the point of them is for those packaged ideas to be realised by other musicians. Prioritising body-storming over brainstorming is key, as the only true place to evaluate your digital score is by turning it into music.
We would love to hear about your ideas and how they develop through using these creativity cards. Whenever you see INSTAGRAM in the instructions, please:
- Take a photo of the cards you are currently in conversation with (usually 3).
- Write a short note about the group discussion and how your digital score has developed (maximum 50 words)
- Post it to Instagram and tag @digiscoreERC and #creativitycards
Phase 1 - idea generation
- 1.1 Write down a couple of imaginative ideas for a digital score or a musical composition. Artistic vision stuff. What is your musical idea about? What bit of culture/ humanity/ news/ life are you wishing to make music about? (10 minutes)
For each of these ideas in turn:
1.2 Take 3 opportunity cards (+) and spend 5 minutes (time it), exploring how each of the cards would change, enhance, transform, expand, deform, or diminish your idea. (See Open conversations with the cards above).
1.3 Take another 3 opportunity cards (+) and repeat the 5-minute open conversation.
1.4 One last round of 3 opportunity cards (+) and repeat the 5-minute open conversation.
1.5 Move on to the next idea.
Phase 2 - idea development
2.1 choose one of your ideas to explore further.
2.2 Draw 2 challenge cards (-) and 2 question cards (?). Spend 3 minutes exploring how your idea transforms because of these cards.
2.3 Repeat 2 more times with chosen idea (another 2+2 cards, and 5 minutes)
Phase 3 - idea realization
3.1 Discuss the core of your digital score (10-15 minutes) As a group discuss what the core experience is with your digital score (see examples below). This is to be the focus of your experiments. The aim is to get to the TRUTH of the experience of your digital score idea. This TRUTH is the undeniable experience of your digital score and the foundation of what is being communicated. Be prepared to be surprised by this – what you “think” is being communicated might not be in practice.
3.2 Make a rapid prototype version of your digital score (60-90 minutes) This requires you to be amazingly creative and inventive. The priority is to turn your digital score into music, and in doing so find the TRUTH of the experience by testing out the core communications system of your digital score.
Obviously, given the timescale, you cannot develop a new AI, or design the specific game environment in Unity, so you need to use rapid prototyping and Wizard of Oz’ing to realise your idea into a quick and dirty working model (explained below). Put your musician(s) into it (don’t worry about the simplicity of the experiment - tell them your vision and let them imagine their way through it).
Invent imaginative ways to represent your idea quickly, and effectively that are also fun and playful. For example, you could use the video function on your phone to quickly make an animated score with screenshots, or post-it notes; you could build a small-world model of your games world from toys and use a WhatsApp video call to fly through this world; you could pretend to use an AI to generate text and images by having someone type stuff and screen share via Zoom in the next room, or control parameters of a VST sound processor (like the Wizard of Oz in the film). The priority is USER EXPERIENCE… what is their EXPERIENCE of your digital score? Was it what you expected? What exciting new stuff emerged THROUGH this experience?
Phase 4 - idea refinement
Using the feedback from your musician’s EXPERIENCE, develop your digital score to the next level. And quickly get the musician BACK INTO IT. You want them to share their experience with you QUICKLY AND OFTEN. This is the Agile way https://en.wikipedia.org/wiki/Agile_software_development
Examples of Rapid Prototyping Digital Scores: getting to the core of the experience.
Example 1: small world game design
Musicians X, Y and Z have designed a digital score that is based in a computer game world. It is an environmental-focused score where musicians need to collect birds by playing melodies of bird songs. Once collected these birds are then ‘saved’. The composition ends when enough birds have been saved to rebalance the ecosystem.
For their rapid prototype, they built a small world using LEGO. This world looked a little like a jungle (one of the scenes in their score). They had written out melodic lines on paper manuscripts and cut them into small chunks. They made small birds out of 3 LEGO blocks; each bird a different colour and attached the manuscript extracts with sticky tape. This world was placed in a cardboard box, with the front missing, and quickly painted to look like the sky.
Within 60 minutes of doing this, they were ready for an external musician (soprano sax) to try the rapid prototype. The saxophonist was told what the LEGO world was meant to represent, and the goal of the score along with the initial performers’ instructions. The musician sat in front of a large TV screen in a different part of the room from the LEGO world; the small world was transmitted to the screen using WhatsApp video. Musicians X and Y would move the birds around the small world while Z moved the phone around like the camera function in a computer game. They were listening and responding to the saxophonist’s sounds.
This small-world experiment of the digital score was chosen because its core experience was being communicated through the interactivity between the saxophonist sound and the game world. If this aspect of the digital score design did not work or was substandard musicians X, Y and Z would never know the core communications functions of the score.
The demo lasted 6 minutes. Immediately following the demo, the team got together, and the saxophonist explained how it felt to be playing inside the score, what they expected to be happening, and surprising areas they liked. The team discussed the next steps, and very quickly made amendments and enhancements, made a second small world (a seascape) and added different animals as requested by the saxophonist. The saxophonist also wanted to have the camera movement linked to her saxophone, so she was in control of the live image.
After a couple more sessions, musicians X, Y and Z had deep insights into how the digital score might work when it is made in a Unity games engine. At the same time, because of their hands-on experience of moving the birds around and listening to the musician’s live performances, they were able to programme the non-player-character aspects of the game world with ease and accuracy.
This process enabled musicians X, Y and Z to get quickly to the core experience of this digital score, and because of this to build quickly and effectively the digital score proper.
Example 2: animated score
Musician A used the cards and designed a digital score as a fixed animated film. It was to deconstruct a violin and build a collage of ‘materials’ pertaining to a violin such as wood, metal string, pegs and violin music. The fixed animated film was to journey across these materials laid out in a sequence, for example, a piece of string leading to a bit of music manuscript from Mozart, leading to pieces of wood. Musician A, a violinist herself, would use these elements to “reconstruct” the violin through improvised sound.
The core experience with this digital score was to be felt in the balance between how the abstract and musical materials were presented on the screen, and the tempo of the journey in the collage. A rapid prototype experiment was set up that used bits of stuff that were lying around the classroom that resembled the materials of a violin. These were laid out in a journey, and a quick video made with her phone. Following the first test, a new journey was laid out and recorded, then another.
Through these early experiments, core experiences of balance, tempo, complexity of materials and assemblage were determined, and a final version was made relatively quickly once the proper materials were found.
Example 3: AI ensemble
Musicians R, S and T designed a digital score using AI, which would listen to a live musician playing a fixed melodic line and build a unique backing track for them based on their emotional mood. The AI would generate chords by analysing the live musicians playing in realtime and measure their arousal through on-body sensors (EDA). Together the AI and the live musician would make the music in the here and now, with each performance of the digital score creating a different version of the piece but the melodic line remaining fixed.
The core experience would be how the AI measured the live musician and made a response in realtime. This would be felt as new and engaging, perhaps like a duet. Musician R would perform the melodic line on guitar with musicians S and T operating the AI.
To kickstart the experiment the musicians composed a short melodic line and built a quick harmonic matrix that would work with it. The rapid prototype experiment used a Wizard of Oz approach for the AI. This involved musicians S and T pretending to be the AI in another room (like the Wizard of Oz in the film). Musician S would watch musician R through a live video feed and make judgments about their arousal state, signalling this to musician T by lifting or lowering his arm. Musician T would then play chords using a found keyboard sound in Garageband and referencing the harmonic matrix as a guide to what he was playing. This was then broadcast back onto musician R over the video feed.
After one experiment the trio discussed the core experience. Decisions were made about extending the melodic line and building rules for how the arousal measurement could be translated to the harmonic matrix. Extra AI control qualities were also introduced.
Over a 3-hour process, the musicians felt that they understood the key behaviours of the AI and were then able to spend the lengthy process of building it in Python knowing that it would work. They also knew what material worked for the melodic line, and harmonic matrix.