top of page
Sound Augmented Activities
28 April - 21 June
Design Breif
Design a sonic augmentation to the activities of drawing, canoeing, or baking.
PAGE TOP
In Partnership with
BBC Sounds is where you can listen to BBC Radio stations, programmes, podcasts, music and a whole bunch of other lovely stuff online. With over 80,000 hours of BBC audio available, BBC Sounds introduces you to new audio you may not otherwise have discovered.
Team
Introduction
With the rise of mobile phone ownership, advances in headphone technology, the rise of apps and virtual assistants, more people enhance activities with sound. How can activities like drawing, canoeing and baking become more personal, contextual and experiential with sound augmentation (See Fig 1)?
Fig 1: Activity Enhancement
Week 1 / Week 7
Design direction
What does drawing, canoeing and sound mean to people and where can BBC intervene to make it more meaningful?
Thinking micro and macro
Canoeing
During literature rivew, we found that canoe as an actor in the activity of canoeing is submerged in political, economical, capitalistic and cultural bindings of society (See fig 2). It has been seen as the voice of indigeneous people (Silver et al.), the saviour of lives during nature's terror (Monbiot et al.), the driver of mental peacefulness (“Canoeing and Mindfulness Retreat, Devon”) and the gateway of escape into adventures of water. Besides it's macro values it constitutes of micro felt experiences of disblance, wind, sounds, pain and so on.
Initial telephonic directive story telling conducted by Mita, Malavika, me and Kesha unrevelaed that every one had their own precious stories of canoeing which made the overall experience valuable. It being unbalanced, explorative, navigatory, adventurous, fearful and personal is seen as a escape to another world away from their daily life.
Fig 2: Canoeing as a system
Drawing
During literature rivew, drawing as an activity was also seen submerged in political, economical, capitalistic and cultural bindings of society (see fig 3). It is used as a metaphor of social sepaeration (Walzer, M., 1998), the fundamental element of capitalist design (Menger, P.M., 2014.), the tool of opperesion and suppresion in memes of social media (Kanai, A., 2016) and a medium of expressing history and precious moments of time (Petrie, W.M.F., 1889).
Fig 3: Drawing as a system
Sound in context of drawing & canoeing
Quick telephonic conversations with a few known canoeists reflected on caneoing as a journey of sound in time and space. Chirping of birds, movement of wind, displacement of water, increasing and fading sound of the city, party and date with music. When I asked them how would a silent canoeing experience feel like they compared the experience to salt less food and leafless tree showing the importance of sound within the journey.
In the talk with a few art and design students, several participants reflected on listinging to music and drawing. The ambient sound already present in the emvironment was not a prime focus until it emerges strongly. Infact my visit to "Our time on earth" exhibition organized at barbican center gave me a good understanding on using spatial sound and audio first experiences in an exhibition setting. I personally loved noise aquarium unrevealing the sounds of unheard micro-organisms under the sea.
Fig 4: Our time on earth exhibition
Value of BBC
BBC sounds as an organization offers music, podcast and radio. On conductiong value analysis, we found that at storytelling, personalization of content, diversity of choice and audio immersion are the core of their experience. Interestingly they were not limited with their sounds application and were constantly engaged in experimental work (BBC Taster).
.png)
Fig 5: BBC Sounds value analysis
WEEK 1
Inspiration and direction
People have been creating audio-visual experiences around canoeing for mental health, games and storytelling. While the idea of using music to add a personalized layer to the ambience of canoeing was interesting but I was more intrigued the opportunity to enhance the context situated human-environment relationship with sound. In case of drawing I was yet not sure where should we intervene but the idea of drawing with augmented sounds positioned at different places within space felt intriguing to me.
.png)
.png)
Fig 6: Canoeing Ideation
What are we thinking about?
Raising environmental awareness while canoing.
Facilitating ambience personalization
Aiding mental peacefullness with sound augmented canoing.
Drawing with the sound as a connection between specially abled and the abled society
Enabling cultural connect unfolding in time and space.
Designing for the fear of upside down or disbalance on canoe.
Week 2 / Week 7
Design direction
What and how do we situate our story telling experience within a specific canoeing place and moment? What and where do we position ambient / reactive sounds within the activity of drawing?
Canoeing contexual research
Our conversations with people made us realize that canoeing is highly dependent on the environment and context. We completely were aligned with this statement after our observation and canoeing experience at paddington and Richmond. We leaned more towards Richmond as a target place due to nature submerged experience and a relatively higher frequency of canoers.
Observing canoeing in Paddington
High rising buildings touching the sky, delicious smell from commercial restaruants reaching peoples nose, the glaze of alcohol in the boat bars. Paddington area is a canal where canoeing facilitated by "Active 360". The experience is compact, commercial to resedential in space-time and is filled with indsutrial and capitalist boats all around. More than canoers we exhibited people enjoying there meals on boat rides.
Experiencing canoeing in Richmond
Green leaves, silent water, swimming ducks, hefty wind and co-paddlers around. Canoeing in Richmond facilitated by "Back of Beyond Adventures" was a nature submerged experience, way different than paddington. We decided to split as a team Mita and Kisha were interviewing canoers and me with Malvika were canoing for 3 hours.
Handpicked interview responses
Meriel Resenkrana - Assistant at back of beyound
Why do you canoe?
Happy being in the water, forget that they’re in London.
What do you like the most?
Meditation in Motion class on the padddleboard, concentrating on all of their senses.
Who comes for this?
People come for, bachelorette party, corporate events, dates, competitions, sight seeing.
Pierce Osborne - Owner at back of beyound
Why do you canoe?
Peace and tranquility and
meditation: Cant do anything else when canoeing
What do others come for?
Others like the thrill, fun and peace of canoeing together.
What's challenging for you?
Finding new environment, looking at different abilities in group, pushing people to limits
What do you think new people might need in the water?
Navigation: Maps, charts, planning in advanced, compass, GPS, radio
What do others come for?
Others like the thrill, fun and peace of canoeing together.
What's challenging for you?
Finding new environment, looking at different abilities in group, improving communication, pushing people
Tim Gold - Canoer at Richmond Canoe Club
Why do you canoe?
Likes the meditation, sounds of birds, water, boat noise against water, NATURE AND BEING OUTDOORS AWAY FROM TECH, WHY HE GOES PADDLING
What do others come for?
Listen to BBC sounds on phone, when driving, 5 hour journey but not while canoeing
What's special in BBC sounds for you?
DIVERSE, EDUCATIONAL, ENTERTAINING
Analyizing our personal canoeing experience
We categorized our 3 hours of canoeing journey in four segments scared, excited, tired, anxious. The categorization was more relvent to a newer millenial audience canoeing for long time. On a telephonic review, canoeists who were not frequent said they had similar joueny patterns but it changed over time. Infact participants who had hydrophobia were more stressed in the joueney. I personaly felt any of these categories could be a great place to intervene and enhance the experience with sound augmentation.
Fig 7: Response to why students listen to music while drawing
Drawing contexual research
Informed by conversations, we made our hypothesis of drawing being highly dependent on your senses, canvas and tools. The question was where should we intervene and how? Our sensory drawing activity inspired us to percive sound though other senses and our questions to LCC and CSM students brought interesting reasons for listen to music while drawing. Mita was more intrested in augmenting the music that they hear ambiently while I personally was more interested in augmenting the sound generated by the tools, canvas and their personal movements while drawing.
Sensory activity of drawing
Drawing is seen and shared as visual first experience. We wondered if it could become a felt shared experience amongst comminity. 3 of us with another participant tried drawing on each others back based on the felt experience of pressure and motion. Our drawings were inspired by sounds around us and changed drastically with felt experience of each other. Interestingly the particpant saw this drawing activity as a new way of building trust amongst community of art and drawing.
What and why do students listen while drawing?
Mita, Malvika and Kisha designed and put posters around the university asking "what and why do you listent to while drawing" gave us focus, ideas, calmness, ambience, vibes, imagination, companionship and clear mind as 8 broad reasons for musical augmentation of drawing. We used affinity mapping as the method to form 5 categories and intervention points for sound to enhance the activity.
Visiting ASMR exhibition
Mita visited ASMR exhibiton in search of sound stimulated sensations that might be useful to unreveal hidden sounds in our activities. Although it was a great experience, we collectively thought it was not very useful in the context of canoeing and drawing.
WEEK 2
Visualizing Ideas
What are we thinking in canoeing now?
With stories as the common value possesion for both BBC sounds and Canoeing I was seeing canoeing as a medium to navigate through the story of nature and people. Specially at the point when people et tired of paddeling it could be a great place to intervene with such audio scapes. The muddy color of thames river conceals what's inside and we were able to see curiosity in people to find out what's under the water. But the curiosity didin't came alone, it tagged along with fear of disbalance and drowning. To me nature was both under the water and over the water, while in conversation with people I saw dominance over the water experiences than under. Although it was the deliver mechanism of audio was unclear at this point yet I felt it could be a potential usecase.
Design an immersive soundscape experience spread in time and space that tells underwater stories without words.
We still had to answer several questions within this idea. The biggest challenge I felt was delivery mechanism and time. People are busy paddeling the canoeing which is the core job to be done to move in space and time. So the audio delivery needs to be seamles without distrubing the canoeing experience.
Art and design students are alreay using music as an ambient element to their enviornment for various reasons. Can BBC sounds have a playlist specificaly curated for drawing students. What's interesting here was to see how can we create a universal playlist for multicultural audience published by BBC as a section on their application or deliverd by an AI or radio program.
Design and curate a drawing playlist and think about the delivery channel.
What are we thinking in drawing now?
Week 3 / Week 7
Design direction
Analyze and define micro aduience and intervention points for canoeing and drawing. Design a low fidelity experience to quickly test the ideas with the particpants.
Planning
It was becoming hard for us to focus on both the ideas so we decided to split into two teams exploring and testing drawing and canoeing based on interest and see if both can be combined at a point. Me and moubani took the charge for designing and testing the ideas of canoeing while Malvika, Mita and Kisha were testing drawing.
Activities to test our hypothesis for drawing & canoeing
Reflections by canoeists on underwater sounds
Hypothesis: Hearing ambient underwater sounds during canoeing can help people make better relationship with nature.
I sent a under water audio downloaded from BBC Sounds archive to 3 participants on whatsapp and asked them if they would like to listen to these sounds in addtion to other nature sounds while canoeing. While we were waiting for their response we went to sound lab to get a hydrophone. Sound technitian in the lab said there is a high probablity you will end up recodring noise more than underwater sounds. Even though there was uncertainity I got the hydrophones to test. This soundscape could be part of BBC's location based experiences within there application. It could be scaled in terms of themes, places and context around the globe.
Fig: Testing the hydrophone with H4N recorder
In the ongoing research I found that BBC did a similar VR experience of diving into thames with Zoological Society of London to make people expereince the under thames life which is presumed to be a dead place due to it's muddy colour. I also found an artists, Jana Winderen, exploring under sea sounds and creating art installations to raise awareness and sympathy towards under water animals. Looking at these examples helped me gain confidence but we still had to make sure if something like this will be a good experience in the context of canoeing.
Unfortunately only 1 participant responded back saying "he will like to hear such underwater sounds if he is alone. If he goes in a group canoeing becomes more about conversations. Although it could be present as an ambient sound with other nature sounds."
Wizard of Oz activity for drawing
Hypothesis: Voice AI could be a great interface to deliver personalized audio cues and songs.
On the other hand, Mita, Malvika and Kisha were exploring if audio only AI could be a good medium to deliver personalized music to artists and drawers. They conducted a small activity with a participant where Malvika and Mita role played AI assistant turn by turn giving audio cues to help the participant generate ideas for drawing and play music accordingly. At the end of the activity particpant said the experience was too driective where as drawing in general is explorative in nature.
Corse correction with learnings
Canoeing
Canoeing majorly came as a group activity in the responses from our participants. Underwater sounds didin't make much impact in the way I thought it would. To accomodate our learning that people go in pairs or group and they talk even if they are experiencing the nature ambiently we decided to go back and analyze our resarch to recenter our focus. During our interview process we saw how canoeing was a great spot for couples to spend time with each other. I personaly was intruiged by this thought while moubani was more interested in hyperlocal history of Richmond. We ideated for both the case and came down to two ideas.
We were finding a huge challenge in locating our audience due to off season and bad weather. Since the experience has to be in the context and with people we collectively decided to merge with drawing activity again and put a hold on canoeing.
Drawing
Early in the morning Mita and I attended Samsung's Creative Shift event which provides platform to emerging creative poeple for showcasing there work. A musician and a photo journalist presented there work. "The unheard stories inspire my music" said by the musician in our conversation with him. Later, Mita created a LCC drawing playlist from the responses we recived in our research. Mita, Kisha, Malavika and I bought a long paper roll and sat in CSM exploring drawing with different materials on a long scroll while listing to the playlist. Mita gave a brilliant idea where the scroll could become our research and experience. While drawing with the playlist we analyzed how music was enabling inspiration and was affecting our pace.
The micro interaction of two materials caught my attention. The sound generated when the tool is rubbed in the canvas changed drastically as we switched material. Now was the time to observe how our playlist changes people's behaviour. Luckily we found some really cute kids who were drawing fearlessly on our canvas dancing on the music. Kids were rolling all over the paper, sketching on the walls and dancing on the music. Their drawings were more colourful flowing all over the paper while ours was limited with color and space.
Although kids were not our target audience but personaly for me the activity thrashed my brain to focus on where is the sound situated in the environment and how is it getting generated? I got interested in three following ways of sound augmentation in the activity of drawing...
Pre-recorded ambient sound plyed on headphones or speaker
Sound generated and augmented with body movement
Sound generated and augmented through matrial friction
WEEK 3
WEEK 4
Week 4 / Week 7
Design direction
Conduct a workshop with art and design students drawing with ambient sound. Plan the delivery of mid term presentation.
Strategic Resource Planing
We collectively decided to split work amongs us. I focused on delivering personalized audio only presentation while Mita, Kisha, Malvika and Moubani were facilitating the workshop.
Workshop
Aim of the workshop: To understand how sound influence physical and cognitive behaviour of participants while drawing and wether it influences their outcome.
We asked people to think of there drawing as an organism that evolves as the sound changes. It'll be very free flowing, and there's no wrong way to go about it, so they should not feel intimidated… just draw what comes to mind and have fun on the long roll of paper! Lara helped us to edit the script and add icebreakers to loosen up people. We provided a range of materials from Charcoal, Stamp pad ink, Paint, Sketchpens, Eraser and Oil pastels. we deliberately asked participants to stand and draw allowing them to freely move around.
Research questions:
- Can people visualise sounds through the activity of drawing?
- Can sounds then be prompts to help people find inspiration for drawing?
Questions for participants:
- When you draw, do you usually listen to any music? If yes, what kind of music helps you feel inspired while drawing?
- Did any of the sounds during the workshop help you draw?
- Did any of the sounds during the workshop make it harder to draw?
- Ask them to explain their drawings and why they chose them to match each sound:
- How did you make the shift from audio to visual and then back to audio?
- Did the audio provide any cues for your drawing?
- Did you recognise any of the sounds that were playing?
- Were you more comfortable/enjoying the instrumental or songs?
- That stroke/line/shape looks interesting can you tell me a bit more about it.
- You seemed to be enjoying the track, can you tell me a bit about it ?
Workshop analysis
What people had to say after the workshop...
It createad a very personal space for me attched to old memories.
Sound was ambient and put me into meditative state
Body posture changed based on music
Rhythm of stroke changed based on music
Music made the drawing more unconcious but explorative
Music change the feeling and perception of drawing
The use of colors and strokes changed
It created the urge to use differnet textures
It inspires and then fades in the background
Experimenting drawing with body and sound
Inspired by bodily movements from kids and other participants in our activities, I wanted to explore the harmony of movement and sound creating a sonic drawing. Participants were seen to move there head frequently on music. I wondered what if sound is generated and augmented by their personalized movements occuring while they create drawing with their head. I used p5.js and Tone.js to create a low fidelity prtotype tracking noseto draw and generte sounds using sign wave synthesizer. Although, it was a great way to look at drawing the sound was overpowering drawing which participants did not want in our activities.
Planning audio only presentation
Scroll being our research and experience was planned to put up on the wall which felt like an exhibition. The most crucial part of the presentation was allowing people to experience the scroll in their own space and time. Mita and I wanted particpants to hear the music as they move along the scroll. I visited sound lab to see if there is a solution to create hyperlocal spatial audio but ideas like camera tracking, distance tracking or button was hard to execute in 2 days. I thought we can use arduino and MP3 trigger attched with a capacitive touch button to play aduio but I wasn't able to figure out code quickly. At last we decided to make use of QR code as a audio delivery interface.
Me, Mita and Lara curated the script for the presentation. Later I got an H4N recorder and recorded the script in Faiza's voice. Her voice was deep and builds helps in building emotional connection with the drawing. After multiple rounds we got it at right pace and quality. I edited the audio to create and sonic ambience for the presnetation. Dede on the other hand worked on generating QR codes for the audios.
BBC Sounds Audio Presentation
00:00 / 03:05
Week 4 Presentation
Week 4 Feedback
Go more micro in materiality of drawing and explore.
Explore beyond music and try noise and sounds
Think and plan for technichal redundencies
Annotations should not be on the scroll
Think how is it different from Spotify.
It was collective but still individual with earphones
Week 5/ Week 7
Design direction
Re-ideate and experiment with micro interactions of materials, sounds and voices invorporating the learnings and feedback from partcipants and jury members.
Crazy 8 ideation
Mita came up with this great ideation session called crazy 8 where participants ideate same idea in 8 different ways under a minutes time. This was really helpful in forcing every body in our team to thrash their brains and generate ideas in the essence of time. After carefully reviewing, refining and merging of ideas we all got interested in augmenting the sounds generated from material or canvas while drawing and audio only masterclass ideas. The masterclass idea was inspired by responsive radio (BBC Taster) where personalized stories are created for the length participants would like.
Resource allocation
To efficiently explore and execute ideas we delicated the work to people. I helped in exploring material and bodily sound augmentation experiments. Lara and Kiesha helped us in scripting, recording and building bot the AI masterclass experience. Leon and Moubani helped in collating theme based sound files. Mita and Malvika helped in planning the experience of masterclass.
Body and material sound augmentation
Drawing with sound and body
I experimented with full body tracking in p5.js using Pose Net library and augmented existing sound's amplitude and speed based on the position of the body parts within the canvas.
Drawing with sound augmeted tool
I experimented with full body tracking in p5.js using Pose Net library and augmented existing sound's amplitude and speed based on the position of the body parts within the canvas.
Sound augmentation to palm movement in physical drawing
In our activities we saw people exploring different ways to use a same material. Hence I wondered if we can turn our palm movements while drawing on physical canvas to sound. I used touch designer and leap motion to convert hand movements to basic sawtooth wave sound. Based on your distance from the leap motion Frequency and amplitude of wave changed along the canvas.
Augmenting the canvas with sounds
---
Themes and sound collection for platform
I was imagining these solutions as a digital platform powered by BBC sounds with customizable themes for sound canvas. This experience would be positioned an sonic inspiration board for artists and designers allowing them to think with sound. Although platform could have any no. of themes we restricted ourselves with a few in the essense of time. Moubani and Leon helped us in collecting sound files for our platform.
.png)
Role-playing AI drawing voice assistant
Mita, Kisha, Malavika and me roleplayed AI assitant several times but we felt it was very shallow and directive in nature similar to wizard of oz activity of ours. We also looked at Google's dialog flow for building voice bot at this point.
Digital experiences was fun to play and build but they lacked harmoney. We felt that the sounds were overpowering the drawing experince while our learnings from the activities reflected on the need of ambient background sounds. I was personally facing challenge in generating sounds from tone.js oscillators since I was new to that javascript library. Physical experience of drawing gave freedom to partcipants for personalized creation while digital was limiting the way they used tools in other activites done on scroll.
We decided to explore the physical experiences next week.
What are we thinking now?
WEEK 5
WEEK 6
Week 6/ Week 7
Design direction
Explore ways to bring sound augmentation on the physical canvas. Planning and testing the audio masterclass idea.
Augmenting souds of material
Low fidelity explorations
Fall, rub, push, pull, microinteractions between two materials produces sound using vibrations. Drawing on paper with a pencil has it's own unique sonic identity which is ambiently present without making us much concious. Skin of the world is canvas of drawing and every skin has it's own unique sonic identity when encountered with other materials. To augment and enhance the sonic experience of drawing on paper we experimented with new layers of skin and new styles of creation. Me Malvika, Mita and Kisha experimented and recorded the sounds and I edited the audio to pass highs of sound more than lows for clear audio.
Digital sounds on physical paper
During the activities I was imagining sound canvas as a theme based sonic augmented toolkit that BBC sounds can sell it to art and design students. It will be positioned as sonic art inspiration board helping them get inspired and generate ideas. Overlaying a skin of digital sounds on paper wasn't a easy job. Either I could use camera tracking to augment the activity or conductive material on paper that witholds sound in it. While conversation with Joanne in CTL we realized camera can be tricky to capture motion if you plan to go large scale. We were interested in drawing with the body as tool, I decided to go with augmenting paper and in turn augmenting sound of it.
I initially started by making a capacitive touch button with foilpaper and arduino. Once I was able to read the touch I thought of using sparkfun mp3 trigger as a medium to play sound files as the capacitance is high on LED pins. The challenge was I did not realize mp3 trigger can only fire one sound at a time and generates a lower quality sound. Other options were to take the pin readings from arduino to max msp or pure data and trigger sounds. Both were fairly new softwares for me and we were running out of time. We decided to get feedbak from particpants on the prototypes we have until now.
Feedback:
- The lines on paper was overpowering playfullness than drawing as a motive.
- The sounds were chopy and they were not played every time we touch.
Experiencing colours through sound
Chromesthisia a phenomenon of listening to a piece of music and seeing colors with every pitch, change in timbre, or different chord progressions (Makhlin, J., 2014). This inspired us to think what if people could percive colours in association with sound. How would that change the way artists and designers draw today? With such questions in mind we thought of creating a physical prototype with colors attached to sounds changing based on the particpants movement, pressure, direction etc. On discussing the idea with CTL team they suggested going phyiscal in a week's time is tough and it's better if we think a digital implimentation of it. Although I was personaly more intrigued by the idea of physicality but I knew that to detect every motion, pressure or direction we will have to use a individual sensor spread out in space.
In the essence of time we decided to build a digital version through HTML, CSS and Java Script where each colour picker function is attached to a mp3 sound file. The song.play() function would only be called if a color is selcted and drawing stroke() is true. It took me struggle and learning of a day to build a basic version. The challnenge was to produce amplitude, frequency and speed variations through out the artboard. I wasn't able to figure out how to use tone.js library that gives acces to Attack, Decay, Sustain and other proerties of sound. Even CTL team wasn't able to help me with this.
I was imagining this experience as a sonic canvas powered by BBC Sounds to inspire artists. Leon helped us create screens for the platform. Participants of Sound Canvas could choose from a range of themes to play with or create there own personalized sonic-colors.
Planning audio masterclass
Live streamed audio masterclass could become forced and annoying if the content is not personalized to partipants needs. Mita, Malavika and Lara used a short quiz to get the essence of particpants taste just before the masterclass. Based on the responses an AI based algorithm curates a playlist of sounds whose metadata is best matches the requirement.
Testing Ideas with art and design participants
We conducted a workshop with 2 participants to test and get feedback. We chose 4 ideas to test
- Sound Canvas: Drawing with sonic colors
- Masterclass: Personalized inspiration and education with sounds
- Body draw: Drawing with sound generated by body movements
Feedback from particpants and tutors
Drawing with colors is a great experience and the playfullenss can help me fight my design mental blocks.
Masterclass and the sonic drawing could be combined and made one experience
Drawing with the body is playful but complex for me to draw with. May be with improvements in brush strokes can help.
Can drawing be a sound first insted of visual first experience
In the masterclass they liked the physical openess and they would have loved a few more educatory prompts.
Can it be a large scale expereince where particpants have freedome of movement
What are we thinking now
Participants are loving the playful sounds and materiality of drawing. It is allowing them to explore and learn in their own way without being limited to harcoded functions in the digital. Paper with a second skin of sound can withold the explorativeness and yet allow them to experience their drawings with the sense of hearing. It was friday by now and we just have a week. I knew that improving physical could bring technology dependent risk but our team was coolectively ok with that which gave us a push.
Designing and improving physical sound canvas for the sonic drawing experience.
Exploring ways to paint and have sound on paper
Scope of work:
- Find a way to hide the conductive lines and electonics.
- Improve upon the quality of audio delivered.
- Play and sustain multiple audios at once for smooth experience.
- Think for scaling this experience.
- Plan how are we delivering this experience.
- How to combine masterclass and sonic drawing?
- Thinking sound first drawing.
Resource allocation
- Malvika and I coupled to build and improve on prototpe
- Mita started to think about branding and delivery of experience.
- Lara started scripting audio cues
Expolring conductive material and ways to augment it on paper
To diminify the strength of conductive material we tried graphite on black paper but unfortunately the lines were still visible clearly. We also tried conductive paint as a conductive material and conductive copper tape. Although conductive paint on balck paper would have worked but to hide the electronics completely I thought of a flap based mechanism where depending on the pressure put on paper flaps with conductive material would touch each other and the circuit will get completed with sound as result. I tested this on a small paper and it worked well with graphite.
Although black conductive paint on black paper was working well it limits participants to use only conductive material and worked best with human touch. Flaps felt to be a better solution until this time. It was giving plain canvas for particpants and freedom of material exploration.
Week 7/ Week 7
Design direction
Scale, build, test, showcase the experience.
Scaling up experience
Initially we were thinking this as a small size toolkit for people to play with. But after the feedback reflecting on a full room experience allowing freedom of body movement and multi participant experience me and Malvika began to protype it big. I decided to use condcutive tape isnted of graphite this time to save time on filling boxes with graphite. After a day's effort it was a flop show and it didin't work.
Issues:
- Contact of conductive tape flaps did not complete the circuit
- It worked only when I touched it with my fingers
Analysis:
- Graphite went into the pores of paper and pressing with finger on the other side completed the circuit in the small scale prototype.
Challenges:
- Monday was gone and we din't have time since presentation was on Thursday.
- Sparkfun MP3 Trigger can fire one sound at a time insted of multiple.
What I wish I would know before hand:
- Poor quality delivery of MP3 trigger
- 1 sound file trigger only at once
- Would have hands on experience on reading arduino data and taking it into Max Msp or pure data.
Guilt:
- Letting the team down without a backup plan
Options in hand:
- Build a smaller toolkit again in the essence of time which again had a risk of technichal redundency.
- Build on masterclass Idea and just show that.
- Show one of the digital canvas experience as an extension to master class experience.
Decision making:
- Lara proposed that we should not show something which won't work at high quality and efficiency.
- Mita and I proposed that we could show digital sound canvas and masterclass
We agreed on showing both and course corrected our journey.
Planning and preparing for the workshop
Resource Allocation
As a design director in her professional life Lara Mendoca was great at facilitating workshop and handeling teams which reflected well during the hard times. Leveraging her skills and Mita's graphic skills they prepared the script, prototype and planned for the masterclass activity. While I had to figure out the code to enable touch based interaction in soundcanvas and deploy it for Ipads. Kisha did a fabulous job overnight in recording and editing audio experience. Leon helped us in generating QR code for sound files. Me, Moubani and Malvika collected Ipads, paints, papers, cameras, and inquired about prints, vinyls and laser cutting machines.
Masterclass Activity steps
- Breif: Draw your Aura
- Quiz: The activity would start with a quiz to generate personaliezed audio content.
- Introduction: Participants are introduced to the BBC app prototype having themes to choose from.
- SoundScape: Particpants listent from there selected theme and playlist with cues at the begining and end.
- Drawing: Participants explore and draw with any material availaible to them while listing to the sound scape
Quiz and theme selection prototype
Mita designed and created the prototype which we tested befor the workshop (Click here to use the prototype).
Preparing the sound scapes
Kiesha Mundin curated the sounds from our earlier research and recorded the sound cues in one of her friends voice.
Sad
00:00 / 11:56
Happy
00:00 / 10:41
Energetic
00:00 / 10:53
Sad
00:00 / 03:05
Prototype made by Mita Chavan
Refining the sound canvas
After the effort of whole night I was able to figure out the indexing and reading method of touch and used it with ontouchstart and ontouchend functions of java script. I added few other functins of delete, undo and downloading a png image of drawing. I was not able to figure out how to tweak the sound based on movement in canvas or download and hear the sound composition after drawing. I deployed the website to firebase hosting so that we can open it as a web application through link on ipads and use the apple pen to draw. I could have done far better with some technichal assitance.
WEEK 7
Week 7/ Week 7
Design direction
Scale, build, test, showcase the experience.
Scaling up experience
Initially we were thinking this as a small size toolkit for people to play with. But after the feedback reflecting on a full room experience allowing freedom of body movement and multi participant experience me and Malvika began to protype it big. I decided to use condcutive tape isnted of graphite this time to save time on filling boxes with graphite. After a day's effort it was a flop show and it didin't work.
Issues:
- Contact of conductive tape flaps did not complete the circuit
- It worked only when I touched it with my fingers
Analysis:
- Graphite went into the pores of paper and pressing with finger on the other side completed the circuit in the small scale prototype.
Challenges:
- Monday was gone and we din't have time since presentation was on Thursday.
- Sparkfun MP3 Trigger can fire one sound at a time insted of multiple.
What I wish I would know before hand:
- Poor quality delivery of MP3 trigger
- 1 sound file trigger only at once
- Would have hands on experience on reading arduino data and taking it into Max Msp or pure data.
Guilt:
- Letting the team down without a backup plan
Options in hand:
- Build a smaller toolkit again in the essence of time which again had a risk of technichal redundency.
- Build on masterclass Idea and just show that.
- Show one of the digital canvas experience as an extension to master class experience.
Decision making:
- Lara proposed that we should not show something which won't work at high quality and efficiency.
- Mita and I proposed that we could show digital sound canvas and masterclass
We agreed on showing both and course corrected our journey.
Planning and preparing for the workshop
Resource Allocation
As a design director in her professional life Lara Mendoca was great at facilitating workshop and handeling teams which reflected well during the hard times. Leveraging her skills and Mita's graphic skills they prepared the script, prototype and planned for the masterclass activity. While I had to figure out the code to enable touch based interaction in soundcanvas and deploy it for Ipads. Kisha did a fabulous job overnight in recording and editing audio experience. Leon helped us in generating QR code for sound files. Me, Moubani and Malvika collected Ipads, paints, papers, cameras, and inquired about prints, vinyls and laser cutting machines.
Masterclass Activity steps
- Breif: Draw your Aura
- Quiz: The activity would start with a quiz to generate personaliezed audio content.
- Introduction: Participants are introduced to the BBC app prototype having themes to choose from.
- SoundScape: Particpants listent from there selected theme and playlist with cues at the begining and end.
- Drawing: Participants explore and draw with any material availaible to them while listing to the sound scape
Quiz and theme selection prototype
Mita designed and created the prototype which we tested befor the workshop (Click here to use the prototype).
Preparing the sound scapes
Kiesha Mundin curated the sounds from our earlier research and recorded the sound cues in one of her friends voice.
Sad
00:00 / 11:56
Happy
00:00 / 10:41
Energetic
00:00 / 10:53
Sad
00:00 / 03:05
Prototype made by Mita Chavan
Refining the sound canvas
After the effort of whole night I was able to figure out the indexing and reading method of touch and used it with ontouchstart and ontouchend functions of java script. I added few other functins of delete, undo and downloading a png image of drawing. I was not able to figure out how to tweak the sound based on movement in canvas or download and hear the sound composition after drawing. I deployed the website to firebase hosting so that we can open it as a web application through link on ipads and use the apple pen to draw. I could have done far better with some technichal assitance.
Facilitatingt the workshop and collecting feedback
We conducted the activity with 5 participants from diverse art and design background. The activity was timed for 20min consisting both Masterclass and Sound Canvas experiences. We offered snacks for the particpants in and at the end we sparked a group discussion to build healthy feedback session.
Sound Canvas Outcome
Feedback from particpants
Latency in the feedback sound made them frustated.
Strokes became more playful with sounds attached to them.
It could be a great exercise to draw mindlessly and break mental blocks.
People would love to add their own sounds to their tools or canvas.
A physicl experience same like masterclass would be fun for exploration.
It sparked imagination in the bran as we draw.
Masterclass Outcome
Feedback from particpants
When particpants had plan in their mind then the limitation of choice made hard to match music and drawing.
It could be a great experience when you draw mindlessly and hear the music of your choice.
Beats of the music affected the way brush was used and the choice of colors.
For particpants music induced various imagination and scenarios.
One of the participant was imagining shapes, sizes and distortions based on music. That's his style of drawing charecters in general.
It helped a participant to concentrate and relax since she gets easily distracted.
A partipant suggested this could be an exercise where people can generate unknown outcomes inspired by sounds.
A partipant suggested to have only instrumental music other wise words can tangle up the thoughts.
Techno beats based music can disturb both mindless and mindfull drawing.
Final Presentation
Feedback from the audience and experts
Sound canvas seems very basic and it could have incorporated dynamic changes like pressure, speed, direction change.
You should have explore sounds apart from music.
Showing analysisand future projection would have been great.
Prsentation could have more context for people who are hearing it for the first time.
Personal Reflection
Future Projection
Both digital and phyaiscl experience have potential to be scaled and made for larger audience. Sound canvas could be pivoted as aduio-visual inspiration board for artists and designer. It can have several customizable playlist carefully curated under themes of drawing. The masterclass could become an official radio streamed program for inspiration and mindless drawing to relive mental block.
Challenges
The project wasn't easy at all. Managing task allocation, aligning people's time and lack of delivery became the biggest challenge through out the project. We switched directions several times which was resulting in several low fidelity solutions more than a high fidelity solution. In the canoeing activity the largest roadblock was finding participants and a canoe to situate our work in real context while in the drawing activity working with bare minimum technical support and knowldge became the bottleneck.
Learning
- Learnt to manage the diversity of thoughts in the room.
- Learnt to work with neurodivergent friends.
- Learnt to self learn technology by trial and error.
- Learn about the physical and cognitive value of sound as an experience design tool.
- Learnt performance based research methods to uncover deeper understanding of context.
Citations
BBC Taster. (n.d.). Responsive Radio. [online] Available at: https://www.bbc.co.uk/taster/pilots/responsive-radio [Accessed 18 Jun. 2022].
“Canoeing and Mindfulness Retreat, Devon.” The Guardian, 27 June 2021, www.theguardian.com/travel/2021/jun/27/canoeing-and-mindfulness-retreat-south-devon-river-dart. Accessed 31 May 2022.
Kanai, A., 2016. Sociality and classification: Reading gender, race, and class in a humorous meme. Social Media+ Society, 2(4), p.2056305116672884.
Makhlin, J., 2014. Chromesthesia as Phenomenon: Emotional Colors.
Monbiot, George, et al. “George Monbiot Canoes across the UK Floods – Video.” The Guardian, 17 Feb. 2014, www.theguardian.com/commentisfree/video/2014/feb/17/george-monbiot-canoes-uk-floods-video. Accessed 31 May 2022.
Menger, P.M., 2014. The economics of creativity: Art and achievement under uncertainty. Harvard University Press.
Petrie, W.M.F., 1889. Historical Scarabs: A Series of Drawings from the Principal Collections Arranged Chronologically. D. Nutt.
Silver, Marc, et al. “The Amazonian Tribespeople Who Sailed down the Seine – Video.” The Guardian, 10 Dec. 2015, www.theguardian.com/environment/video/2015/dec/10/the-amazonian-tribespeople-who-sailed-down-the-seine-video. Accessed 31 May 2022.
Walzer, M., 1998. Drawing the line: Religion and politics. Soziale Welt, pp.295-307.
bottom of page