Exploring with Yarn and Sound for Expressing Personal Data Narratives

How do making and materials contribute to our understanding of personal data representations? To explore how materials, data, and humans collaborate to produce physical data representations, we created a series of artefacts from personal data we collected (about commuting, forgetting, and busy-ness) in different media: yarn and sound.

A commuting mini-quilt, conveying the emotions Mikhaila felt while commuting.
Mikhaila’s commuting yarn artefact: “I wanted to represent segments of my commute (car or bus), the decisions behind them (convenience, price, time), and how I generally felt while commuting (frustrated, anxious, indifferent). Since I had been playing with crocheting different shapes the week prior, I wanted to create both squares and triangles. I originally wanted to create a larger patchwork (to possibly turn into a pillow), but I ran out of time and instead chose one day to transform into the artefact.”

A schematic diagram showing the sonic layers in Jordan's commuting sound artefact.
A schematic diagram showing the sonic layers in Jordan’s commuting sound artefact, which you can listen to here: “For commuting, I was unsure of my abilities in sonification and thus focused on a straightforward mapping between duration data and the length of sound elements. The result is a short composition scaling down five complete days of data, using sounds from Freesound.org and a few of my own recordings to construct sonic blocks for each commuting segment.”

We used these data artefacts to interrogate the boundaries between maker and interpreter, and to ask who—or what—has the authority to interpret narrative and assign meaning to data things? We exchanged these artefacts without providing guidelines for how to interpret them in order to study where the boundary between maker and interpreter emerges. In exchanging the artefacts, we explored the role of the interpreter as a re-maker and how multiple narratives can productively co-exist. We conclude with a discussion about how reimagining the roles of maker and interpreter might lead to new interactions with personal data narratives.

This photo shows close-up detail of the “forgetting” scarf, with elements such as diagonal lines of holes and groups of bulbs, as well as the creator, Mikhaila Friske, wearing the scarf.
Mikhaila’s forgetting scarf: “I knew that I wanted to make a scarf before I had even finished collecting information on forgetting. I have knitted scarves many times before with bulky yarn, thus I knew I would have no problem finishing a scarf within a week’s time. It then became a question of what to visualize, and I decided to focus on how disruptive remembering I forgot had been and the feeling of just forgetting.”

A schematic of the forgetting sound artefact, wich was presented on a website. Each sound has layers representing the category of forgettances, as well as how many items were in that category.
A schematic showing Jordan’s sound forgetting artefact: “I challenged myself to create a sound artefact that didn’t unfold temporally and could be explored non-linearly. This guided my data collection and analysis, which produced categorical clusters of types of forgettances. During data collection, I found and fell in love with a dataset of humans imitating familiar sounds, like birds or home appliances. The strange-yet-familiar echo of multiple voices imitating the same sound resonated with my experiences of forgetting. Using these sounds, I created 30 short compositions, one for each forgettance, and presented them via a web interface: the listener clicks a button to hear a random sound.”

Through creating hand-crafted physicalizations and sonifications, we present three themes on making personal data narratives:

  1. matching data to the materials (and vice versa),
  2. accepting the materials’ will to co-author,and
  3. negotiating between the experience of the data and data of the experience.

The busyness mug cozy, with a mess of tangled yarns on the inside.
Mikhaila created a mug cozy to represent their forgetting data: “In trying to think of what I could make, I thought about what is a thing people do when they are really busy: drink lots of coffee. My last personal data narrative became a mug cozy, something to put on a coffee cup to keep your hands from burning; its function relates to when I am at my busiest and its look mirrored a calendar.”

A comit representing the busy-ness sound artifact, which was a dialog with an Alexa assistant about perceived versus actual busy-ness.
Jordan represented her busy-ness data through an imagined interaction with a voice assistant: “I collected daily journal entries describing the things I had to do each day along with a 1 to 10 score for the perceived and actual effort. I was struck by how wrong my perceptions were and imagined a voice assistant-controlled to-do list that could help me recalibrate my own perceptions. I presented this in the form of a short audio drama that contains short vignettes of conversations between me and an Alexa device.”

We found that our relationship to the roles of maker and interpreter is a circular one—we are constantly being reborn from one to the other.

To learn more about our work, read the full paper here or watch a video presentation: