Exploring with Yarn and Sound for Expressing Personal Data Narratives

How do making and materials contribute to our understanding of personal data representations? To explore how materials, data, and humans collaborate to produce physical data representations, we created a series of artefacts from personal data we collected (about commuting, forgetting, and busy-ness) in different media: yarn and sound.

A commuting mini-quilt, conveying the emotions Mikhaila felt while commuting.
Mikhaila’s commuting yarn artefact: “I wanted to represent segments of my commute (car or bus), the decisions behind them (convenience, price, time), and how I generally felt while commuting (frustrated, anxious, indifferent). Since I had been playing with crocheting different shapes the week prior, I wanted to create both squares and triangles. I originally wanted to create a larger patchwork (to possibly turn into a pillow), but I ran out of time and instead chose one day to transform into the artefact.”

A schematic diagram showing the sonic layers in Jordan's commuting sound artefact.
A schematic diagram showing the sonic layers in Jordan’s commuting sound artefact, which you can listen to here: “For commuting, I was unsure of my abilities in sonification and thus focused on a straightforward mapping between duration data and the length of sound elements. The result is a short composition scaling down five complete days of data, using sounds from Freesound.org and a few of my own recordings to construct sonic blocks for each commuting segment.”

We used these data artefacts to interrogate the boundaries between maker and interpreter, and to ask who—or what—has the authority to interpret narrative and assign meaning to data things? We exchanged these artefacts without providing guidelines for how to interpret them in order to study where the boundary between maker and interpreter emerges. In exchanging the artefacts, we explored the role of the interpreter as a re-maker and how multiple narratives can productively co-exist. We conclude with a discussion about how reimagining the roles of maker and interpreter might lead to new interactions with personal data narratives.

This photo shows close-up detail of the “forgetting” scarf, with elements such as diagonal lines of holes and groups of bulbs, as well as the creator, Mikhaila Friske, wearing the scarf.
Mikhaila’s forgetting scarf: “I knew that I wanted to make a scarf before I had even finished collecting information on forgetting. I have knitted scarves many times before with bulky yarn, thus I knew I would have no problem finishing a scarf within a week’s time. It then became a question of what to visualize, and I decided to focus on how disruptive remembering I forgot had been and the feeling of just forgetting.”

A schematic of the forgetting sound artefact, wich was presented on a website. Each sound has layers representing the category of forgettances, as well as how many items were in that category.
A schematic showing Jordan’s sound forgetting artefact: “I challenged myself to create a sound artefact that didn’t unfold temporally and could be explored non-linearly. This guided my data collection and analysis, which produced categorical clusters of types of forgettances. During data collection, I found and fell in love with a dataset of humans imitating familiar sounds, like birds or home appliances. The strange-yet-familiar echo of multiple voices imitating the same sound resonated with my experiences of forgetting. Using these sounds, I created 30 short compositions, one for each forgettance, and presented them via a web interface: the listener clicks a button to hear a random sound.”

Through creating hand-crafted physicalizations and sonifications, we present three themes on making personal data narratives:

  1. matching data to the materials (and vice versa),
  2. accepting the materials’ will to co-author,and
  3. negotiating between the experience of the data and data of the experience.

The busyness mug cozy, with a mess of tangled yarns on the inside.
Mikhaila created a mug cozy to represent their forgetting data: “In trying to think of what I could make, I thought about what is a thing people do when they are really busy: drink lots of coffee. My last personal data narrative became a mug cozy, something to put on a coffee cup to keep your hands from burning; its function relates to when I am at my busiest and its look mirrored a calendar.”

A comit representing the busy-ness sound artifact, which was a dialog with an Alexa assistant about perceived versus actual busy-ness.
Jordan represented her busy-ness data through an imagined interaction with a voice assistant: “I collected daily journal entries describing the things I had to do each day along with a 1 to 10 score for the perceived and actual effort. I was struck by how wrong my perceptions were and imagined a voice assistant-controlled to-do list that could help me recalibrate my own perceptions. I presented this in the form of a short audio drama that contains short vignettes of conversations between me and an Alexa device.”

We found that our relationship to the roles of maker and interpreter is a circular one—we are constantly being reborn from one to the other.

To learn more about our work, read the full paper here or watch a video presentation:

Quantuition: Exploring the Future of Representing Biometric Data

In a future where we track everything, how will data representations dictate how we relate to ourselves and the world? This speculative design project explores the relationship between personal biometric data and the meaning we find in it.

Quantuition is a speculative self-tracking system that collects data from body-based nanosensors. The system renders that data into 3D data sculptures. Presented in the form of an Instagram feed, this speculation highlights how data-design influences the process of individual and social sense-making. We often ascribe power and authority to data representations — while simultaneously overlooking the hidden decisions embedded in those representations about what to measure, analyze, emphasize and display.

This schematic shows the basic steps of using the Quantuition system, from ingesting and applying the nano-sensors, to collecting and transmitting the data, to producing 3D-printed data objects.

When self-tracking becomes pervasive, are we ruled by data or do we rule it? In the near future, personal sensors track everything: how fast our hair is growing, the amount of dust we inhale, how many tears we cry. As we become aware of these myriad personal data points, they could overwhelm us. How do we draw meaning from this data? How do our interpretations of this data influence our actions, and what are the implications of these new feedback loops?

This Instagram feed tells the story of a user integrating Quantuition into her life.

This project raises several provocations for tangible data futures:

  • What new interpersonal interactions does data physicalization uncover?
  • What positive and negative feedback loops are present in a hyper-quantified future?
  • How will emerging technology shape the relationship between data representations and actions?
  • Where does the user’s control – and free will – begin and end?

This work was submitted to the TEI 2019 Student Design Competition, where it won the award for best concept/design.

Read the project paper | View the poster