I was recently invited to speak at DUXcamp hosted by NPR and then again at Microsoft Research around the subject of Mobile and Storytelling. I created a rather stream of consciousness presentation, bringing together various thoughts about storytelling in the mobile space. Still very rough around the edges but a central theme is beginning to emerge: Mobile allow stories have scale.
Here’s the presentation deck:
2011-11-02 Mobile Storytelling (@ Microsoft Research)
Preamble
With the arrival of smartphones, it’s amazing how much data we are collecting and consuming on our mobile devices. We tweet, checkin, google, blog, instagram, post status updates, yelp and a host of other things from our handheld devices. And somewhere on the internet this information is quietly collecting. In the ancient times, pharaohs had scribes that shadowed them, recording what they said. Now we have our mobile devices diligently collecting our data. There was once a time when people used to record their lives and thoughts in leather-bound diaries. Now we have smartphones, whose data, when strung together form a story of our lives.
My Story
I can’t remember when I heard it for the first time, but someone said, our identity is the story we repeat to ourselves. This is so true. I keep on telling my story of how I moved between the East and West, between physical environments (architecture, urban design) and virtual (web and mobile development and strategy), between technology and the humanities. I don’t have an identity grounded in an single culture, nation or land. At one time, I would have referred to this as being nomadic. Now I can just say I’m Mobile Me to borrow a term Apple has abandoned.
Our Stories
Today we have a wild abundance to the ways we collect our stories. Many of them track us automatically: Nike Plus tracks my run, Mint.com tracks my finances and spending patterns, and Trip It neatly organizes my travel plans.
At the rate that memory capacity of devices are increasing, in a couple of years we will have an iPhone which would hold 256GB of data. Battery-life permitting, this would mean (albeit at a low resolution) you’d be able save your whole life by dangling your iPhone around your neck and recording every moment. This is often referred to as life caching or lifelogging. But what’s the point? When will you have the time to go back through hours of video to find and edit the interesting or meaningful parts. Jorge Luis Borges points out that 1:1 scale map is useless. Aren’t we doing just that when we don’t filter to good from the mundane?
Nicholas Felton has been obsessively collecting data about himself and publishes them in annual reports about himself since 2005. And now with his own iPhone app Daytum, you too can be as obsessive about your data as he is.
But Felton does provide us with a insightful clue. What data is meaningful? For his 2010 Annual Report he compiled and presented data around his father’s life. It is surprisingly moving. He masterfully abstracted meaningful data from the numbers and constructs a picture that pays a deeply personal and loving tribute.
An iPhone app called Memento compiles the data from your various disparate personal information repositories such as Twitter, Facebook and Flickr, and brings them back into a diary format, of all things. What used to be manual labor is automagic and becomes personal again. You can even add diary entries. What emerges is a story – your story. You see densities of information where you had memorable events, and long silences where you were buried in depression being dumped.
Other People’s Stories
We live in the age of Facebook. But Facebook is horrible when it comes to telling stories. It presents fragmented pieces of people’s lives that we are often forced to react to rather than engage. The timeline, in its quest to present ever growing amounts of information to us, become as fleeting as the stock ticker feed in Times Square, and belittles the personal importance of each post, by rendering it in the same small block, with the same small profile icon, in the same small font as everyone else. Some people are simply more important than others and we want to pay more heed to them. They are larger in our minds. Why are they the same size as the person whom I casually had a short conversation with at a conference I don’t even remember? Facebook is addressing this issue by adding filters, but with all the data crunching power that they use already around analyzing my relationship with my friends, shouldn’t they know who is important to me already?
Newspapers know how to present information. They’ve had enough years to refine their art. Typeface sizes matter. The fold matters. Sections matter. Photos matter. They bring your attention to what they deem important. Flipboard is an iPad app that tries to do that, by providing an illusion of priority through a tactful manipulation of layout, font sizes and images. It provides much needed difference and rhythm we are attracted to, over the often mind-numbing flat Twitter or Facebook feed.
Our Collective Stories
Daum Communication, a leading internet services provider in Korea offers a map service with a streetview option, much like Google Maps does in the States. As of Feb 2011 however, they have added a feature that goes a step beyond: streetview history. You can select from various past dates when the streetview camera captured the image. As one example, you can view the building where Daum is located now, under construction in 2008.
Maybe it’s possible to take this further by using tools like Photosynth to crowdsource forgotten images from people’s photo albums or maybe even add historic archival images, so that when you are viewing a certain place through the streetview tool, you can actually go back in time and take a historical journey through a neighborhood. Historians can narrate stories of a city’s development or you can tell your own story of fond childhood memories. What was once a personal memory can now build up a crowdsourced collective memory.
Curtis Wong of Microsoft Research has an wonderful presentation of Microsoft’s World Wide Telescope project where the tool for presenting the universe around us sets a stage for storytelling by allowing researchers and students alike to create a narrative through the interface. Something like an interactive version of Charles and Ray Eames’ masterpiece Powers of Ten.
Usahidi, an interactive map-based information collection tool was born out of a need to capture and report post-election violence during Kenya’s 2008 presidential elections. Usahidi means testimony. Since then it has been used widely to crowdsource data through mobile devices and present them dynamically on a map: from neighborhood snow removal updates to crime reporting. Most noteably it was deployed in the aftermath of the devastating earthquake in Haiti to crowdsource unsafe conditions and aid relief coordination.
How did you hear about Steve Jobs’ death? A lot of us heard through Twitter or from someone who heard it through Twitter, as a collective gasp went through the twitterverse at the news of his sooner-than-expected death. Tweets per second (TPS) is now a proxy for the velocity of the spread of news. When it comes to TPS, surprisingly Jobs’ death ranks #5. It’s the news of Beyonce’s pregnancy announced during the MTV Video Awards that takes the honor of #1. A newborn life wins over death.
Adding Our Life to Data
Jawbone, which produces high-performace mobile headsets, just came out with a very affordable health monitor bracelet called UP. Coupled with a smartphone, this bracelet tracks your eating, sleeping and exercise habits and “nudges” you to adopt better habits. You can imagine market-research groups like Nielsen paying people to don a device like this to track how people really react to what they are watching. Nike Plus gathers data about your run, but what would it be like if global events were tracked not just in the number of media reports but as bio-metric data? What kind of story would that tell? What would a collective “gasp” look like when people heard of Steve Job’s death or Beyonce’s pregnancy?
Interactive artist, Jonathan Harris is an amazing story teller. We Feel Fine is the project he is best known for. But his Whale Hunt is an incredible project in many ways. Here’s what he did:
I documented the entire experience with a plodding sequence of 3,214 photographs, beginning with the taxi ride to Newark airport, and ending with the butchering of the second whale, seven days later. The photographs were taken at five-minute intervals, even while sleeping (using a chronometer), establishing a constant “photographic heartbeat”. In moments of high adrenaline, this photographic heartbeat would quicken (to a maximum rate of 37 pictures in five minutes while the first whale was being cut up), mimicking the changing pace of my own heartbeat.
The result is very close to how our minds actually work – we capture more information and memories in relation to how intense our experience is. Time slows down because we are collecting more information (often for our survival).
This is exactly what happens in the way we collect data through our mobile devices. The more significant the event or location, the more photos, tweets, status updates, blog entries we create about it. You can see it on an individual level, but also on a greater collective level. If you were to represent this in a graphical way, you’ll see something analogous to World Wide Telescope’s universe, where you would have stories instead of stars. What would it mean to look at galaxies of stories across time and distance, zoom into individual shining stars of stories, or encounter black holes where a natural disaster abruptly muted thousands of voices in a single horrific event. You can almost imagine ripples of story supernova spreading at the speed of light as the news of the disaster spreads in its aftermath.
Scale of Stories = Scale of Identity
Recently, overcoming a freak October snowstorm in Washington DC, I went to the National Museum of the American Indian, and then to the National Museum of American History. There I witnessed two institutions telling stories. One of a frequently muted story of the American Indian, whose so many tribes are now forgotten because their stories did not survive the diseases, conflicts and forced migrations. In contrast I saw the victorious stories being told of a young nation who overcame colonial powers, native inhabitants and inner division, whose short story is still unfolding, and needs to be remembered and repeated because its identity and survival as a nation depends on it.
When I showed Google Earth for the first time to my dad on an iPad, the first thing he did was to look for the house he grew up in, deep in North Korea, having left it behind some 60 years ago during the Korean War. I saw the concentration and the emotion that poured over his face as he searched for his childhood home by scanning the geography but also his memory, desperately inferring its location through the landscape of streams, valleys and railroad tracks he remembered.
Zooming in, it’s my dad’s childhood story. Zooming out, it’s the tragic story of the Korean War and the subsequent division of Korea. Further out, it’s the historic story of the fear and ideological power struggle between the superpowers following World War II.
Our identity is the story we repeat to ourselves. If that is so, what is the story we repeat to ourselves as an individual, family, community, region, nation or as a humanity? For the first time in history, as we collect so much data about ourselves, we have the potential to simultaneously see our stories unfold dynamically at different scales. And maybe that can teach us something about ourselves.
—
Update (2011-11-09)
Microsoft Research has just posted the presentation online. It’s in 2 parts. See the second half.