While on my way back from New York, for some odd reason I started playing around with Foursquare and plotting my check-in data using a handful of apps. Very quickly I realized two things: the amount of time I spend in airplanes has doubled every year since 2009, and when I am in San Francisco, I lead a very predictable life and go to only a handful of places — a lot.
Except for one small thing: While the data shows that I lead a pretty boring life, it doesn’t reflect the “emotions” behind the data. Why, you might ask, is this important? The answer is that as we move towards a quantified society, one shaped by data, we start to dismiss things that aren’t easily quantifiable. Empathy, emotion and storytelling — these are as much a part of the business as they are of life. Without these, we might as well residents of starliner Axiom.
The problem with data is that the way it is used today, it lacks empathy and emotion. Data is used like a blunt instrument, a scythe trying to cut and tailor a cashmere sweater. Some folks do a better job of making data interesting — like the fine folks at Foursquare. They use cutesy phrases to remind me of my coffee addiction and occasionally point out that Jared Kim and I are besties when it comes to eating ramen noodles or visiting Hakkasan, but they don’t really tell the whole story and they need to do more. I will get to that in a minute.
The idea of combining data, emotion and empathy as part of a narrative is something every company — old, new, young and mature — has to internalize. If they don’t, they will find themselves on the wrong side of history.
The Uber-side Story
Let’s explore this idea further with what is my most used service: San Francisco-based local transportation startup Uber. As someone who doesn’t drive and doesn’t own a car, Uber has been a godsend. And that is why I am hoping that they stay in business — for a long time. Selfish, much? Of course! That is why I am perhaps harder on them than any other startup, poking holes in their strategies, ranting on Twitter and occasionally praising them for their awesomeness. Time and again, Uber finds itself in the eye of the (public relations) storm, and to me the reasons are pretty obvious.
If you look at the relatively young history of Uber, you would see that it is a company that has done many things right and a couple of things very wrong. It has figured out how to remove friction between a traveler and transportation by reducing it to mere minutes. In doing so, Uber has also become the first next-generation commerce company to use connectedness to its advantage. It has also figured out how to organize what is a disorganized and poorly managed business: local transportation.
Uber’s secret weapon is the data it is collecting, but what it has failed to do — use that data in the most powerful kinds of ways. Data tells the service that there are fewer cars on the road and massive demand, and out comes surge pricing. This makes perfect business sense — except when it is in New York City battered by Hurricane Sandy. Ooops! The company failed to factor in the “emotion” and “humanness” in its data.
A better approach would have been to give people discounts based on the time they had to wait, and make up the difference to the drivers (or even give them bonuses for working around the clock). Yes, it would have been costly, but it would have cemented the “Uber cares” sentiment.
The whale theory
Uber (and I don’t mean to pick on them, because it applies to all companies) should be thinking about using data to create positive experiences. A good way to start is to take a cue from the casinos (or Zynga): make the “whales” happy.
If someone is a big spender on their service, then they should get to the front of the line and get an Uber car before everyone else. Sure, they can create loyalty discounts, but time saved (and a clean experience) is what’s more important to an Uber customer. (This shoe store in Manhattan knows how to use data to make customers happy, and perhaps others should take a hint from them.)
Uber should give the whales an experience that puts a higher premium on their time than that of occasional users. Match up these whales with the best cars and the top-ranked drivers so they will keep spending. In other words, use data to shape experiences.
Some might say that this is yet another example of data darwinism at work — and in one sense it is — but big spenders get big perks from casinos and hotels (and some airlines). Uber wouldn’t be doing anything wrong if it followed suit, as I don’t think of them as an essential service. I am and will remain wary of the idea of data darwinism creeping into essential services.
Using data to shape experiences has to become the default for all startups, regardless of whether their focus is on consumers or large companies. Almost every day I come across an app that has an astonishingly beautiful interface, only to find it incredibly vapid and unintelligent. A lovely interface comes alive when married to data and the insights and context it brings.
Coffee and empathy
What will it take to build emotive-and-empathic data experiences? Less data science and more data art — which, in other words, means that data wranglers have to develop correlations between data much like the human brain finds context. It is actually not about building the fanciest machine, but instead about the ability to ask the human questions. It is not about just being data informed, but being data aware and data intelligent. Sean Gourley, co-founder of Quid, in his keynote speech at Structure: Data, noted:
Data scientists are presented with a set of parameters to optimize over, yet they don’t take the time to step back and say, should I even be optimizing this at all. Data science, I believe, we need to re-imagine it, because data is incredibly powerful. We need to step back from the scientific notations and start thinking of it as data intelligence. Data intelligence has a slightly different philosophy that embraces some of the messy and unstructured nature of the world that we do live in.
Let me explain by using my favorite coffee shop as an example. The raw number of check-ins at the coffee shop tells me that I am boring and predictable. Anyone who doesn’t know me well will draw the same inference. But parse the data in more granular manner and correlate it with other information and you start to see a picture that understands the emotional value of that data.
Let’s run with this example. The service needs to realize that I visit said coffee shop first thing in the morning (something that can be inferred from the time of the check-in.) It also needs to learn that the coffee shop is a few miles from my home, which tells you that I go out of my way to go there.
The service also knows that I check-in with a handful of people, whose relationship to me can be inferred from the social graph. Add to the mix the fact that I have left tips and taken photos at that spot. Now, compare it to all other coffee shops I have checked into and how they rank against this one location. Add all of them up, and you end up with a rudimentary conclusion: I don’t go to this coffee shop simply because it’s an interchangeable part of my daily routine, or because it’s on my way to work. I visit it because it is my happy place, my one cup (or dozen) of zen. And a company like Foursquare could use that fact to package even more compelling experiences for me.
Data needs stories
The key to getting the context is to think in terms of stories. Sean put it best when he said:
“Data needs stories, but stories also need data. Data, when its put up in front of you as a number, it gets stripped of the context of where the data came from, the biases inherent in it, and the assumptions of the models that created it.”
The symbiotic relationship between data and storytelling is going to be one of the more prevalent themes for the next the few years, starting perhaps inside some apps and in the news media. I was reminded of the future filled with data narratives when I saw this visualization — Out of Sight, Out of Mind, by Pitch Interactive. It takes data about drone attacks and makes them visual and easy to understand, and in doing so, elicits a strong reaction.
But it merely scratches the surface — presenting a slight improvement on an infographic that might have appeared in the pages of a magazine. In a future where we have tablets and phones, packed with sensors, the data-driven narratives could take on an entirely different and emotional hue.
As for Uber and Foursquare, they should start by thinking, how do we make our customers feel special?
Post and thumbnail image courtesy of Shutterstock / Chepko Danil Vitalevich
Fantastic post. How about managers who love to say “I want to see the data” because it makes them feel like they’re making informed decisions, but then exclude the story and end up with a poor decision instead? My boss chewed me out because the goal conversion rate on our website hadn’t improved year over year and wanted me to change my strategy. Somehow he managed to gloss over the graph showing that at this time last year the 6-month trend line was pointing straight down, and now it’s up and to the right.
Thanks Andrew for your comment and your feedback. Interesting – that we continue to see data in small chunks and don’t factor data over the longer term. Good to know that things have changed for you and your company.
It’s seems pretty straightforward that businesses should make better use of transactional data to provide customers better experiences. There’s huge room for improvement with internal data (why does United not contact me after I stop taking the same monthly flight I’ve taken for 10 month straight? Maybe it has something to do with that $150 change fee in their database) and external (ah, there’s a negative tweet from me right before I stopped making reservations).
But services like Foursquare are never going to be able to lay down much soul because they have no access to the transactional content… and trying to infer something material out of some location data is really just not a good use of time. Actually, the same thing can probably be said of the bulk of big data initiatives… trying to find a soul in an ocean of weak signal data. It’s not there until we solve the “small data” problem of getting more context/intent into the data itself.
Great post! With the recently found fascination for all things “Big Data” and “data driven decision making” the emphasis seems to be more about the data, the algorithms used and less about the story or context around it. This I attribute to the Technologists love for more sophisticated features and the belief that increase in capabilities always yields proportionately better results. Truth is that story telling is an Art. When it comes to providing insights from data, there’s both art and science involved. Just increasing the megapixel count (data collected) of a camera doesn’t yield photos that have soul. No doubt, like any other gadget geek I am enamored by the ability to take pictures in low light conditions and capture more detail, but in the end what matters is my artistic capability, my ability to use available light and perspective (or I guess Photoshop), to tell a story that connects with the audience.
Agree with you that Data is powerful, but that’s like saying words are powerful. There’s a simplistic notion that more data is always better. When I read 6 word memoirs in the book “Not quite what I was looking for” I was amazed and touched. It wasn’t the words used, but the stories that just 6 words could tell. Having more data tremendously increases the complexity of telling compelling stories. Tools can help, but at the end of the day, I’d rather be an editor of a 300 page manuscript than a 3000 page manuscript.
Unfortunately, the term “Business Intelligence” is over twenty years old and seems to be associated more with “reporting”. I don’t know if “Data Intelligence” as mentioned by Sean Gourley is any better because the way I interpret Business Intelligence – you need the business knowledge and the Intelligence (both IQ and EQ) to parse and interpret the data within the right context to provide insight
Great points @michaelBrill and @prakash. You both added a nice dimension to the conversation.
My point right now with my post is that data has to tell stories if it wants to shape experiences. The Foursquares of the world etc have to start thinking about taking data and putting a social/individual/human context around it.
It is the lack of that human thinking is why Facebook feels less than intimate when it should have more feeling of closeness. Anyway at least I think so.
Reblogged this on kwalitisme and commented:
That’s Qualtism (Kwalitisme): the search for the soul in everything that surrounds us in this over and over quantified society.
Love your post. Yet, not all big data needs soul or is used to create unique customer experience. Sometimes you just need to fight a pandemic (or forest fire), predict an earthquake (or early baby delivery) or manage complex logistic operations. Things defenitely go wrong though, if my visit to a coffee shop or shoe store is treated like a complex ligistic operation – and nothing more. Great stuff!
Everybody talks about context, and very few know what it is and more important how it’s created. Then there are those who think context is just a mix of sensor data, which can be manipulated by p-value hacking and everybody looks smart. Complexity after all makes you look smart. How these companies will get to natural personalized systems is beyond me, that seems is what you want? Or how do you get from data to context to information and how does it relate to story telling?
The “soul” of the data should be represented in the strategies and procedures that is put in place around the data. First strategy should be what data is going to be looked at and what does the company want to achieve with the data. Then different strategies on how to deal with the results of the data.
Data in itself is neutral and without a soul, that is the essence of data. How the data is used and acted upon either makes the data valuable or just another bland excersize.
The one issue that you ignore in this article, is the fact that this data is all owned by some third parties. While you might think that this is a separate discussion, it is actually not. The companies might choose to share data about you among themselves, but they might not as well. And partial emotions, even if deduced, mean nothing. For instance, uber might know that you were in a hurry and booked a cab from X to Y and Y to X twice. But foursquare might know where exactly you were. Neither of them know why you were there. Facebook has some hints and Twitter as well.
It has been a bad idea to let these services handle data, instead of protocols. We wrote about it here:
http://statspotting.com/instagrams-policy-changes-is-it-time-to-move-everything-to-our-pockets/
Wow, glad to see that I am not the only one thinking about this. When I think of data, to me at least, it is just plot points. It is a measurement that has been taken, plain and simple. Let’s not quibble over the type of data we are collecting. All we need to know is something happened, it got measured, and now we want to do something with it.
Then the human brain gets its hands on this and begins to interpret. That is the trouble, if we don’t understand the context of the information than we can’t do a proper analysis. The numbers in of themselves are meaningless and it is our assumptions that will color the data.
I just wrote about this on my personal blog, albeit not so eloquently. This is something that really concerns me about the startup world. Data is great, but don’t lose the humanity that lies just beyond the numbers.
Absolutely brilliant and spot on with much deeper implications than a typical business and tech story. Thank you.
Sometimes I want to tell businesses that are tracking me what’s going on with me so they will understand the data better. For example, if I’m stressed I want to watch happy shows on Netflix, so I abandon House of Cards for Parks & Recreation for a while. Doesn’t mean I don’t love House of Cards.
Om,
Thanks for sharing your thoughts. I am a high school teacher, and for the past three years, all of education is frantic about “data”. We have to collect data, read data, and use data to make decisions. The reasoning is sound (the more you know, the more you can focus your teaching), but the individual student and the human-ness of teaching are left behind and lost.
Grant
Insightful post. I hope Foursquare reads it- I’ve been thinking about the soulless aspect of tracking checkins lately .. this post nailed it. Adding the emotion/ story as you say would make for a more meaningful experience . One to hold on to over time. Like a personal archive. What really site point of these check-ins if not ?
lovely post as always, Om
continuing Nico: The “soul” of the data should be represented in… the vectors which could be found during data representation over human UI. ‘use data to shape experiences’ – is a motto for any marketer.
Data needs creativity..and badly!
Om your fine post ties nicely to Derrick Harris 5 ways big data is going to blow your mind and change your world http://gigaom.com/2013/03/22/5-ways-big-data-is-going-to-blow-your-mind-and-change-your-world/
You make a great tag team 😉
Agreed – Derrick is the lead-off hitter, I am playing DH though
Great, timely article.
Using data to create wonderful experiences assumes that companies have at least two pieces in place: (1) a complete view of their customer and (2) a culture that will promote a customer centered view of the world.
Most companies and prospects I work with first need to assemble that complete, cross-channel view of the customer. Typically customer data is stored across several silos—what are, in effect customer data prisons—and not organized to understand and predict customer behavior. Solving this problem is the first step.
You also need senior managers with the vision to reorient their marketing strategy to be customer-centric. This means prioritizing the customer’s experience over quarterly numbers with the goal of driving long term growth.
I’m optimistic these will come with time but expect continued frustration in the intervening period. 🙂
Best,
Brian Ivanovick
http://www.causata.com
I always refer to this as quantification sans humanity. Lots of data is great, but if you don’t understand the real meta-data – the human context – the conclusions drawn are at once surprising and potentially perilous. As a thought exercise, imagine a data-driven evaluation of earth and its antagonists. If you set as your objective preserving the delicate balance that is life on this blue planet, it will not take long before you conclude (looking just at the data) that all the causal interactions suggest the single biggest non-Black Swan (ie asteroids, volcanos etc) risk is the presence of a widespread pestilence… aka humans. Let’s hope any advanced observers in the cosmos don’t compute that equation in an emotional vacuum.
Matching big data with curation (actual human supervision and intervention in the data driven process) is perhaps the best way to avoid the worst aspects of Data Darwinism. Old fashioned magazines, newspapers, and television networks, all data driven yet “curated” by very smart human intervention, have done this in the legacy world for years with very good results. The modern development of content, blindly married to big data, cannot approach the quality of this older product. But it is early days yet, and eventually curation will routinely find its way back into the big data mix.
Really great post- in a compelling and entertaining way, to me, it summed up one of the few key debates on big data. I work for UNDP and we’ve recently started looking into methods in which we can generate loads of data on people we work with so we can figure out how best to invest our resources to provide support, as well as layer it with context and stories… we’ve just started but it is incredibly exciting as it turns everything i’ve ever known about traditional research methods on its head and provides such rich information! Here’s a teaser http://bit.ly/YSQlYi
thanks again for a great post!
@ElaMi5
Still digesting this one. Great article.
Thanks @Hilton. Appreciate the note of encouragement
Good article, thanks. How come 99% of VC’s think non-automated parsing cannot “scale”?
“parse the data in more granular manner and correlate it with other information and you start to see a picture that understands the emotional value of that data.”
@AndyLeSavage,
Thanks. It is not just VCs, most of the world assumes that the kind of parsing is not scalable. I am doing a follow up post to deal with exactly the same topic and hopefully will have comments from many experts after some reporting.
@AndyLeSavage — it goes deeper than 99% of VCs views on non-automated parsing for emotions.
The machines themselves can only auto-parse against known lexical libraries like Princeton’s Wordnet.
Take a look at various sentiment analytics attempt to parse the emotions expressed across our social media. They reference against those known sources and It’s messy and noisy.
The problem goes as far back as the earliest days of our species documenting our experiences (whether that was in the cave paintings, papyrus scrolls, Egyptian hieroglyphs, Bayeaux tapestry, Johnson’s dictionary, the Guttenberg press and Babbage’s machine).
In any case, for the machine algorithms to “have soul” and be able to auto-parse emotions requires our society to completely overhaul code structure so that it’s more akin to organic chemical molecules rather than flat, logical, functional layers of code that build up to “Big Data”.
Out of interest, Om, do you consider Amazon’s utilization of data as data **with a soul** or **without a soul**?
I ask, because, while Amazon is one of the lesser aesthetically pleasing services in terms of emotive design, it’s arguably one of the most human services in terms of contexts and well-codified workflow that are bound to clear jobs and outcomes for the user.
Totally without soul and they need to start thinking about the world beyond what they do right now on their website. I think the Amazon Kindle Store is much better experience as it is highly focused but could be more informed and exciting.
Actually, that is a good point. My comments largely focused on Amazon website, as the mobile apps have a bolt-on, silo’d feel about them.
Summary: Qualitative beats Quantitative 🙂
Inside most companies (including Microsoft) as long as the graphs move towards the upper right for growth nobody stops to really get into why or whether the data is still valid. Breaking apart data is often a translation issue and can be easily done with a infographic visualisation (visual thinking).
In the case above isolating the “why” for checkins is enough of a thread to start piecing together a story or visual thinking. In that custX keeps checkins to ACME Coffee Shop because Their Coffee Beans or Girl behind the counter is smoking hot (or both).. Yes its shallow but there’s no judgement just reality.
Let the organic data tell the story is my point but dont try and shape it or panel beat into pre-defined patterns or templates that suites a global story… thats the trick here.
Thanks for you comment @scottbarnes!
Reblogged this on The Dream Blog and commented:
Interesting thoughts on Big Data and it’s practical applications
Om,
Great post — it inspired one of my own for Search Engine Watch: http://searchenginewatch.com/article/2259101/Big-Data-Has-It-Blinded-Us-With-Science
Thanks!
Dave
Om,
Data with head, heart and soul is what Senseus is all about:
* http://www.senseus.co/blog/?p=58
“Big Data” and their probability-quant approaches are not the panacea in our view.
Da Vinci said, “All our knowledge has its origins in our perceptions” and so Senseus restores perceptions (head, heart and soul) to its rightful place in the way we decide why we buy into something — whether that’s coffee, content, relationships or experiences.
One of the best post that i have read in some time on ‘data’.
We collect data and use data effectively to enhance our experiences and tell stories. But this requires understanding relationships among disparate data items. And that is where the importance of Big Data really lies.
A major transformation going on in society currently is the nature of story telling. At one time story telling was based on subjective experiences by an individual that were only qualitatively available to the storyteller. The last decade has seen the arrival of ‘data’ and objective story telling. Human memory is powerful but has its limitations. Subjective human memory results in powerful anecdotes. Add to this availability of large volumes of data and ability to attach it to strengthen those anecdotes and you got powerful and compelling factual stories.
I recently wrote something on this topic: https://dl.dropbox.com/u/5644217/IEEE%20Multimedia%20Storytelling%20Jan%202013.pdf
Once again, great article.
Hi Ramesh
It is good to see you here again and thanks for your insightful comment. The article link is deeply appreciated.
Kinda flipped out.
Very well thought out piece, Om. Raw data needs to be made sense of, it needs to be interpreted to create a context. Context, that could help machines “understand” their job and realize things without explicit directives. Context would help humanize them, and not commit errors of sentiment as did Uber’s system during Sandy.
I wrote an essay on similar lines a few weeks ago. You can find it here: http://spinhalf.net/context/
Om,
I quote “Data without soul is meaningless” on my site: http://senseus.co/#menu_overlay
Senseus has a system to include the expression of emotions that’s different from vote up-down, 5-stars and Osgood sentiments which are used in existing Big Data practices.
There are a couple of graphics at the bottom which explain this further.
best,
Twain.