41 thoughts on “Coffee & Empathy: Why data without a soul is meaningless”

  1. Fantastic post. How about managers who love to say “I want to see the data” because it makes them feel like they’re making informed decisions, but then exclude the story and end up with a poor decision instead? My boss chewed me out because the goal conversion rate on our website hadn’t improved year over year and wanted me to change my strategy. Somehow he managed to gloss over the graph showing that at this time last year the 6-month trend line was pointing straight down, and now it’s up and to the right.

    1. Thanks Andrew for your comment and your feedback. Interesting – that we continue to see data in small chunks and don’t factor data over the longer term. Good to know that things have changed for you and your company.

  2. It’s seems pretty straightforward that businesses should make better use of transactional data to provide customers better experiences. There’s huge room for improvement with internal data (why does United not contact me after I stop taking the same monthly flight I’ve taken for 10 month straight? Maybe it has something to do with that $150 change fee in their database) and external (ah, there’s a negative tweet from me right before I stopped making reservations).

    But services like Foursquare are never going to be able to lay down much soul because they have no access to the transactional content… and trying to infer something material out of some location data is really just not a good use of time. Actually, the same thing can probably be said of the bulk of big data initiatives… trying to find a soul in an ocean of weak signal data. It’s not there until we solve the “small data” problem of getting more context/intent into the data itself.

  3. Great post! With the recently found fascination for all things “Big Data” and “data driven decision making” the emphasis seems to be more about the data, the algorithms used and less about the story or context around it. This I attribute to the Technologists love for more sophisticated features and the belief that increase in capabilities always yields proportionately better results. Truth is that story telling is an Art. When it comes to providing insights from data, there’s both art and science involved. Just increasing the megapixel count (data collected) of a camera doesn’t yield photos that have soul. No doubt, like any other gadget geek I am enamored by the ability to take pictures in low light conditions and capture more detail, but in the end what matters is my artistic capability, my ability to use available light and perspective (or I guess Photoshop), to tell a story that connects with the audience.

    Agree with you that Data is powerful, but that’s like saying words are powerful. There’s a simplistic notion that more data is always better. When I read 6 word memoirs in the book “Not quite what I was looking for” I was amazed and touched. It wasn’t the words used, but the stories that just 6 words could tell. Having more data tremendously increases the complexity of telling compelling stories. Tools can help, but at the end of the day, I’d rather be an editor of a 300 page manuscript than a 3000 page manuscript.

    Unfortunately, the term “Business Intelligence” is over twenty years old and seems to be associated more with “reporting”. I don’t know if “Data Intelligence” as mentioned by Sean Gourley is any better because the way I interpret Business Intelligence – you need the business knowledge and the Intelligence (both IQ and EQ) to parse and interpret the data within the right context to provide insight

    1. Great points @michaelBrill and @prakash. You both added a nice dimension to the conversation.

      My point right now with my post is that data has to tell stories if it wants to shape experiences. The Foursquares of the world etc have to start thinking about taking data and putting a social/individual/human context around it.

      It is the lack of that human thinking is why Facebook feels less than intimate when it should have more feeling of closeness. Anyway at least I think so.

  4. Love your post. Yet, not all big data needs soul or is used to create unique customer experience. Sometimes you just need to fight a pandemic (or forest fire), predict an earthquake (or early baby delivery) or manage complex logistic operations. Things defenitely go wrong though, if my visit to a coffee shop or shoe store is treated like a complex ligistic operation – and nothing more. Great stuff!

  5. Everybody talks about context, and very few know what it is and more important how it’s created. Then there are those who think context is just a mix of sensor data, which can be manipulated by p-value hacking and everybody looks smart. Complexity after all makes you look smart. How these companies will get to natural personalized systems is beyond me, that seems is what you want? Or how do you get from data to context to information and how does it relate to story telling?

  6. The “soul” of the data should be represented in the strategies and procedures that is put in place around the data. First strategy should be what data is going to be looked at and what does the company want to achieve with the data. Then different strategies on how to deal with the results of the data.

    Data in itself is neutral and without a soul, that is the essence of data. How the data is used and acted upon either makes the data valuable or just another bland excersize.

  7. The one issue that you ignore in this article, is the fact that this data is all owned by some third parties. While you might think that this is a separate discussion, it is actually not. The companies might choose to share data about you among themselves, but they might not as well. And partial emotions, even if deduced, mean nothing. For instance, uber might know that you were in a hurry and booked a cab from X to Y and Y to X twice. But foursquare might know where exactly you were. Neither of them know why you were there. Facebook has some hints and Twitter as well.

    It has been a bad idea to let these services handle data, instead of protocols. We wrote about it here:


  8. Wow, glad to see that I am not the only one thinking about this. When I think of data, to me at least, it is just plot points. It is a measurement that has been taken, plain and simple. Let’s not quibble over the type of data we are collecting. All we need to know is something happened, it got measured, and now we want to do something with it.

    Then the human brain gets its hands on this and begins to interpret. That is the trouble, if we don’t understand the context of the information than we can’t do a proper analysis. The numbers in of themselves are meaningless and it is our assumptions that will color the data.

    I just wrote about this on my personal blog, albeit not so eloquently. This is something that really concerns me about the startup world. Data is great, but don’t lose the humanity that lies just beyond the numbers.

  9. Sometimes I want to tell businesses that are tracking me what’s going on with me so they will understand the data better. For example, if I’m stressed I want to watch happy shows on Netflix, so I abandon House of Cards for Parks & Recreation for a while. Doesn’t mean I don’t love House of Cards.

  10. Om,

    Thanks for sharing your thoughts. I am a high school teacher, and for the past three years, all of education is frantic about “data”. We have to collect data, read data, and use data to make decisions. The reasoning is sound (the more you know, the more you can focus your teaching), but the individual student and the human-ness of teaching are left behind and lost.


  11. Insightful post. I hope Foursquare reads it- I’ve been thinking about the soulless aspect of tracking checkins lately .. this post nailed it. Adding the emotion/ story as you say would make for a more meaningful experience . One to hold on to over time. Like a personal archive. What really site point of these check-ins if not ?

  12. lovely post as always, Om

    continuing Nico: The “soul” of the data should be represented in… the vectors which could be found during data representation over human UI. ‘use data to shape experiences’ – is a motto for any marketer.

  13. Great, timely article.

    Using data to create wonderful experiences assumes that companies have at least two pieces in place: (1) a complete view of their customer and (2) a culture that will promote a customer centered view of the world.

    Most companies and prospects I work with first need to assemble that complete, cross-channel view of the customer. Typically customer data is stored across several silos—what are, in effect customer data prisons—and not organized to understand and predict customer behavior. Solving this problem is the first step.

    You also need senior managers with the vision to reorient their marketing strategy to be customer-centric. This means prioritizing the customer’s experience over quarterly numbers with the goal of driving long term growth.

    I’m optimistic these will come with time but expect continued frustration in the intervening period. 🙂

    Brian Ivanovick

  14. I always refer to this as quantification sans humanity. Lots of data is great, but if you don’t understand the real meta-data – the human context – the conclusions drawn are at once surprising and potentially perilous. As a thought exercise, imagine a data-driven evaluation of earth and its antagonists. If you set as your objective preserving the delicate balance that is life on this blue planet, it will not take long before you conclude (looking just at the data) that all the causal interactions suggest the single biggest non-Black Swan (ie asteroids, volcanos etc) risk is the presence of a widespread pestilence… aka humans. Let’s hope any advanced observers in the cosmos don’t compute that equation in an emotional vacuum.

  15. Matching big data with curation (actual human supervision and intervention in the data driven process) is perhaps the best way to avoid the worst aspects of Data Darwinism. Old fashioned magazines, newspapers, and television networks, all data driven yet “curated” by very smart human intervention, have done this in the legacy world for years with very good results. The modern development of content, blindly married to big data, cannot approach the quality of this older product. But it is early days yet, and eventually curation will routinely find its way back into the big data mix.

  16. Really great post- in a compelling and entertaining way, to me, it summed up one of the few key debates on big data. I work for UNDP and we’ve recently started looking into methods in which we can generate loads of data on people we work with so we can figure out how best to invest our resources to provide support, as well as layer it with context and stories… we’ve just started but it is incredibly exciting as it turns everything i’ve ever known about traditional research methods on its head and provides such rich information! Here’s a teaser http://bit.ly/YSQlYi
    thanks again for a great post!

  17. Good article, thanks. How come 99% of VC’s think non-automated parsing cannot “scale”?

    “parse the data in more granular manner and correlate it with other information and you start to see a picture that understands the emotional value of that data.”

    1. @AndyLeSavage,

      Thanks. It is not just VCs, most of the world assumes that the kind of parsing is not scalable. I am doing a follow up post to deal with exactly the same topic and hopefully will have comments from many experts after some reporting.

      1. @AndyLeSavage — it goes deeper than 99% of VCs views on non-automated parsing for emotions.

        The machines themselves can only auto-parse against known lexical libraries like Princeton’s Wordnet.

        Take a look at various sentiment analytics attempt to parse the emotions expressed across our social media. They reference against those known sources and It’s messy and noisy.

        The problem goes as far back as the earliest days of our species documenting our experiences (whether that was in the cave paintings, papyrus scrolls, Egyptian hieroglyphs, Bayeaux tapestry, Johnson’s dictionary, the Guttenberg press and Babbage’s machine).

        In any case, for the machine algorithms to “have soul” and be able to auto-parse emotions requires our society to completely overhaul code structure so that it’s more akin to organic chemical molecules rather than flat, logical, functional layers of code that build up to “Big Data”.

  18. Out of interest, Om, do you consider Amazon’s utilization of data as data **with a soul** or **without a soul**?

    I ask, because, while Amazon is one of the lesser aesthetically pleasing services in terms of emotive design, it’s arguably one of the most human services in terms of contexts and well-codified workflow that are bound to clear jobs and outcomes for the user.

    1. Totally without soul and they need to start thinking about the world beyond what they do right now on their website. I think the Amazon Kindle Store is much better experience as it is highly focused but could be more informed and exciting.

      1. Actually, that is a good point. My comments largely focused on Amazon website, as the mobile apps have a bolt-on, silo’d feel about them.

  19. Summary: Qualitative beats Quantitative 🙂

    Inside most companies (including Microsoft) as long as the graphs move towards the upper right for growth nobody stops to really get into why or whether the data is still valid. Breaking apart data is often a translation issue and can be easily done with a infographic visualisation (visual thinking).

    In the case above isolating the “why” for checkins is enough of a thread to start piecing together a story or visual thinking. In that custX keeps checkins to ACME Coffee Shop because Their Coffee Beans or Girl behind the counter is smoking hot (or both).. Yes its shallow but there’s no judgement just reality.

    Let the organic data tell the story is my point but dont try and shape it or panel beat into pre-defined patterns or templates that suites a global story… thats the trick here.

  20. Om,

    Data with head, heart and soul is what Senseus is all about:

    * http://www.senseus.co/blog/?p=58

    “Big Data” and their probability-quant approaches are not the panacea in our view.

    Da Vinci said, “All our knowledge has its origins in our perceptions” and so Senseus restores perceptions (head, heart and soul) to its rightful place in the way we decide why we buy into something — whether that’s coffee, content, relationships or experiences.

  21. One of the best post that i have read in some time on ‘data’.
    We collect data and use data effectively to enhance our experiences and tell stories. But this requires understanding relationships among disparate data items. And that is where the importance of Big Data really lies.
    A major transformation going on in society currently is the nature of story telling. At one time story telling was based on subjective experiences by an individual that were only qualitatively available to the storyteller. The last decade has seen the arrival of ‘data’ and objective story telling. Human memory is powerful but has its limitations. Subjective human memory results in powerful anecdotes. Add to this availability of large volumes of data and ability to attach it to strengthen those anecdotes and you got powerful and compelling factual stories.
    I recently wrote something on this topic: https://dl.dropbox.com/u/5644217/IEEE%20Multimedia%20Storytelling%20Jan%202013.pdf

    Once again, great article.

  22. Very well thought out piece, Om. Raw data needs to be made sense of, it needs to be interpreted to create a context. Context, that could help machines “understand” their job and realize things without explicit directives. Context would help humanize them, and not commit errors of sentiment as did Uber’s system during Sandy.

    I wrote an essay on similar lines a few weeks ago. You can find it here: http://spinhalf.net/context/

  23. Om,

    I quote “Data without soul is meaningless” on my site: http://senseus.co/#menu_overlay

    Senseus has a system to include the expression of emotions that’s different from vote up-down, 5-stars and Osgood sentiments which are used in existing Big Data practices.

    There are a couple of graphics at the bottom which explain this further.



Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.