Open Letter: Why Cloud Business Needs Better Metrics

Steve Ballmer, the former chief executive officer of Microsoft and the current owner of the L.A. Clippers, has a lot to say about Microsoft — especially about the Microsoft Azure cloud business. In an interview with Bloomberg, he argued that since the company was betting big on the cloud, it should report revenues and profits and not just the run rate, which he said was sort of bubkes! He argued that since software has higher margins than hardware, it is important for shareholders (he is a big shareholder) to see information around margins.

Ballmer is right: Revenue and margins are the most important metrics. But there are also several other metrics pertaining the cloud that need to be standardized and reported.

The Impact of the Cloud

I have tracked the cloud since its emergence in the early 2000s. It was pretty obvious even in the those days that the cloud wasn’t as much of a technical shift as it was an ideological, philosophical and economic shift that would reimagine how we consumed technology resources. This belief led to the creation of the Structure cloud conference in 2008.

Even though the hype around cloud startups and investments has ebbed, the cloud ideology is now an essential part of the modern ecosystem. Hardly any new product or offering is untouched. Just look at Amazon Web Services’ remarkable growth within a decade of its existence and you can see the cloud’s impact. Of course, growing popularity has also come with marketing malarkey and what we old tech nerds affectionately call FUD. Oracle woke up, rubbed its eyes and yelled, “We got clouds!”

Another company that likes to tout its cloud credentials is IBM, which has always been good at issuing press releases but less so at actually having a major impact on the cloud. I asked my Twitter followers about the possible impact IBM could make on the cloud, and a whopping majority of 85 percent said it will have a marginal, minimal or no impact. Only 16 percent of the 470 who voted thought IBM will have an impact.

The public cloud for now is a three-horse race, and Amazon leads by a mile. Amazon recently started to report its AWS-specific revenues and margins because it is in the driver’s seat and also because AWS is giving a boost to the overall company’s bottom line. During the fourth quarter of 2015, AWS revenues were about $2.41 billion, a 69 percent boost over the same quarter in 2014. AWS has revenues of almost $10 billion a year.

It is rumored that the only way for Google, to catch up: Offer steep discounts on the overall service and in some cases offer it for free. The recent deal with Spotify is rumored to be one of those free-ish deals, as my sources tell me.

Standardize Those Metrics Already

Either way, I think is time for the industry to come up with a standardized set of growth and usage metrics from those who say they offer cloud services. In order to qualify as a public cloud service provider, the company should offer compute, storage, networking and memory in a flexible, on-demand basis and allow for fractional payment. At present, I believe that AWS, Azure and Google Cloud are three public cloud companies that offer those at scale. Here are the growth and usage metrics that I would propose we require:

1. Total storage capacity and the percentage of the total capacity being used. In addition, it would be good to see sequential quarterly growth. It would be important to track the growth curve of the persistent user data for the storage used per customer per month, excluding replication factor and ephemeral or scratch storage usage. This will prevent some gaming of stats.
2. Total compute capacity, alongside the percentage of the total capacity being used as well as sequential quarterly growth. It would be important to track the compute cycles per customer per month.
3. Total developers using the cloud platform. For example, Azure and the quarterly growth in the developer community.
4. Total i/o volume to quantify the amount of data moving between the cloud and us, as a way to measure the network utilization. It is important to measure latency and robustness of the network. Network performance measurement has to be part of the equation, especially when looking at the emerging workloads per application such as the recent Google Cloud Vision API service, which need higher network performance.
5. Total number of platform-owned apps and how much of the total usage is from platform apps, as a separate report. In addition, there should be a total install base and daily active usage in compute terms.

I admit that the usage of disk and growth can be doctored by playing pricing games, but it isn’t easy, as Google has found out the hard way. The number of developers using the platform is a good indicator of a platform’s health. It points to future growth and revenue and profit prospects.

But the most important metrics that every cloud platform needs to report are what Ballmer focused on: revenues and margins. It is time for Microsoft, Google and Amazon cloud to report:

• Revenues per quarter
• Gross margins per quarter
• Per customer quarterly revenue

Given that these are all three public companies, they shouldn’t shy away from revealing data that allow shareholders to understand their prospects. Similarly, if IBM and Oracle want to be part of the cloud conversation, they should also report such numbers. I am sure global players like Baidu and Alibaba would love to participate.

So far we have not asked companies to report these metrics, and as a result we have an ambiguous understanding of the market and the landscape. It is time for the three leaders — Amazon, Microsoft and Google — to step up and set the standards.

P.S. For now consider this note as a working draft. Your input is welcome!

Updates: Following were the reactions to the forementioned piece, including mentions on Twitter and Facebook.

• Steven Sinofsky formerly of Microsoft and more recently of A16Z, did a tweetstorm which is worth a few minutes of your time. He posted the final version of it on his Medium page.
• Greg Knieremen on my Facebook page said: “I would only add that it would be useful to understand average customer usage of storage and compute *in production* (as opposed to the large number that use public cloud for test/dev). I think this would provide enterprises some context as to how much is actually deployed in production.”
• Will Charnock on my Facebook page said: “Interesting take – but really those metrics mean very little except to help quantify the “size” of the cloud platforms which might help analysts in determining market share, etc, but don’t really help a consumer make an educated decision about which cloud they should choose. But yes – some set of standardized metrics would be nice.”
• Fortune magazine’s Barb Darrow wrote: “One thing many cloud watchers (including blogger Om Malik and yours truly) would like now is a uniform way to compare and contrast the price and usage of given cloud resources between vendors. Here’s hoping that happens in our lifetime.”
• Shira Ovide of Bloomberg on Twitter: “I’m with @om. REAL cloud numbers, please.”
• Nitasha Tiku of Buzzfeed on Twitter: “True point fr @Om: why don’t publicly-traded companies like Amazon/Microsoft/Goog report real #’s on cloud computing.”
• Charles Fitzgerald on Twitter: “Would certainly make things easier for those of us watching the horse race, but that level of transparency is never going to happen.”