Back in the day, when PC stocks were kings on Wall Street, a pesky college kid named Michael Dell figured out that he could do an end run around the then-established PC makers by developing a smarter way of making and selling boxes. His strategy was simple: get components and PCs from the factories in Asia to the U.S. as fast as possible, but only after he had charged for the machine.
By squeezing the supply chain as hard as he could, he turned Dell into a fearsome (and loathsome) competitor. With his help, the supply chain for the PC era came to consist of foundries, ships, U.S. assembly plants and UPS trucks. Google (GOOG), with over $200 billion in market capitalization, is following a similar strategy, fine tuning and adapting it for the Web & broadband.
Instead of trucks and assembly plants, however, Google’s supply chain is made up of fiber networks, data centers, switches, servers and storage devices. From that perspective, its business model is no different than that of Dell’s (DELL): Google has to deliver search results (information, if you want to be generous about their other projects) as fast as possible at as low a cost as possible.
To better understand Google and its business model, one needs to break it down into three data inputs.
- Relevancy of results.
- Speed of search.
- Cost of executing a search query.
While their results aren’t optimal, they are good enough. Just like Microsoft Windows was good enough to dominate the market. Google, according to Hitwise, now has 64 percent of the total search market. And although a typical Google query can often be an act of futility, we put up with it because the results are fast. If they’re wrong, we can just start all over again.
The faster the results show up on our browsers, the less inclined we’ll be to switch to a rival search engine, no matter how great the rival’s search methodology may be. The faster (and more efficient) its infrastructure, the more easily Google can keep serving the ad-based money machine.
In other words, the company has to make sure that the speed of its search is really, really fast. Any random search on Google these days takes between 0.12 to 0.06 seconds. Now that is really, really fast. Google does this by indexing the Internet quite well. The magic is in delivering the search results from this index at lightening speed, and that requires an infrastructure — oodles of bandwidth and specialized hardware — that is finely tuned, much like a Formula One Car.
Against this backdrop, it makes perfect sense for Google to build their own servers, storage systems, Internet switches and perhaps, sometime in the future, even optical transport systems. Let me rephrase that: Imagine connecting thousands of hosts (storage and server systems) at speeds of, say, 10 gigabits per second, in a manner that allows any-to-any connections.
The number of racks, fiber, routers and everything in between is mind-boggling. If this system were built using gear from established hardware makers, it would take a superhuman effort to make it all work together. In other words, the sheer cost to keep such a beast going would suck up a major component of the infrastructure.
A better option is to have gear that is customized for your processes, ones in which you have a major operational expenditure advantage. In the telecom bubble, large service providers were brought to their knees by operational expenditures.
With the exception of optical systems, Google has built or is building the gear. It has been rumored to be a big buyer of dark fiber to connect its data centers, which helps explain why the company spent nearly $3.8 billion over the past seven quarters on capital expenditures.
You can argue that building customized gear is an expensive strategy, but when you are the scale of Google, it starts to become less of an issue. Why? Because process-optimized infrastructure ensures that Google’s cost of executing a query keep going down.
To sum it up, Google’s gigantic infrastructure is the big barrier to entry for its rivals, and will remain so, as long as the company keeps spending billions on it. That said, there’s another thing Google could learn from Dell: Maintain the quality of your search results — customers will only put up with shoddiness for so long.
Note #1: Ethan, you are absolutely right about the software aspect of Google architecture, and I was going to do a separate post. This one is already 750 words.
Note #2: Earth2Tech has a post about Google’s vertically integrated green energy strategy.
Great article Om. The cool thing in both cases (Dell and Google) is that all this fast mechanism is hidden behind the scenes and transparent to the user. Dell has been the leader for quite a while but its competitors have in the meantime learned a lot, I expect the same to happen with Google sooner or later, probably later…
You can compare this entry deterence strategy as a “building excess capacity” strategy. As economic literature goes, this is a very credible threat to deter entrants.
Google’s search algorithms blow me away. The speed is mind-boggling considering the task. Go Google!
I don’t infrastructure explains much of Google’s strength
$3.8BN of investment over two years for a company that makes that much in Net Income in a year? Peanuts. If that was all it took to win in search, Microsoft and Yahoo! wouldn’t have to worry!
Rather, there are entry barriers in terms of brand power (consumers think Google when they think search), and in terms of the liquidity and scale of their adsense market place (they can better monetise queries because they can offer more relevant ads)
I would agree that Google’s infrastructure is indeed part of their competitive advantage, but the other part is the Google employees that built it. They have what is arguable the brightest group of people ever assembled for a publicly held company.
Excellent article, Om. It really shows that Google has the lead in not just infrastructure, but also strategy. I agree with you, though, their search results need to improve in quality! Speed is important, as long as they continue generating money, they can keep improving their infrastructure and becoming more efficient: http://fishtrain.com/2007/08/30/googles-trading-floor/
I am excited to see how the new G*Drive project works. It’s great that they are offering these services and it will be fun to play around with yet another Free Google tool…
The breadth of their projects really is the key to their success. They have branched into so many channels online that AdSense will continue to grow with and into these multiple channels.
Do you see Google’s push into renewable energy as part of its drive into building infrastructure? Or is that more of just a pet project of Eric Schmidt’s growing interest in green technology?
Wow. This blog is so much better than Techcrunch. You guys actually have well thought out posts. Awesome. Finally, an alternative.
Om, have you already forgotten how they lost to YouTube (and had to acquire them) in spite of their 100-fold infrastructure advantage? If anything, video is one area where such an advantage would (a priori) appear overwhelming. YouTube didn’t have much by way of technology, so Google should have killed them on the infrastructure front – except that they didn’t.
Facebook is not winning on any infrastructure advantage either.
When you have large amounts of capital lying around, it seems like it could be used to erect an insurmountable advantage. But that is increasingly a myth. I offer Amazon S3 & EC2 as examples of leveling forces at work.
@ Zhou,
The YouTube if it was a stand alone company right now would be needing Facebook-style investments to support its growth. While YouTube got the brand, it is Google’s infrastructure that is eventually allow them to make money.
Where is Facebook spending money? Infrastructure of course, and will continue to do so in the near future.
The point of this article is simple: Google’s search-and-advertising business is where infrastructure is playing to their advantage. Last week while chatting with Jimmy Wales of Wikipedia, we talked about how Google is pressing this advantage and a new distributed approach to search (and hence the infrastructure) is the only way to compete with Google.
Amazon S3 and Ec2 while great when you are small are going to become a huge cost factor as companies grow. Google is fine tuning to make its own business run ‘leaner & meaner’.
That’s my two cents, but of course your opinions are equally valid.
@ jeff and others
I think you are right about Google’s employees and their software advantages. I think those are very well chronicled by much smarter folks than me, but I just wanted to bring up the “supply chain” analogy.
Hopefully, I didn’t offend anyone for not pointing out the employee-software issue.
Om,
I disagree that Google’s infrastructure is its strategic advantage. If that were the case, Microsoft would have (and has) poured tons of money into infrastructure development and gained a strategic advantage for itself. However, Live search is still behind both Google and Yahoo.
The strong Google brand associated with search and the strong advertisers network they have developed because of their high traffic has made it very difficult for others to catch up.
For sure, they need to improve their algorithm and quality of search results going forward.
Thanks for the shout-out, Om. I look forward to your piece on the software side of the G house – this article was great stuff.
Google’s success as a moneymaker is based on the ability to provide relevant ads to each page.
Their search results differ less and less from those of Yahoo and MS Live, but because it takes only 20 days to form a new habit (of using Google), many have stuck to Google even though others’ results are as good (which they were not in the not-so-distant-past).
great article. i think the key message for those looking to build “the next big thing” is that investing in infrastructure is NOT the way to go — you’ll end up running right into google.
@Om
All of the Google employees I know are top notch without an exception. I am not claiming this reflects their entire workforce, but I know enough of them to say that I am more than impressed. I can only imagine what must be discussed behind closed doors at that company…or maybe I can’t. I’m probably not smart enough 🙂
Om,
Great post. As a counter-point, consider the situation in the semiconductor industry. Intel has the infrastructure/capital advantage just like Google. Yet, a whole industry of fabless companies have thrived. They depend on massive capex by the likes of TSMC, Chartered and so on.
Financially speaking, debt markets have efficiently financed capital-intensive companies – at low interest rates, therefore requiring relatively low ROI – but they need low risk, which they achieve by aggregating the volumes of numerous fabless companies. Individual fabless companies would have much higher volatility (therefore need to have relatively debt-free balance sheets to withstand the ups and downs of the cycle) while the fabs trade lower risk for cheaper debt finance.
My point is that both models will co-exist.
Google’s total dominance in search is a disaster for small business. Untold hours are wasted on trying to figure out how to improve SERPs and PageRank instead of on building the business. Is Google Really Good for Small Business?:
http://smartstartup.typepad.com/my_weblog/2007/07/a-fable-doing-b.html
Om —
So glad to see you picking up on this thought. I made “operations as competitive advantage” a key part of the first O’Reilly Radar Executive Briefing back in 2006, and we’ve now launched a whole conference on the topic, called Velocity.
Google is, of course, not the only company playing this card. What are Amazon’s S3 and EC2 but an operations play? And Microsoft understands this, even though they haven’t yet been able to execute on it as well as Google or Amazon. Last year, I quoted Windows Live VP of Operations Debra Chrapaty uttering one of the most profound (and as yet not fully appreciated) statements of the Web 2.0 era: “In the future, being a developer on someone’s platform will mean being hosted on their infrastructure.” It turns out that Ray Ozzie had said something very similar in a Fortune interview.
So yes, this is the new game, and there are very few people who can afford to play it. Google is a master, but don’t count out Amazon or Microsoft, and maybe even Yahoo!, quite yet.
Om – Great article!
Does anyone know the cost of serving a query in Google and how does it compare against the query serving or ad serving for other companies?
Om:
Great stuff. I agree infrastructure plays a bigger role in this ecosystem. However, don’t forget the biggest asset and true value to Google is their brightest geeks working round the clock churning out all those awesome applications.
BTW,
I found this funny and awesome web 2.0 bubble video, guess what, u feature in this video. If u have not watched it, take a look at it
Hope u are not blogging with one hand and smoking with the other 🙂
Cheers,
Omfut
Om
as i have often said in the past – Google is (re/creating) the new Bell System. If the bell system had the best per-call cost of infrastructure, Google has the best per-bit compute+store+delivery and continues to leverage this infrastructure as a key differentiator.
Meanwhile, telecom is mired in the imagined struggles of years past, even as Google is putting giga-bit-miles between its own network and everyone else’s. In a few years, I bet ATT, VZ, and others tending to their previously captive landline and wireless businesses are going to be woefully out of touch with the customer needs and out of step with where technology is going (see) as another example. In decades past they had the luxury of matching their snail’s pace of evolution with the equally slow rate of networking technology growth. Analog transmission and switching (manual + strowger) lasted five decades, digital transmission and finally SONET took another few decades (1960s to 1990s).
The future is not looking as kind. The telcos today believe that they ‘own’ the customer by virtue of owning the somewhat broadband pipes that link the customer with the network. I believe what really matters to the customer are the various ‘clouds’ of interest – Google, Yahoo, Amazon, Ebay, and Microsoft and perhaps someone like Facebook (though one never hears of facebook excelling at infrastructure) and the services possible in the clouds. Connectivity is hardly the value-added piece of the chain.
I wonder if any decision maker at the telcos even knows what memcached, lucene, hadoop, GFS, mapreduce are and how these technologies influence fundamental decisions about internet data center switching and resultant network architecture.
As enterprise data centers morph from legacy architectures to mimicking internet data centers and applications/services out of Blue or Google clouds become pervasive, lessons learnt from Google Infrastructure engineering may just become standard practice across all kinds of networks. Are the telco engineers taking notice?
It is clear that Google has decided that for long term success infrastructure is a competitive advantage. It is also clear, based on this and other moves, that Google uses commodity hardware and open source software to a point. When it comes to gaining a leg up, they do their own thing, often on top of existing technology.
Will these investments pay off in the long run? Only time will tell, but my bias is towards companies that have a solid foundation, especially one that is a competitive advantage. My money is on Google. Now they have to figure out how to go beyond search and advertising and take advantage of this competitive advantage.
Parul
http://www.bhopu.com
@ Rohit,
Lots to talk about I guess 🙂
@ Omfut,
That is one funny video. Though the photo of me has a really old phone – not so good.
great post, totally on point..
the next question is why do even need the internet? once they have everything served up from their own infrastructure and massive supercomputer?
Informative and insightful. Thanks, Om
Dell and Google both understand and demonstrate that critical behavior that more companies need to embrace–adapt or die.
Something to think about: Google is probably one of the top three server builders in the world, the catch is they don’t sell to anyone else.
Google only lacks Access and ownership of the Last Mile, which it has a shot at with the 700Mhz Spectrum to dominate the Broadband Distriubtion networks in the US and (future) world markets. When and if it gets the spectrum it will be able to create a Centrally managed end to end High Speed, QoS based, Low Latency distribution network for its existing search as well as an entirely new series of Content/Applications (Hosted Apps).
What this is is a replacement for much of the Best Effort Internet that many Service Providers will link (using fiber)to in order to provide the Premium Level Broadband Services and access to content they need.
Examples of new Content/Apps that Google can provide out of their Data Centers Link to SP via Fiber:
1. MultiPlayer Gaming Network-that relishes the Low Latency and serious Broadband links this Network can provide-This is massive revenue service with excellent Ad revenue opps.
TV Programming Distribution Source, replacing the Satellite based distribution using Aggregator-Like TNN the MSO use today-At very High fees- % of Revenues). Why use Satellite when they can connect and route everything over their Fiber backbone and connect to the CATV headends via fiber. Seriously reducing cost (CAPEX & OPEX) to install and operate a Headend.
Replacement of the CDN like Akamai who specialize in Content delivery.
VoiceIP Network: Its low latency excellent QoS (precedence capability), security as well as the 24X7 Mangement of the Network will dramatically improve the end to end delivery of the IPVoice links. This will also reduce the costs to the Local Incumbent to deply a new Softswitch/Feature Server (VoiceiP based) Network and lower LD costs Worldwide.
Many more like this but these are big ones.
Jim (aka Jacomo)
@ Parag,
Microsoft and Live Search may benefit from money being poured in to it, but if they are stacking bricks, a concorde they do not get…
For more on Microsoft Research’s ideas on Rethinking Data Centers, see powerpoint from Chuck Thacker’s talk at Stanford in Oct
http://yuba.stanford.edu/~nbehesht/netseminar/seminars/10_25_07.ppt
From his talk it is clear that what is ‘practice’ at Google and others is merely ‘talk’ @ Microsoft.
rohit
just cuz of this post, i will keep coming here rather than techcrunch.
Hey Om, Great post. I heard awhile ago that Google owned fiber, but didn’t know how it fit into the picture. If infrastructure is really their game, I’m wondering if we could apply your thinking to what they are doing in wireless? You can see my post on the subject here.
Om,
I enjoy your articles a lot but for this one I will take you to task. First, I am amazed that you would quote “0.12-0.06” as what it takes to do a search. Have you ever checked your TCP/IP roundtrip delay (e.g. by ping)? I got 100+ ms a minute ago from my FIOS, now it is 70, this is the floor for the delay, it makes no sense to talk about 60 ms delay when it becomes dominated by transport.
The other point is that the speed is simply about perception, once you get to fractions of seconds nobody can notice anyway.
Google has always played the speed card, and I agree, it is cool but is no big deal these days, you can do really fast search with amazingly small resources (hint: see Gigablast).
The fact is, despite oohhing and aahhing by mesmerized onlookers and assorted Wallstreet types, Google search results have barely changed from 10 years ago. The quality of a multitude of results is visibly deteriorating as they are being search spammed to death, by tens of thousands of COMPANIES, not to mention legions of individuals . This is hardly a secret as it is being openly discussed among search experts.
It is true that Google has amassed enormous infrastructure and they are playing the card of giving an impression that all that is absolutely REQUIRED since they keep mum about what it is being used for. But how about how much of that is really needed, and how efficiently it is deployed?
The size of the crawlable Web is small, 20B pages will fit into 200TB which today one can jam into a couple of racks for two hundred grand.
Sure you want to back it up, cache it, have copies for speedup etc but that hardly an imposing infrastructure make.
As for bandwidth, that is really funny since for $100K/mo one can get 10Gbps in the Bay Area, you can crawl 10B pages/DAY with that, well, nobody dares to try that incl Goggle, Yahoo and Microsoft. The moral of the story is that crawling DOES NOT SCALE! Google probably spends more that $100K/mo on coffee so crawling is certainly not any kind of “infrastructure advantage”.
The fact is, IMHO, that Google has been getting a free ride as they have not been faced with serious competition (from Microsoft, Yahoo nor anyone else). Couple that with a perception, I would say perfectly exemplified by your article, that it takes some mythical, enormous resources to do search and presto, we get Google invincibility.
There are other claims you make in there, e.g. “indexing the Internet really well”. What is so special about that? It is well known that indexing is the game of RAM i.e. if you stuff your index into RAM then it will be really fast and that is it. The size of a really large Internet index, with all the languages words, product names, acronyms, people names, parts of URLs and whatever else these days is few billions (Google actually admits that in one of their earlier papers), throw in few more billion for N-grams (most frequent combinations of 2,3,4 .. keywords, they have actually released these numbers) and you end up with, say, tens of TB of RAM. The cost of RAM itself is not much few hundred grand, perhaps few million for machines to stuff the RAM in but what is the big deal about it?
You also mention “imagine connecting hosts with 10Gbps and any-to-any connections”. You seem to be confusing the connectivity speed with throughput of the switching fabric. But even with the full crossbar, what is the point? Will that help Google rotate their index faster? Are Google customers happy with the current rotation frequency? Will it allow them to have say, incremental index, instead of the current, write-once one?
OK, I will stop here. Do no get me wrong, I like your stuff, just thought to give you a little workout on this one 🙂
Finally, I guess as a disclaimer too, one can say if I know and say all this, what am I doing about it? Fair question, I have experience (founded two search startups) and resources now, and am doing something about it, will have more to say pretty soon 🙂
Great post. It helped me to validate my recent blog post.
http://www.livrona.com/blog/technology/clash-technology-titans-recap
Google both understand and demonstrate that critical behavior that more companies need to adapt or die.
Very well written! Thanks, Om
Om,
Interesting post – thanks! But fwiw (and I know it’s peripheral to your main point), the premise at the beginning is incorrect. Dell’s initial breakthrough wasn’t in supply chain, it was in distribution. As “PCs Etc.”, it didn’t have enough clout to elbow its way onto retail shelves, so it developed a different way of reaching customers. The supply chain stuff came later…
Google should go green with some serious virtualization. Here’s a good article:
http://www-03.ibm.com/systems/virtualization/view/071807.html
I’ve done a lot of investigation of Google’s infrastructure. For more on software and hardware on Google check out:
For Microsoft’s best idea, look at this article on a Microsoft Research project named Boxwood. Honestly though, I don’t think Ballmer gets it. Ozzie maybe.
Robin Harris
Interesting read, Om. While Goog’s infrastructure is a competitive advantage, I would argue that it is not Google’s only one. Otherwise, it would be in Google’s best interest to collect the Ricardian rent on it by allowing others to build on it.
I believe Google’s strengths run deeper. I suspect the infrastructure they’ve built out is closely aligned to how they carry out their other activities – process of developing new products, their software stack, etc. The intertwining of these different activities make Google’s infrastructure more valuable to itself than to others (at least for the present), and, in my mind, justifies Google’s experiments to churn out its dizzying array of products.
The more interesting question is how sustainable is the advantage?
great post, totally on point..
the next question is why do even need the internet? once they have everything served up from their own infrastructure and massive supercomputer?
hi everyone,,,
i have a question regarding googles business,just a bit curious as we were having a discussion in the class that does google get paid for the search result or they get paid only for the ads and sponsored link when people click on them?i mean if i search for the term “economy” and they provide me with few pages of search result and i click on one of them,,,do googleget paid?
plz let me know if anyone knows it,,,
thx in advance
I can agree with most of this article, except the part apart their search results. Every now and again, I will jump on bing or yahoo and they just dont give me as many quality results. Google is just better. And Now they have picked up their real time search game adding twitter and the like. Type in Los Angeles weather in the search bar, you get the results before your done typing. No need to even hit enter. You can’t get those type of results anywhere else. So now its there job to keep innovating.