Google's Infrastructure is its Strategic Advantage

87 thoughts on “Google's Infrastructure is its Strategic Advantage”

  1. Great article Om. The cool thing in both cases (Dell and Google) is that all this fast mechanism is hidden behind the scenes and transparent to the user. Dell has been the leader for quite a while but its competitors have in the meantime learned a lot, I expect the same to happen with Google sooner or later, probably later…

  2. I don’t infrastructure explains much of Google’s strength
    $3.8BN of investment over two years for a company that makes that much in Net Income in a year? Peanuts. If that was all it took to win in search, Microsoft and Yahoo! wouldn’t have to worry!

    Rather, there are entry barriers in terms of brand power (consumers think Google when they think search), and in terms of the liquidity and scale of their adsense market place (they can better monetise queries because they can offer more relevant ads)

  3. I would agree that Google’s infrastructure is indeed part of their competitive advantage, but the other part is the Google employees that built it. They have what is arguable the brightest group of people ever assembled for a publicly held company.

  4. I am excited to see how the new G*Drive project works. It’s great that they are offering these services and it will be fun to play around with yet another Free Google tool…

    The breadth of their projects really is the key to their success. They have branched into so many channels online that AdSense will continue to grow with and into these multiple channels.

  5. Om, have you already forgotten how they lost to YouTube (and had to acquire them) in spite of their 100-fold infrastructure advantage? If anything, video is one area where such an advantage would (a priori) appear overwhelming. YouTube didn’t have much by way of technology, so Google should have killed them on the infrastructure front – except that they didn’t.

    Facebook is not winning on any infrastructure advantage either.

    When you have large amounts of capital lying around, it seems like it could be used to erect an insurmountable advantage. But that is increasingly a myth. I offer Amazon S3 & EC2 as examples of leveling forces at work.

  6. @ Zhou,

    The YouTube if it was a stand alone company right now would be needing Facebook-style investments to support its growth. While YouTube got the brand, it is Google’s infrastructure that is eventually allow them to make money.

    Where is Facebook spending money? Infrastructure of course, and will continue to do so in the near future.

    The point of this article is simple: Google’s search-and-advertising business is where infrastructure is playing to their advantage. Last week while chatting with Jimmy Wales of Wikipedia, we talked about how Google is pressing this advantage and a new distributed approach to search (and hence the infrastructure) is the only way to compete with Google.

    Amazon S3 and Ec2 while great when you are small are going to become a huge cost factor as companies grow. Google is fine tuning to make its own business run ‘leaner & meaner’.

    That’s my two cents, but of course your opinions are equally valid.

  7. @ jeff and others

    I think you are right about Google’s employees and their software advantages. I think those are very well chronicled by much smarter folks than me, but I just wanted to bring up the “supply chain” analogy.

    Hopefully, I didn’t offend anyone for not pointing out the employee-software issue.

  8. Om,

    I disagree that Google’s infrastructure is its strategic advantage. If that were the case, Microsoft would have (and has) poured tons of money into infrastructure development and gained a strategic advantage for itself. However, Live search is still behind both Google and Yahoo.

    The strong Google brand associated with search and the strong advertisers network they have developed because of their high traffic has made it very difficult for others to catch up.

    For sure, they need to improve their algorithm and quality of search results going forward.

  9. Google’s success as a moneymaker is based on the ability to provide relevant ads to each page.

    Their search results differ less and less from those of Yahoo and MS Live, but because it takes only 20 days to form a new habit (of using Google), many have stuck to Google even though others’ results are as good (which they were not in the not-so-distant-past).

  10. @Om

    All of the Google employees I know are top notch without an exception. I am not claiming this reflects their entire workforce, but I know enough of them to say that I am more than impressed. I can only imagine what must be discussed behind closed doors at that company…or maybe I can’t. I’m probably not smart enough 🙂

  11. Om,
    Great post. As a counter-point, consider the situation in the semiconductor industry. Intel has the infrastructure/capital advantage just like Google. Yet, a whole industry of fabless companies have thrived. They depend on massive capex by the likes of TSMC, Chartered and so on.

    Financially speaking, debt markets have efficiently financed capital-intensive companies – at low interest rates, therefore requiring relatively low ROI – but they need low risk, which they achieve by aggregating the volumes of numerous fabless companies. Individual fabless companies would have much higher volatility (therefore need to have relatively debt-free balance sheets to withstand the ups and downs of the cycle) while the fabs trade lower risk for cheaper debt finance.

    My point is that both models will co-exist.

  12. Om —

    So glad to see you picking up on this thought. I made “operations as competitive advantage” a key part of the first O’Reilly Radar Executive Briefing back in 2006, and we’ve now launched a whole conference on the topic, called Velocity.

    Google is, of course, not the only company playing this card. What are Amazon’s S3 and EC2 but an operations play? And Microsoft understands this, even though they haven’t yet been able to execute on it as well as Google or Amazon. Last year, I quoted Windows Live VP of Operations Debra Chrapaty uttering one of the most profound (and as yet not fully appreciated) statements of the Web 2.0 era: “In the future, being a developer on someone’s platform will mean being hosted on their infrastructure.” It turns out that Ray Ozzie had said something very similar in a Fortune interview.

    So yes, this is the new game, and there are very few people who can afford to play it. Google is a master, but don’t count out Amazon or Microsoft, and maybe even Yahoo!, quite yet.

  13. Om:
    Great stuff. I agree infrastructure plays a bigger role in this ecosystem. However, don’t forget the biggest asset and true value to Google is their brightest geeks working round the clock churning out all those awesome applications.
    BTW,
    I found this funny and awesome web 2.0 bubble video, guess what, u feature in this video. If u have not watched it, take a look at it

    Hope u are not blogging with one hand and smoking with the other 🙂

    Cheers,
    Omfut

  14. Om

    as i have often said in the past – Google is (re/creating) the new Bell System. If the bell system had the best per-call cost of infrastructure, Google has the best per-bit compute+store+delivery and continues to leverage this infrastructure as a key differentiator.

    Meanwhile, telecom is mired in the imagined struggles of years past, even as Google is putting giga-bit-miles between its own network and everyone else’s. In a few years, I bet ATT, VZ, and others tending to their previously captive landline and wireless businesses are going to be woefully out of touch with the customer needs and out of step with where technology is going (see) as another example. In decades past they had the luxury of matching their snail’s pace of evolution with the equally slow rate of networking technology growth. Analog transmission and switching (manual + strowger) lasted five decades, digital transmission and finally SONET took another few decades (1960s to 1990s).

    The future is not looking as kind. The telcos today believe that they ‘own’ the customer by virtue of owning the somewhat broadband pipes that link the customer with the network. I believe what really matters to the customer are the various ‘clouds’ of interest – Google, Yahoo, Amazon, Ebay, and Microsoft and perhaps someone like Facebook (though one never hears of facebook excelling at infrastructure) and the services possible in the clouds. Connectivity is hardly the value-added piece of the chain.

    I wonder if any decision maker at the telcos even knows what memcached, lucene, hadoop, GFS, mapreduce are and how these technologies influence fundamental decisions about internet data center switching and resultant network architecture.

    As enterprise data centers morph from legacy architectures to mimicking internet data centers and applications/services out of Blue or Google clouds become pervasive, lessons learnt from Google Infrastructure engineering may just become standard practice across all kinds of networks. Are the telco engineers taking notice?

  15. It is clear that Google has decided that for long term success infrastructure is a competitive advantage. It is also clear, based on this and other moves, that Google uses commodity hardware and open source software to a point. When it comes to gaining a leg up, they do their own thing, often on top of existing technology.

    Will these investments pay off in the long run? Only time will tell, but my bias is towards companies that have a solid foundation, especially one that is a competitive advantage. My money is on Google. Now they have to figure out how to go beyond search and advertising and take advantage of this competitive advantage.

    Parul
    http://www.bhopu.com

  16. great post, totally on point..

    the next question is why do even need the internet? once they have everything served up from their own infrastructure and massive supercomputer?

  17. Google only lacks Access and ownership of the Last Mile, which it has a shot at with the 700Mhz Spectrum to dominate the Broadband Distriubtion networks in the US and (future) world markets. When and if it gets the spectrum it will be able to create a Centrally managed end to end High Speed, QoS based, Low Latency distribution network for its existing search as well as an entirely new series of Content/Applications (Hosted Apps).
    What this is is a replacement for much of the Best Effort Internet that many Service Providers will link (using fiber)to in order to provide the Premium Level Broadband Services and access to content they need.
    Examples of new Content/Apps that Google can provide out of their Data Centers Link to SP via Fiber:
    1. MultiPlayer Gaming Network-that relishes the Low Latency and serious Broadband links this Network can provide-This is massive revenue service with excellent Ad revenue opps.

    1. TV Programming Distribution Source, replacing the Satellite based distribution using Aggregator-Like TNN the MSO use today-At very High fees- % of Revenues). Why use Satellite when they can connect and route everything over their Fiber backbone and connect to the CATV headends via fiber. Seriously reducing cost (CAPEX & OPEX) to install and operate a Headend.

    2. Replacement of the CDN like Akamai who specialize in Content delivery.

    3. VoiceIP Network: Its low latency excellent QoS (precedence capability), security as well as the 24X7 Mangement of the Network will dramatically improve the end to end delivery of the IPVoice links. This will also reduce the costs to the Local Incumbent to deply a new Softswitch/Feature Server (VoiceiP based) Network and lower LD costs Worldwide.

    Many more like this but these are big ones.

    Jim (aka Jacomo)

  18. @ Parag,

    Microsoft and Live Search may benefit from money being poured in to it, but if they are stacking bricks, a concorde they do not get…

    For more on Microsoft Research’s ideas on Rethinking Data Centers, see powerpoint from Chuck Thacker’s talk at Stanford in Oct
    http://yuba.stanford.edu/~nbehesht/netseminar/seminars/10_25_07.ppt

    From his talk it is clear that what is ‘practice’ at Google and others is merely ‘talk’ @ Microsoft.

    rohit

  19. Hey Om, Great post. I heard awhile ago that Google owned fiber, but didn’t know how it fit into the picture. If infrastructure is really their game, I’m wondering if we could apply your thinking to what they are doing in wireless? You can see my post on the subject here.

  20. Om,

    I enjoy your articles a lot but for this one I will take you to task. First, I am amazed that you would quote “0.12-0.06” as what it takes to do a search. Have you ever checked your TCP/IP roundtrip delay (e.g. by ping)? I got 100+ ms a minute ago from my FIOS, now it is 70, this is the floor for the delay, it makes no sense to talk about 60 ms delay when it becomes dominated by transport.

    The other point is that the speed is simply about perception, once you get to fractions of seconds nobody can notice anyway.

    Google has always played the speed card, and I agree, it is cool but is no big deal these days, you can do really fast search with amazingly small resources (hint: see Gigablast).

    The fact is, despite oohhing and aahhing by mesmerized onlookers and assorted Wallstreet types, Google search results have barely changed from 10 years ago. The quality of a multitude of results is visibly deteriorating as they are being search spammed to death, by tens of thousands of COMPANIES, not to mention legions of individuals . This is hardly a secret as it is being openly discussed among search experts.

    It is true that Google has amassed enormous infrastructure and they are playing the card of giving an impression that all that is absolutely REQUIRED since they keep mum about what it is being used for. But how about how much of that is really needed, and how efficiently it is deployed?

    The size of the crawlable Web is small, 20B pages will fit into 200TB which today one can jam into a couple of racks for two hundred grand.
    Sure you want to back it up, cache it, have copies for speedup etc but that hardly an imposing infrastructure make.

    As for bandwidth, that is really funny since for $100K/mo one can get 10Gbps in the Bay Area, you can crawl 10B pages/DAY with that, well, nobody dares to try that incl Goggle, Yahoo and Microsoft. The moral of the story is that crawling DOES NOT SCALE! Google probably spends more that $100K/mo on coffee so crawling is certainly not any kind of “infrastructure advantage”.

    The fact is, IMHO, that Google has been getting a free ride as they have not been faced with serious competition (from Microsoft, Yahoo nor anyone else). Couple that with a perception, I would say perfectly exemplified by your article, that it takes some mythical, enormous resources to do search and presto, we get Google invincibility.

    There are other claims you make in there, e.g. “indexing the Internet really well”. What is so special about that? It is well known that indexing is the game of RAM i.e. if you stuff your index into RAM then it will be really fast and that is it. The size of a really large Internet index, with all the languages words, product names, acronyms, people names, parts of URLs and whatever else these days is few billions (Google actually admits that in one of their earlier papers), throw in few more billion for N-grams (most frequent combinations of 2,3,4 .. keywords, they have actually released these numbers) and you end up with, say, tens of TB of RAM. The cost of RAM itself is not much few hundred grand, perhaps few million for machines to stuff the RAM in but what is the big deal about it?

    You also mention “imagine connecting hosts with 10Gbps and any-to-any connections”. You seem to be confusing the connectivity speed with throughput of the switching fabric. But even with the full crossbar, what is the point? Will that help Google rotate their index faster? Are Google customers happy with the current rotation frequency? Will it allow them to have say, incremental index, instead of the current, write-once one?

    OK, I will stop here. Do no get me wrong, I like your stuff, just thought to give you a little workout on this one 🙂

    Finally, I guess as a disclaimer too, one can say if I know and say all this, what am I doing about it? Fair question, I have experience (founded two search startups) and resources now, and am doing something about it, will have more to say pretty soon 🙂

  21. Om,

    Interesting post – thanks! But fwiw (and I know it’s peripheral to your main point), the premise at the beginning is incorrect. Dell’s initial breakthrough wasn’t in supply chain, it was in distribution. As “PCs Etc.”, it didn’t have enough clout to elbow its way onto retail shelves, so it developed a different way of reaching customers. The supply chain stuff came later…

  22. I’ve done a lot of investigation of Google’s infrastructure. For more on software and hardware on Google check out:

    For Microsoft’s best idea, look at this article on a Microsoft Research project named Boxwood. Honestly though, I don’t think Ballmer gets it. Ozzie maybe.

    Robin Harris

  23. Interesting read, Om. While Goog’s infrastructure is a competitive advantage, I would argue that it is not Google’s only one. Otherwise, it would be in Google’s best interest to collect the Ricardian rent on it by allowing others to build on it.

    I believe Google’s strengths run deeper. I suspect the infrastructure they’ve built out is closely aligned to how they carry out their other activities – process of developing new products, their software stack, etc. The intertwining of these different activities make Google’s infrastructure more valuable to itself than to others (at least for the present), and, in my mind, justifies Google’s experiments to churn out its dizzying array of products.

    The more interesting question is how sustainable is the advantage?

  24. hi everyone,,,
    i have a question regarding googles business,just a bit curious as we were having a discussion in the class that does google get paid for the search result or they get paid only for the ads and sponsored link when people click on them?i mean if i search for the term “economy” and they provide me with few pages of search result and i click on one of them,,,do googleget paid?
    plz let me know if anyone knows it,,,
    thx in advance

  25. I can agree with most of this article, except the part apart their search results. Every now and again, I will jump on bing or yahoo and they just dont give me as many quality results. Google is just better. And Now they have picked up their real time search game adding twitter and the like. Type in Los Angeles weather in the search bar, you get the results before your done typing. No need to even hit enter. You can’t get those type of results anywhere else. So now its there job to keep innovating.

This site uses Akismet to reduce spam. Learn how your comment data is processed.