Silicon Valley and the Talent Crunch

82 thoughts on “Silicon Valley and the Talent Crunch”

  1. A big part of this is the fact that startups insist on hiring developers with plug-and-play skill sets, no matter which edge-case languages and architectures they have chosen. The cost of hiring people with non-commodity skills doesn’t factor into their thinking very often.

    Hiring smart people and investing in them through training (as opposed to waiting around for an expensive plug-and-play candidate to show up) seems like a better strategy.

    1. Jeffrey

      Thanks for your thoughts here. I think the problem is that as you train people, there is also the element of people just leaving when they are “baked” and many folks I talk to, complain of that.

      By the way, if you have any thoughts on how to hire smarter, share them with us.

      1. I agree with Om here, but I think if we do spot potential, we should also make it as part of the offer to incentivize these folks on the upswing as things grow. Most of the time, when people are hired with a need to train, they are paid just salary – probably lower – and they take it, take the training and find that now they are grown up and can probably do something better.

        The right stratey for these guys – maybe after the first year of getting them onboard and bringing them upto speed – should be how to get them “involved”

      2. To follow up with Jeffrey McManus’ suggestion of “hiring smart people and investing in them through training,” I would add, “hiring smart YOUNG people.”

        I’ve had surprising luck with college juniors & seniors in internship roles. They’ve come from both marquee names like Stanford University to smaller universities like College of San Mateo. The common denominator has been a kernel of passion in programming and a zest for playing with new technologies.

        It’s tough to identify the right spark, but once you do, make sure you use a different management style. As Om Malik wrote, many of these people leave after they’re well-trained. That is inevitable and should be expected. But it can be minimized with the right kind of management.

        Some successful tips I’ve used:

        + Train someone on your team to be a good mentor. Being a good mentor is tough, but if someone is good at it, they’ll be able to inspire a whole new generation of talent.

        + Give them ownership & responsibility for a feature, minor or major.

        + Have a grand vision and hire those who share a natural passion for it. Make demonstrable advances toward that vision and keep your entire team (not just these young hires) inspired.

        + Maintain a fun & engaging culture. Sure, company culture itself isn’t enough to keep someone around. But without an engaging culture, you’ll lose top talent that much faster.

        + Tell them you’ll teach them how to be an entrepreneur themselves one day. Since some will inevitably leave, embrace this fact and be a mentor to them. Sometimes they’ll stick around just to learn some more.

        + Give them a chance to interact with, and perhaps work in all aspects of your startup. Chances are, this is happening already. I’m a big fan of the cross-pollination of ideas.

        Obviously, some of these tips work for experienced hires as well as inexperienced hires (though the first point is especially pertinent for them). I don’t mean to imply that you should give inexperienced hires special treatment too. But not everyone can be managed the same way.

        In short, expect some churn, but don’t let that stop you from the daily activity of recruiting. Give everyone, experienced & inexperienced, a chance to learn & grow – and focus on giving inexperienced hires a strong mentor.

        My $0.02.

      1. And after you hire those young people, explain to them how you’ll work together to help them out of the technical track, where ageism and cost-cutting are going to declare them obsolete by the age of 30. After that many years in software, I still learn a language or two every year, but I’ve also been around enough not to make the wet-behind-the-ears mistakes that too-commonly doom career-starting startups. Out of maybe sixty good developers I’ve known who started roughly when I did, all but about five are in completely non-technical jobs now, or are long-term unemployed. Three are senior development managers in large firms, and there’s one other guy who learned Chinese and moved to Shanghai ten years ago. Very, very few other crafts, trades or professions kill off their talent as systematically and arbitrarily as development does.

        Sorry for restating the obvious, but every time I see someone emphasize how “YOUNG people” are somehow intrinsically superior to others, I get angry. It’s the most reliable, inexcusable troll bait I’ve seen in thirty years online.

    1. I think the problems are more acute for companies that are not big enough to have offices in different locations. But as the companies become more mature, it becomes easier for companies to be using non-SV locations.

      1. Om,

        I’m in agreement with Don about Atlanta but I don’t think at this point there’s as much of a cost in employing remotely as there used to be.

        You don’t need an office outside of SV. Coworking facilities combined with any number of tools (low cost/free/opensource) can provide for all situations except when you really need to be locked in a room with someone to work out issues.

        There are plenty of successful groups like Sonian and Opscode that have a distributed workforce with only the occasional travel back to the mothership. I think that’s a true test of how well your team gels if they don’t even have to be in the same room to get work done.

      2. I was just about to suggest the same thing as Don 🙂

        Om – To me, the problem you’re describing is yet another reason why SV investors should consider putting at least some of their money to work outside the valley. Then companies wouldn’t have to wait until they’re large enough to have remote offices in order to take advantage of the great development talent we have in places like Atlanta…they could just headquarter here. It’s not that crazy a thought!

      1. Actually the tech crunch in Asia is just as tough if not moreso as far as talent scarcity and compensation. My own company is having a hard time pulling experienced talent in the region without breaking our base comp curve. We build financial softare and offer enterprise services.

  2. The lack of engineering talent has been an issue for the past 12 to 18 months and I agree it is going to get a worse. As an anecdote, I met with a NYC seed funded company last week and they are moving to the Bay Area, since there is more engineers available here. The CEO mentioned that he hired three Facebook engineers and they didn’t have an interest to move to NYC, so the founders are moving to the Bay Area.

    Although the big companies you mentioned are hiring, there is always a certain percentage of engineers who don’t want to work for big companies, so Zynga and Facebook are not that compelling for some.

    Lastly, Palantir is another growing startup that I hear is actively hiring engineers.

    1. Shai,

      THanks for bringing up Palantir. I have heard similar stories about them on a hiring binge too. As for the engineering talent crisis — it is actually more acute for folks in the mobile space.

  3. I sincerely hope this means some startups will start considering remote leadership. I ran Digg during it’s explosive years 3 out of 4 weeks a month from Ohio. And with a terrible housing market here in the Midwest, I’m not likely to moving out to SF anytime soon (now that I can!).

    If you know of anyone willing to entertain top notch talent working remotely, please let me know!

  4. Om,I think you are more right than you realize. There is a macro-economic impact on success rate, future valuations, etc.. And then you get kids like the creator of youtube instant search, itunes instant search that release some beta code hacked together in a weekend and everyone is offering them jobs like they are the next shawn fanning or Zuck. The talent grab is crazy right now. As for better hiring strategies, Joel Spolsky uses high paid interns from college and gives them great perks in hopes that he gets a few back. Pretty smart to get’em young and identify high talent early on…. 🙂

      1. Brilliant Strategy here. Get em young, make an impression and hope they come back – requires a bit of mature folks around to guide these folks, but well worth the investment – in a way people who can manage interns, do become well versed in communication skills 🙂

      2. Lots of companies are actually doing this. Universities across the globe are starting co-op programs, such as the University of Waterloo (where it’s required for all engineering programs) and startups are keen to hire.

  5. While there is a huge talent glut for engineers, with so many open positions in engineering, its interesting that the same isn’t true for related job roles.

    Product development in particular doesn’t seem to be experiencing any where near the kind of demand engineering is, yet it’s an equally vital role.

    I wonder if the new trend in smaller funding rounds is simply pushing product development onto founders or engineers themselves or whether we’re building products in a different way now.

    1. Ben

      I think at Mobilize, Christian Lindholm while talking about devices etc., pointed out that unless you are a product guy, you cannot be a successful founder/CEO

      In my experience I have seen that too. I would point to Zuck as an example. Amongst my close friends, @hnshah of @kissmetrics is a good example. I think @jack at Square is also a product guy. I mean there are ample examples that show that “we are thinking about companies differently.”

      Love to hear your/other folks thoughts.

    2. Unfair though it may be to people with real experience in product development, “product guy” is sort of a catch-all where silicon valley wannabes with no real skills end up positioning themselves. In a boom cycle the ones who can talk fast get funded, and the first thing they need are actual designers and engineers to build their vision.

      Meanwhile, the move from consumer goods to software, and SaaS in particular has lowered the cost of product development, increased the urgency of time-to-market, and allowed rapid iteration and mechanization of things that previously were under the purview of production development (think A/B testing vs market research).

      This dynamic places a high premium on the doers rather than the thinkers. It’s to the point now where even designers are getting frozen out if they don’t have enough low-level skills. Given the increasing technological immersion of youth, and the lowering barriers to entry of development thanks to open source and improved mobile platforms, I only see the demand for overlapping skills increasing.

      1. I think doing over thinking *is* the new paradigm. Approaches like Lean Startup are making iteration and testing very effective and low-cost, and to some degree that replaces the need for deep thinking.

        At the same time, you can’t make huge leaps forward without deep thinking. But without the budgets for deep thinkers, startups have to be smart and hire thinkers that are also doers.

  6. Since I left the Bay Area for Atlanta, I have found startups here also have a people problem, but the problem is different in nature. Namely, there are plenty of people available to hire, but most don’t get startup culture. In many ways, I’d rather fight for people who “get it” in the Bay Area than this any day.

    1. Interesting – I moved from Atlanta to the Bay Area earlier this year. I agree with your sentiment about the Atlanta startup landscape. Atlanta has a lot of great things for startups — cheap local talent (and great developers coming out of Georgia Tech.. but subsequently leaving the area), affordable housing, a vibrant city for single founders, a ton of Fortune 500 companies that could be great customers for B2C companies and a great hub for media and music startups — but there is a lack of early-stage seed funding and of investors that intimately grok the emerging and fast-paced industries of mobile, real-time and their ilk.

      1. I wholeheartedly agree with your comment on the fact there is a lack of early-stage seed funding. I have devised a totally disruptive e-commerce solution, yet nobody here gets it.

        They all want to invest in growth stage companies only. So yeah, it sucks to be a seed stage startup trying to grow a business in Atlanta.

    2. Matt,

      I can’t disagree more that people don’t “get” startup culture. It’s more of an issue of different priorities. I fully understand the startup culture but startup cultures don’t understand someone who can’t put in 20 hour days or who, from past experience, consider 20 hour days a sign of poor management.

      There is some amazing talent available in the metro Atlanta area but they simply value things like family higher than career and have reached the point where they can make that call.

      I haven’t met anyone who isn’t willing to put in extra hours needed to really get things going as long as that’s the exception rather than the rule.

      1. Amen, to both John’s post and Matt’s reply. Those of us who have been through a startup or six generally have a short but significant list of “Nuclear Red Flags” that make us keep at least one eye firmly planted on the exit, no matter how excited we are about the potential of the startup. I’ve seen that in everything from expecting life- and marriage-threatening hours (clichéd but still too real) to my personal “fave,” the guy who’s latched onto Kawasaki’s “don’t worry, be crappy” idea without understanding, let alone meeting, the conditions that Kawasaki sets for that. If you’re a revolutionary, creating an entirely new product space that didn’t exist before you, fine. If you’re creating the 21,472nd clone of Digg and thinking it “revolutionary” because nobody that you know of has ever tailored it to your local market before, not so much.

        I understand effort; I understand ‘short-term sacrifice for the long-term success of the team.’ I also understand that running people into the ground, refusing to buy or borrow (e.g., open source) to solve pieces of your puzzle and reinventing all your wheels instead is a guaranteed, time-honored recipe for failure. You may get your 1.0 out the door, but you’ve got nothing and nobody in any shape to build 1.1, let alone 2.0 or 6.0.

        Having said that, if anybody has any interesting opportunities that they’re interested in working with a highly experienced American dev in Beijing or Singapore, drop me a line. 🙂

    3. I can testify from personal experience (former Bay Area resident currently running a consumer tech start-up in Atlanta) that the primary problem in Atlanta is lack of investors that “get” consumer tech. Investors here are, unfortunately, scared to death of consumer tech. I think they WANT to invest, but they’re just too afraid, and no one is willing to step up and lead with confidence. The main questions I get from investors here are always about finding some B2B angle to my start-up, even though we’re cash-flow positive in B2C!!!

      What I don’t understand, though, is why investors need to be local? If we’ve got a promising start-up community sans investors, shouldn’t investors be willing to hop on a plane?

  7. How does this reconcile with an overall unemployment rate of 11.5% in Silicon Valley? And a 20+% unemployment rate of engineers age 40+ in the Valley?

    Assuming the skills are there and people get the culture, there seems to still be plenty of talent around…

    1. Veit,

      Can you kindly share the source of that data. I would very much like to provide a link to it in my post and add more context.

      As to your point about there being a lot of other engineers who are unemployed, I wonder if that has to do with skills and focus of their talents. I am not sure. Perhaps you or one of other readers will enlighten me.

      1. The 11% (if accurate) rate mentioned above is understated as people who are not actively looking for are not included in the unemployment rate. Some people have taken a break for looking for work as they get frustrated in not landing a job, so therefore they are not actively looking.

        So say that there is some age discrimination that is taking place in hiring engineers, but I don’t have evidence to prove it. Vivek wrote an interesting blog post on this topic, which ties in to this conversation.

        http://techcrunch.com/2010/08/28/silicon-valley%E2%80%99s-dark-secret-it%E2%80%99s-all-about-age/

      2. There is something called Google…

        Silicon Valley’s Dark Secret: It’s All About Age
        TechCrunch. Vivek Wadhwa. Aug 28, 2010
        http://techcrunch.com/2010/08/28/silicon-valley%E2%80%99s-dark-secret-it%E2%80%99s-all-about-age/

        Silicon Valley’s uneven employment rates
        http://marketplace.publicradio.org/display/web/2010/09/17/mm-silicon-valleys-uneven-employment-rates/

        and I am sure a bunch of other links and stories if I had the time to find and post.

        As for “doers” vs. “thinkers” – I agree that technical founders and engineers for a start-up in the early stages of a company should absolutely be the focus. And getting “product” oriented folks rather than just pure programming genius is kind of obvious. But at some point, as you scale, you’re going to want to have a better division of labor between engineering and “product” – product management, marketing, PR, etc. On another related note, experience and judgment still matter. My friend reminds me of some of the mistakes his junior members of his team at Facebook make.

  8. It’s more a Hiring Insanity than a Talent Crunch. The hiring process has become so divorced from software development it’s easy to understand why companies can’t find people. Only a very few people can pass all the silly tests that are part of every interview. If you can’t hire people maybe it’s you and not the people?

      1. Om, a big part of the problem is HR automated (or manual) nitpicking; if a résumé doesn’t have exactly the right words in exactly the right order, HR (or its software) deletes it and the hiring manager never even knows that anybody applied. I’ve found this dozens of times when I’ve sent résumés off and then made networking connections online with the manager or his reports/colleagues who’ve never heard of me and ask me to send directly to them. More often than not, when I do, HR gets their knickers in a knot because I’m “double-submitted” and somebody else (who happens to be younger/cheaper/higher-scoring on the automated-keyword scanner) gets called in instead. It’s been to the point for a couple of years that I don’t even hold out much hope for American companies hiring this forty-something-year-old American. Scraping by on next to nothing is easier in most of SE Asia than it is in Silicon Valley to boot. I think what’s happened is a virtually permanent change; I say “virtually” because we’re really seeing very clearly the defects in and damage done by the current economic order. Once that loses the last pretense of sustainability, lots of things are going to have to change…but there’ll be a world of hurt before and during that whole process. We’ve got to pay the bill sometime

        1. Jeff

          Thanks for sharing your point of view and the problem of HR. You do paint a scary picture and what is worse is tat it is for real. I have no context from a programmer’s perspective, but as a 40+ reporter, I can tell you it is not easy as many of my friends are finding out as well.

  9. Om I think a similar kind of situation is (may be) there in India also in IT these days…especially things like “Some large web company tried to hire my engineer, and I had to boost the salary by 50 percent.”

  10. Om –

    I think smaller companies could benefit if their culture catered to a mix of onsite and remote. IE, not necessarily being in the office every day, but instead three or so days. It would expand the available talent pool available within the Bay Area. People live anywhere from the Almaden Valley up to Santa Rosa, but commuting realities put limits on what is reasonable. For someone in the South Bay, Palo Alto/Menlo Park starts being a reasonable upper limit. I even know people in Los Altos that are interested in companies in San Francisco, but shy away due to the likely commute time needed (45min to an hour each way). I know, personally, I didn’t follow up with an amazing opportunity on the peninsula because it would essentially double the amount of time I would need to be in a car in a given week.

    Modern tools can facilitate remote working in a much better fashion than even just a few years ago. Skype video and the other collaborative tools have come along way. But, I also acknowledge that a lot of the magic / chemistry in a startup is in the ability to just circle up and hash things out.

    Zynga is understanding the geographic talent distribution and has opened a Los Gatos office. Facebook being in PA is pretty central, but with Twitter being only in the city have their limitations on the talent pool they can draw on.

    Personally, I am keeping an eye out for interesting startups while continuing to noodle around with some of my own projects.

  11. My company Web CEO, the leader in SEO desktop software, is located in the Ukraine where there is plenty of top talent. Until one of the giants locates themselves in Odessa or Kiev, I see no problem into the future.

    So far, its the same for Russia. Cisco wants to invest $1 Billion in Russian IT firms.

  12. Interesting that similar problems exist in other parts of the world as well. As the economy is on an upswing (again) in India, smaller companies are having trouble retaining people within a certain age group – those between 26~35. These people leave for larger companies even though the work may be far less challenging, because the pay is usually better, because it is much easier to find a partner in the spouse-market if you are attached to a higher net-worth company, and because you needed that home-loan. This may not be true with Freshers who are looking to get large amounts of experience in a short time, and with older people who find it hard to get a job anywhere anyway.

  13. I couldn’t agree more. I’m happily employed, took all my resumes down off the job sites, and still get 5-10 recruiters bugging me per _day_.

    If you want to hire top engineering talent that will stick around, have a great culture. Yes, having a great culture takes incredibly good management. So I guess if companies are having trouble hiring the best people because of crap culture, they need to look really hard at why they are paying their top management so much money.

    Perhaps harsh, but I’ve seen it to be true from personal experience.

  14. Om, I love your writing by the way. I’ve been reading you for about a year now, ever since I started playing with twitter. Thanks for all the great writing 🙂

  15. I guess there is a trade off of wanting to work in the Start Up environment or move into the relative safety of a Zynga , Twitter , Facebook environment.

    The UK has similar problems and in technology it is either London or Cambridge as the main choices. TechHub in London is a new work environment where start ups can get a workspace and have access to super fast internet. Mike Butcher the European editor of TechCrunch has an interest in the TechHub venture and he uses it to organise Angels and VCs to come in for various events like the recently held SeedCamp.

    Hope the San Andreas fault keeps its ‘big event’ off for another millenium – seems crazy that Silicon Valley is built in and around the most potentially unstable geographical part of the United States.

  16. Infact it is good for early stage startup. If you want to hire someone in early stage small startup (of 4-5 people) and if even he kick out Google,Twitter,Facebook and joining your startup, definitively he is not betting on instant cash/Perk !

    This will definitely create problem for startup in hiring normal talent, but the talent startup will get in this situation will be very helpful in foundation of startup (in long run !)

  17. It was a hard decision to move back to India after few years of stint at US but this makes me feel good about setting up Hireplug offices at Pune, India.

    We have nothing closer to valley in this respect and yet a least expensive high quality talent pool available.

    Now, getting into Alexa 100k while building from here…makes me feel even better 🙂

  18. I love the start-up business. The blood coursing through the people and the buildings is more than palpable, it’s explosive. But all this talk of engineering talent makes me crazy. Start-ups lack marketing chops. The ability to create a product “Is-Does”, for example, (what a product is and what it does) is often a major stumbling block post business plan. Shoe-string budgets are no excuse. 1s and 0s don’t always equal dollars and cents.

    1. As a product person, I feel people on the product side are often over looked. Product people are key contributors in taking the company vision and making it a reality.

  19. Another thing I can think of is difficulty in hiring people from overseas. The boom was fueled by h1b visas. But that is no longer the case. It has been incredibly hard and expensive to hire overseas worker now.

  20. How much of this is exacerbated by the housing crisis?

    The obvious answer to this would be for talent to be pulled from other cities, is that being held back because:

    a. the talent can’t move due to houses they can’t sell

    b. valley companies aren’t interested in pulling in non-local talent

    c. no one with talent lives in other cities

    I don’t believe C is true, so which is it, a or b?

  21. I’m on my fourth startup. Having embraced remote workers from the beginning, I’ve never had a problem finding qualified people, even developers with fairly obscure and hard to find skills.

    Finding them in the Bay Area can be tough, and because it is an expensive place to live, they are highly motivated to take the best offers they can get.

    The thing is there are a lot of very talented engineers, artists, whatever you need, who don’t want to live in a city, don’t care about going out at midnight, and want to live a sane and quiet life, raise their kids, etc. Hint: they don’t live anywhere near San Francisco or New York.

    Give this sort of person a job that is structured around their life, plus the freedom to work remotely, and you get a happy camper, often one who costs significantly less than a “rock star” hire in SF (when I see that label, I think “prima donna” who is going to be a pain in the ass for everyone). If you trust people, and give them good direction, it works out well in most cases, and with Skype and other tools, there are few communication barriers to get in the way.

    On the other hand, I tend to work in industries that get less buzz (telecom, and more recently professional translation), so I’m not looking for people who build Flash games, etc.

    My $0.02

  22. In Silicon Valley there is plenty of talent available (just look at the unemployment rate), but I think there is a shortage of startup talent that is becoming very acute. The number of engineers that have the overall skills to be a single or two digit employee is quite low. Those software, build, and QA engineers need to be able develop and prosper in an environment where there is very little to zero infrastructure in place to make them successful. They need to make their own success, whether it means defining the product itself, or building out the colo, labs, and network, or ordering their own machines. They need to be incredible well rounded and willing take over various parts of building a company.

    My current company has 4000 software engineers, I bet only a couple of hundred of them could really contribute in a startup environment.

  23. It’s great to hear that so many companies are hiring and that unemployed engineers will have some more options available to them.

    However, I disagree that all the good talent is headed to these big companies. I myself, am a “startup-junkie” and thrive best in the entrepreneurial spirit of a startup environment. Job security is not my priority. I really enjoy working to help other people’s great ideas become a reality.

    I work with other very talented and enthusiastic engineers who have little desire to head to big companies no matter what the incentives may be. Don’t be discouraged, entrepreneurs! There is still plenty of great talent that would be excited to participate in your success!

  24. There’s another option. Startups can build a dispersed team and draw from a far larger pool of talent. At Wide Teams I podcast interviews every week with developers and executives from companies – many of them startups – that are finding success with geographically distributed teams. By broadening their horizons they are able to get the right person for the job, wherever she might be.

  25. Fascinating post (as always, sir).

    A couple of years ago, a friend of mine had four job offers (Google, Salesforce, Amazon and Microsoft) before he even finished his studies (Masters, I think). He took one*, of course, and remains happily employed there to this day.

    Moral: if you want the clever ones, find them early.

    (*wouldn’t want to upset the other three…)

  26. I am going to echo Jeffrey’s comment as it was the exact same thing I first thought of. Requiring candidates to have specific expertise in a specific technology is contrary to the whole idea of a stat-up. A start-up by design is doing something that has not been done before. It is hypocritical to turn around and say that everyone has to have experience in certain technologies when in fact, the start-up itself is developing a new idea/technology.

    Just my $0.02 worth.

  27. Silicon Valley gains jobs year over year for the first time in 24 months

    The San Jose Mercury News. 10/22/2010

    Silicon Valley employers added jobs in September, but the unemployment rate in the San Jose metro area was unchanged from the month before, the California Employment Development Department reported today.

    Total payroll employment in the region — which includes Santa Clara County, home to most of Silicon Valley’s big tech employers, and mostly agricultural San Benito County — was up year over year for the first time after 23 consecutive months of declines.

    The combined jobless rate for Santa Clara and San Benito counties was 11.2 percent in both August and September and down from 11.7 percent in September 2009. In Santa Clara County, the jobless rate was 11.1 percent.

    Employers added 4,500 payroll jobs

    http://www.siliconvalley.com/ci_16407201

  28. 99 Weeks: When Unemployment Benefits Run Out

    Scott Pelley Reports On The Growing Number of Americans Who Are Exhausting Their Benefits

    “Even after an extension of unemployment benefits to 99 weeks, many of those about to go off the program are in a quandary. Scott Pelley talks to some of them in Silicon Valley.”

    60 Minutes. Monday, October 24, 2010
    Video (13:14 minutes)
    http://www.cbsnews.com/video/watch/?id=6987699n

  29. In the years that I spent being part of the original team that built vBulletin, we were all distributed around the western world. I’ve seen lots of other very successful groups that are as well, more companies just need to engage with virtual teams. Their cost will fall dramatically as well.

    Finding people for the job isn’t really a challenge, it’s just finding the *right* people.

    And there are many of us who are very experienced who aren’t attracted by the bright lights and understand there is a lot more on offer with a decent company that $ or insane hours playing clean up.

This site uses Akismet to reduce spam. Learn how your comment data is processed.