Critical.
Authoritative.
Strategic.
Subscribe to CIO Magazine »

Facebook Tech Infrastructure Needs Constant Care

Jonathan Heiliger, the top technology exec at the huge social networking site, talks about his efforts to build a technology operations team at Facebook that can both handle millions of users worldwide and a restless, creative culture inside the company.
Page:

What's an example?

Over the last two years, we have had a concerted effort to improve the push tool so site updates are seamless to users. Every couple of weeks, someone checks in some bad code or there's a bad database call or we fail to do full design review and push it into production and see user impact. The site might start running slower or a geography of users will have issues. Cyber liability isolates the problem, reverts the component or reverts the whole thing back to the previous known good state.

At Wal-Mart, we had the belief that we only roll forward, never back. Once you make schema changes in the database, it's difficult to pull back. If you pushed buggy code into production, you had to fix it in production. With user impact covered in the press.

Here, it's the opposite approach. We know there's going to be broken things that happen fairly regularly. We are ready. We have emotional shields for them.

You changed the basic Facebook interface a few weeks ago. What sort of things happened with that rollout?

That's a massive change. Similar to how we rolled out Chat, we turned the new interface on gradually, some percentage of users at a time.

How did you roll out the chat feature?

We had the technology running for about a month [detecting who was online] before we had the user interface visible. We turned it off several times, found a bunch of bugs that way. You can't discover that in a QA environment. You need millions of people pounding on it every day. But the actual rollout is gradual.

Is gradual rollouts an approach enterprises should take with big software rollouts?

That control, that knob, gives operations and development organizations a lot of confidence. You can turn up the heat and if there are issues, only a certain percentage of employees have been affected. It's a mentality shift. In some large enterprise apps, you can't necessarily control technology changes to a subset of users. They all have to be using the same iteration at the same time. But in other instances, you can.

Page:

Recommended

  1. Saving Google Maps destinations for easy navigation

  2. Mixing with marketing: The CIO-CMO partnership

  3. Tapping into social experience: Tourism Australia

    CMO

Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.

References show all
Comments are now closed.
Related Whitepapers
Latest Stories
Community Comments
Latest Blog Posts
Whitepapers
  • Clearing the Clouds for Midmarket Businesses
    Cloud computing promises to help midmarket companies reduce cost and complexity in the IT equation – and gain the flexibility and agility they need to thrive. Yet charting a clear course to the cloud isn’t always easy. In this paper, we aim to clear the clouds. We examine different cloud computing models, discuss the types of requirements that each can best address, and consider what midmarket businesses should look for in a cloud solutions provider.
    Learn more »
  • Tips Choosing a Cloud Service Provider
    Because cloud is still a new and evolving business model, it can be argued that the decision to select a cloud service provider should be approached with even greater diligence than other IT decisions. Many providers use the same term to define very different services, “hybrid cloud” is one example, making it difficult to compare offers. This whitepaper will help enterprises evaluate their options in two critical areas: the cloud service portfolio and the service provider itself. Read now.
    Learn more »
  • Android Malware Exposed
    Take an in-depth look at the evolution of android malware. The world of malware targeting the Android OS is similar yet very different from malware affecting Windows. Explore the rapidly evolving world of android malware and shed light on the various techniques used to exploit devices using this OS.
    Learn more »
All whitepapers
rhs_login_lockGet exclusive access to Invitation only events CIO, reports & analysis.
Recent comments

Computerworld
ARN
CFO World
CMO