Google Made Its Secret Blueprint Public to Boost Its Cloud

Craig McLuckie took the idea to Urs Hölzle, the man who oversees Google’s global network of computer data centers, and Hölzle didn’t like it.

Together with two other engineers in Google’s Seattle office, McLuckie wanted to recreate Borg as an open source project. Borg is the sweeping software tool that drives everything from Google Search to Gmail to Google Maps, letting the company carefully parcel computing tasks across that global network. For years, it was one of the company’s best kept secrets. And McLuckie wanted to share its blueprint—or at least some of it—with the rest of the internet.

“When we went to Urs the first time, he was not really happy about the idea,” McLuckie says. “What we were basically doing is offering up the secret sauce that runs Google as an open source technology.”

But Hölzle and the rest of the Google brass eventually approved the project. Known as Kubernetes—an ancient Greek word for shipmaster or pilot—it launched a year ago today. Over the last 12 months, more than 370 coders have made more than 12,500 “commits” to the project, and a majority of these coders don’t work for Google, including three of the top six contributors.

The tool represents a notable shift inside Google as it strives to compete with the likes of Amazon and Microsoft in the world of cloud computing. Traditionally, Google closely guarded the fundamental technologies that drove its online empire—Borg being a prime example—and to a certain extent, it still does. But now that it’s a serious cloud computing company—inviting companies and developers to build and run their websites and other online applications atop its infrastructure—there’s a little more give and takewith the software community as a whole.

McLuckie and the two other engineers who originally conceived Kubernetes—Joe Beda andBrendan Burns—were part of the team that built Google’s cloud computing services. They saw their open source project as a way of pushing developers onto Google Compute Engine, a cloud service that offers up “virtual machines” where outside companies and coders can run almost any software code—Kubernetes provides a way of more efficiently running code across such cloud services as well as the physical machines companies operate in their own data centers—and eventually, Hölzle came to agree this worth any advantage they might give up in sharing the code. “Cloud is an absolute imperative for Google,” McLuckie says. “We have to operate in a new way. We have to bring our expertise to the community.”

Google Evolves

The move comes in response to a larger change driven by the rise of cloud computing. Now that developers can readily build software atop a large array of machines, they’re using open source to do so. Running proprietary software across so many machines is far more expensive, and molding it to fit particular needs is far more difficult. Whether they’re offering services like Google Compute Engine or building software for such services, today’s cloud companies must embrace open source. Microsoft has also recognized this undeniable trend, embracing open source in an effort to boost the prospects of its Azure cloud computing service. So haveVMware and EMC.

With Google, there’s an added twist. Google-built technologies were one of the primary forces behind the rise of open source software across what we call the cloud. It’s just that the company kept a certain distance from this movement. Now, it’s moving closer.

Over the past decade, the company would build software for running code or juggling data across hundreds of its own machines. It would jealously hide this software, seeing it as a competitive advantage. Then, a few years down the road, it would release a research paper describing the tech, and the open source world would clone it.This happened with so many tools, including MapReduce (which spawned the open source Hadoop) and BigTable (which gave us a world of “NoSQL” databases).

Much the same thing happened with Borg. A few years ago, engineers at Twitter and the University of California, Berkeley built a tool called Mesos, and today, it underpins some big-name internet services, including Twitter and Airbnb. And several other projects are building similar tools around a technology called Docker, which helps developers neatly package their applications into the sort of software “containers” that Borg juggles across the Google network.

But now, driven by the needs of its cloud business and other forces, Google is changing its role. It’s not just sharing a research paper. It’s not just open sourcing small pieces of its online infrastructure. It’s transforming a major system into an open source project—at least up to a point.

As Burns puts it, he and Beda and McLuckie saw so many other running projects that explored the basic ideas behind Borg and containers, and they decided Google could help push things forward. “We really felt like all these puzzle pieces were out there and people were stumbling around trying to put them together,” he says. “We really had the puzzle box. We knew, from ten years of experience, how the pieces all fit together.”

To be sure, Kubernetes is not an open source Borg. It’s not as complex as Borg or Borg’s successor, known as Omega. And as it stands, it won’t juggle tasks across quite as many machines. “We’re still reaching the same scale as Borg,” Beda says. “The focus for Kubernetes out of the gate was getting the core concepts right.”

But some of those who built Borg and Omega also work on Kubernetes, including uber-engineers Eric Brewer, John Wilkes, and Brian Grant. “Having engineers with this heritage working on the project—folks who had worked on Borg or Omega—means a lot,” says Joe Fernandes, who oversees Red Hat’s work on the project. And according to McLuckie, Kubernetes aims to expand beyond Borg and Omega—-to fix their mistakes.

In a world that so values open source software, this is the best way for Google to compete. For many, containers are the future of software development. And Google is now playing a direct role in that future.

Run What You Want

Mark Kropf—who helps build similar systems at a company called Pivotal—questions how serious Google is about the project, pointing out that Beda has left the company. And according to Ben Uretsky, the CEO and founder of cloud service provider Digital Ocean, Kubernetes—and other systems like it—aren’t that widely used. “Containers are not there just yet,” he says. “We’re just at the peak of the hype cycle.”

But whatever the fate of Kubernetes, it still points to the future of software development—in more ways than one.

Inside its data centers, Google uses its own breed of software containers. But when it launched Kubernetes a year ago, it paired with Docker, an open source technology. And when the company behind Docker sought to marry its container technology to some of its own software tools, Google and the Kubernetes project embraced a Docker alternative, known as Rocket, that aims to promote a wholly independent container format.

McLuckie makes a point of saying that Kubernetes now handles both Docker and Rocket. He hopes that the larger community will eventually back a single independent container format, he explains, but it’s not picking sides. Kubernetes, he says, “should be Switzerland.” That may sound idealistic. But this is what a cloud company must do: Give the world open source software, and let them use it however they want. “The disruption that’s happening,” McLuckie says, “is bigger than any one company.”