OpenStack – an open source cloud platform

Rackspace announced the OpenStack project today, open sourcing much of the software it uses to run its own cloud. I spoke with Rackspace’s Jonathan Bryce on the topic to get an in-depth overview, discuss Rackspace’s intentions, and explore the operational future of OpenStack.

This is a big announcement in the cloud world, further widening the technologies that are available to start crafting public and private clouds. The nature of Rackspace as not a software company is also interesting to watch here, as well as what partners do with the project.

Transcript

Michael Coté: Well, hello everybody, I’m here in the Austin Rackspace offices in the Austin City Limits conference room as you can see. It’s, perhaps, one of the more three dimensional conference rooms I’ve ever been in. It’s very exciting and this is always Michael Coté from RedMonk and I’m joined by a guest. Would you like to introduce yourself?

Jonathan Bryce: Sure. I am Jonathan Bryce. I am the Co-Founder of the Rackspace Cloud and I’m currently leading up the technology-end of a new venture that we are just getting started.

Rackspace has been doing hosting, managed hosting, dedicated hosting for about a decade and a few years ago we started a cloud initiative to do virtual servers, cloud storage, Platform as a Service and the big news that we’ve just released is that we’re actually going to be open sourcing most of the software that runs our cloud system. Specifically, our cloud servers’ product line, our cloud files product line.

We’re going to be giving all of the source code away and opening it up under a new organization that’s called OpenStack.

Michael Coté: What does that mean, for “cloud software” to go Open Source? Do you think — I mean to the — you were, kind of, alluding to this a little bit, but a lot of what a cloud is, is the hardware and everything that is running. So, what’s the part that’s the software you can Open Source?

Jonathan Bryce: Well, most of the clouds’ out there – public, private, enterprise, you know, whatever – under the hood it’s built with a lot of Open Source components like Linux, KVM, or Xen or different Open Source hypervisors. But what the missing piece for a lot of that is an orchestrator, a controller that can make tens of thousands of physical devices, the network, all of the different components work together so that you can provision and manage virtual servers in multiple locations across all of this hardware.

So we’ve built a system that does that. Our cloud is very large. It has hundreds of thousands of cores. Our storage system has billions of files, petabytes of data, and there’s a lot of software that glues it altogether and makes all of those other technologies work.

Up to this point, there have been a few Open Source projects that do similar things, but all of the ones that have really been at scale have been proprietary systems that are running in other people’s datacenters like ours and Amazon’s and Google’s and so this, I think, the big announcement and the big change for the industry is that we’re going to have what’s kind of a carrier grade controller for clouds that is, now, going to be available and it’s going to be open and it’s going to be contributed to from a lot of different players, not just us.

Michael Coté: To that point of it being carrier grade, which is, it’s kind of fun, it’s sort of like in the — outside of the enterprise datacenter, instead of using the world “enterprise” you always use “carrier grade” to mean, like, hardcore real stuff, right? But to that point, how long have you guys been using this software internally? What’s the maturity of the Open Source cloud stack?

Jonathan Bryce: The software that we are running has been in development internally for about four years. It’s been running in production for almost that whole time. We have tens of thousands of companies from all over the world that are using it in our environment. So it is something that — it’s been battle tested and it has scaled and I’ve seen a lot of demand and a lot of success in the deployments that we’ve already done with it.

So this is not, kind of, just a think layer of virtualization control that’s meant for a test lab of ten servers or 20 servers. It really is meant for that, kind of, large provider type scale.

Michael Coté: It’s not for like your Beowulf cluster in the closet?

Jonathan Bryce: Right.

Michael Coté: I mean you guys are not really a software company necessarily. So what’s like the motivation for — I mean this is a very software company, sort of, thing to do, to Open Source something. So, why are you guys doing this?

Jonathan Bryce: Well, a big a reason of why we are doing this is because we’re not a software company. We believe that the best technologies out there in the last decade have been driven forward by Open Source, whether they were Open Source systems themselves or whether Open Source provided a real competitor to an existing entrenched closed source player. Open Source has really propelled innovation and what we see, right now, in the cloud is that it’s a huge opportunity, it’s a huge market shift, but it’s being held back a little bit by the fact that a lot of the cloud technology out there is proprietary.

It’s either closed source, commercial, very expensive, or it is only run in a provider’s datacenter. We looked around and we didn’t see anything that met our needs and thought here’s an opportunity for us to really help push this forward, to get a lot of people involved, to use our scale and success to accelerate the adoption of the cloud technology.

Some companies who — you know, you mentioned software companies that Open Source some of their products, sometimes that happens when those products are maybe on the decline or if they need a marketing boost to generate interest and there’s always a little bit of a conflict of interest there and as a software company how do I give away my software and then I also make money off of it? Lots of people have done it and made a lot of money.

For us, though, it’s actually a much simpler decision, because, for us, the software is a piece of the overall service that we deliver. But really what Rackspace is about is operating software at scale, doing it really well, really efficiently, really reliably and then offering great support and great — just an overall awesome experience on top of it.

So, the software, to us, is not a real advantage competitively. We built our company using Linux, using MySQL, and Postgres, and JBoss, and Apache and all of these freely available systems. What we did that set us apart from the competition is we delivered it in a way that it was just a superior experience. When we look at the cloud, what we see is a lot of competing proprietary systems and there isn’t really a true competitive market.

We want to help move the cloud space to that. Where you can compete on support and a premium experience or you can compete on cost or you can compete on operational efficiency or you can integrate it in to your enterprise application, your ERP/CRM system, or whatever needs to be, but really move it to a true competitive market.

Michael Coté: When you open source something there’s lots — hopefully there’s other people who are working with you and collaborating. Like who are some of these other companies that are interested in working with you or partnering?

Jonathan Bryce: In our announcements, we’ve talked about a number of them. Companies like Dell and Citrix. A bigger partner who has really kind of been a surprise to us, but has been an amazing piece of this is actually NASA.

When we started to think about this, we went and we looked at some of the Open Source — actually all of the Open Source options out there to see if any of them would meet our needs and none of them would meet our specific needs. We have a specific set of needs, because of the scale and the type of offering that we run on top of it and so we thought okay, well this is it, we’re going to Open Source our code and really take this on ourselves.

A few weeks before we were getting ready to do all of this, the software that powers NASA’s Nebula Cloud was actually Open Sourced and we saw it, and we looked at it, and we said, “Wow, this is really awesome.” We always said that if you could find something better then we’d love to partner up and work together with that and so we’ve been able to do that and it puts us — actually it gives us a head start, it pushes us forward and helps advance the whole thing and I think it really shows the power of Open Source.

Because immediately NASA has access to more technology that we have, our storage system for instance that they were missing before. We have access to technology that they’ve been running inside of the federal government and so this is really the value and the promise of Open Source.

There are a lot of other players who have been our partners for a long time and who are well known in the cloud space, companies like RightScale, and Cloudkick, JungleDisk. Other companies who have built on top of clouds that are these proprietary public clouds, this is great for them and they’re all excited about it because now it opens up a whole new market for enterprises who are going to be running this in their own datacenters, for other private cloud installations, and I think it really is just going to be an awesome opportunity for the industry as a whole.

Michael Coté: Do you anticipate that people will try to use the software to compete directly with you guys running public clouds?