Archiv für den Monat: September 2012

In Quest of the „Open Cloud“ (updated)

A remarkable and wonderful feature of Zurich is its vivid computer science and technology community. Hardly any week passes without an interesting event around new innovations and technologies, like the Internet of Things, novel programming languages (Go, etc), security, and of course Cloud Computing technologies (e.g. MongoDB) [footnote1]. Particularly interesting – from our / ICCLab perspective – is the ZhGeeks (@zhgeeks) community, run by one of our fellow technology and cloud evangelists Muharem. Last weeks Zhgeeks meeting was about Open Cloud and no less prominent figure than Samj was about to update us on the Open Cloud Initiative. A truly inspiring talk  (download slides).

Notwithstanding of Sam’s comprehensive and sound introduction into the world of „Cloud Openness“ (from an OCI perspective) I can’t help but have to ask myself – hellya, what is this Open Cloud thing?

So let’s see what the universal oracle has to tell [footnote2]. And here we go. „Open Clouds“ all over; what a surprise. So again, what is it then?

Indeed, this is an interesting question if you bet on CC, either as adopter, contributor, or simply user. As Sam summed it up, „Open Cloud is about technology openness such that the market can drive it’s evolution“, reflecting the OCI definition and principles:

Open Cloud must meet the following requirements:

  • Open Formats: All user data and metadata must be represented in Open Standard formats.
  • Open Interfaces: All functionality must be exposed by way of Open Standard interfaces.“

Having sacrificed several hours of sleep for studying the OpenStack Foundation bylaws in order to understand the implications on OpenStack’s future (you my call it „openness“) – see my earlier blog post Open- or NotSo-OpenStack? – I was wondering whether OCI’s definition also poses open governance requirements onto OSS Cloud Computing projects, like OpenStack, CloudStack, Eucalyptus and the likes, but this is seemingly not the case.

Red Hat’s Open Cloud definition goes further by not only requesting open source, standards, and interfaces, but also incorporating „technical governance“ into „The Open Cloud: Red Hat’s Perspective“. It says „An Open Cloud …

Has a viable, independent community. Open source isn’t just about the code, its license, and how it can be used and extended. At least as important is the community associated with the code and how it’s governed. Realizing the collaborative potential of open source and the innovation it can deliver to everyone means having the structures and organization in place to tap it fully.“

Another popular reference is the Open Cloud Manifesto, which defines a set of open cloud principles, similar to OCI but focuses further on features of an open cloud and the defining community, in particular by embracing the fact that Cloud Computing is a community effort,  that different stakeholder groups exist, and that these ought to collaborate to avoid fragmentation.

„6. Cloud computing standards organizations, advocacy groups, and communities should work together and stay coordinated, making sure that efforts do not conflict or overlap.“
RackSpace, notably one of the founding members of OpenStack, has too an opinion; „Open Cloud Computing : History of Open Source Coding and the Open Cloud“ – in this post exclusively linked to Open Source Software.
Alex Williams from TechCruch „spend some time with the technologists at CloudOpen – the Linux Foundation’s first cloud only event“ and summarizes in his article „To sum it up: if VMworld is about the data center then CloudOpen is about the software“ and continues „Over the past few days, I’ve tried to crystallize the conversation to some extent. Here is my take:
  • An open cloud has open APIs.
  • An open cloud has a developer community that collaborates on developing the cloud infrastructure or platform environment.
  • An open cloud has people who have deep experience in running open source projects.
  • An open cloud gives users the rights to move data as wished.
  • An open cloud is federated — you can run your apps and data across multiple cloud environments.
  • An open cloud does not require an IT administrator to provision and manage.
  • An open cloud does not require new hardware.
  • An open cloud is not a rat’s nest of licenses.
  • An open cloud is not a proprietary, new age mainframe.
  • An open cloud is not washed with marketing.
  • An open cloud can be forked.
  • An open cloud has full view into the infrastructure environment.
  • An open cloud is not hosted, legacy software
David Linthicum seems to share equal motivations which led me to this blog post. He tells in his post „InfoWorld: The ‚open cloud‘ is getting awfully confusing“. The article does not go into detail but nicely summarizes the status quo:
„If you’re looking at adopting an „open cloud“ technology, you have complex work ahead. Assessing their value is complicated by the fact that many of the vendors are less than a two years old and have a minimal install base that can provide insight into fit, issues, and value.

As with any significant IT endeavor, you need to do your homework, understand your requirements, and make sure to test this stuff well before you use it in your enterprise. At some point, the „open cloud“ market will normalize, and when that happens, you hope your seat will still be available in the ensuing game of musical chairs.“

 from Gigao picks up this thread, citing Alex and David, for his „Prediction: More Cloud Confusion Ahead„.

Alex Williams from TechCruch cites the CloudOpen conference as trigge and here goes a summary of the discussions, 10 Insights from Linux Leaders in the Open Cloud.

Richard Kaufmann, chief technologist, HP Cloud Services, July 3, 2012, „HP Public Cloud Aims to Boost OpenStack Customer Base.“There are two important APIs out there, one is Amazon’s and the other is OpenStack. And OpenStack has Amazon compatibility. HP will continue to support those Amazon compatibility layers; We’re not trying to lead on a position about what customers should do with APIs… I (personally) believe there should be a popular cloud API for IaaS and it should not be Amazon. It could be anything else but it can’t float from above. It has to be based on popular usage.

Lew Moorman, Rackspace, July 10, 2012, „Open Cloud Key to Modern Networking.“Some people seem to think that APIs are the cloud and one thing that made the cloud so revolutionary is it’s programmatically accessible by API.  But (Amazon) S3 is a really complex distributed system. The issue with a model that says “clone Amazon” is that, unless you have the core technology underneath it, you can’t have a cloud…

OpenStack is really setting out to build an open alternative from end to end. They say we’re going to do networking, not just set out to copy Amazon. We need to really innovate and build a visionary system that can power the future of computing. Amazon, VMware and Microsoft don’t have all the answers.

Christopher Brown, CTO, Opscode, July 17, 2012, „Chef Offers a Recipe for the Open Source Cloud.“The open cloud lies both below and above the waterline of the API. At the beginning we all wanted to treat the cloud as an easier way to get the compute units that looked like the old thing we used to get buying a physical machine. But that’s not actually true. It’s not the same thing and it requires a different design underneath of a common substrate. If you look above the water line at the consumer, the way you build applications, the way they scale, etc., designing the cloud and for the cloud are different than what is now legacy.

Mark Hinkle, senior director of cloud computing community, Citrix July 24, 2012, „Citrix’s Hinkle Proposes Linux Model for an Open Source Cloud.It’s first and foremost that the orchestration platform is open source. The data you store within the cloud is open to you as the end user in a format you can manipulate easily and it’s easily transferable. The API is also open and clearly documented.

Ross Turk, vice president of community, InkTank, July 31, 2012, „An Open Source Storage Solution for the Enterprise.It can mean a cloud stack that is built on open source like OpenStack or CloudStack and that reflects the economic and community advantages behind building something that’s akin to what Amazon has done, but built on commodity hardware. It’s an open alternative to AWS. Another way to think of the open cloud doesn’t exclude AWS. It’s having cloud services with standardized APIs so applications written for one cloud can work on another cloud.

Imad Sousou, director of Intel’s Open Source Technology Center, Aug. 7, 2012, „Open Cloud Standards will Emerge With More Collaboration.“The open cloud must meet these requirements:  Use Open Formats, where all user data and metadata must be represented in Open Standard formats. Use Open Interfaces, where functionality must be exposed through Open Standard interfaces. In addition, in the open cloud, various open source technologies should be available to build the right solutions efficiently, and to drive innovation. These would include software stack and tools, such as the hypervisors or operating systems, middleware, such as databases and web servers, web content management systems, and development tools and languages. Such open source-based software solutions would reinforce interoperability of the open cloud.

Kyle MacDonald, vice president of cloud, Canonical, Aug. 14, 2012, „Canonical: Making the Open Cloud Seamless for Users.True power comes when the users can move from one cloud service to another. That’s the state of nirvana. The cool word is ‘interoperability.’ …  It will be almost required that if you’re a cloud service you publish an API that’s clear. And eventually there will be a common API or it becomes so simple the minor differences won’t be a big deal to end users. And then partners who define the services can use those same open source technologies and provide a good service.

Alan Clark, director of industry initiatives, emerging standards and open source, SUSE Aug. 21, 2012, „SUSE Aims for One-Click Enterprise Deployment on OpenStack.Enterprise IT must deliver the most efficient, scalable and flexible services possible. The open cloud provides that through the ability to have a flexible infrastructure, quick and easy deployment, service management and complete life cycle management. We’re working with partners — many are part of these open source projects – to build this together and that builds interoperability. It’s a collaboration of ideas as well as code. It accelerates bringing a solution to market that works across all the different partners.

Angel Diaz, vice president of software standards and cloud, IBM Aug. 28, 2012, „3 Projects Creating User-Driven Standards for the Open Cloud.“ Our clients who use technology have a heterogeneous environment. They need to take existing systems, extend them and deal with it and they don’t want to be locked into a singe vendor solution. That is how (IBM) defines an open cloud: where end users want to have these points of interoperability.

Joe Brockmeier, open source cloud computing evangelist, Citrix Sept. 6, 2012, „Defining the Open Cloud.Some folks will argue that a cloud service or offering is open if it has open APIs and open standards.  For my money, the best definition of the open cloud came from Red Hat’s Scott Crenshaw: It enables portability across clouds; Has a pluggable, extensible, and open API; Lets you deploy to your choice of infrastructure; It’s unencumbered by patents and other IP restrictions; It’s based on open standards; It has a viable and independent community; It is open source. Having open APIs is necessary, but it’s not enough. If you depend on one vendor to provide your cloud, and you can’t pull it in-house for any reason, it’s just not open.

So are we any smarter after this, only little I fear. There are common themes, like open source, standards, interfaces, royalty-freeness from a technology ankle and freedom of choice and cross-cloud portability on the other. And perhaps the community should indeed take this one Open Cloud Manifesto principle on „open cloud community collaboration“ to heart and drive consolidation; instead of contributing to one of those fundamental innovation hindrances that they try to avoid, that is fragmentation. The CloudOpen conference was one great step into this direction and the Google Hangout on OpenClouds (on Youtube) by Ben Kepes (Ben Kepes on G+) will hopefully be another way to continue this important discussion.

ANNEX

Open Cloud InitiativeOpen Cloud Principles (OCP)

Interoperability (the ability to exchange and use information) between cloud computing products and services is required for unfettered competition between vendors and unrestricted choice for users.

Users must be able to come (no barriers to entry) and go (no barriers to exit) regardless of who they are (no discrimination) and what systems they use (technological neutrality).

Supporting vendors must therefore cooperate on standards, implementing those that exist (where applicable) and collaborating via an open process to develop those that don’t, with a view to competing fairly on quality.
Open Cloud must meet the following requirements:
  • Open Formats: All user data and metadata must be represented in Open Standard formats.
  • Open Interfaces: All functionality must be exposed by way of Open Standard interfaces.

Open Standards must meet the following requirements:

  • Copyrights: The standard must be documented in all its details, published and both accessible and [re]usable free of charge.
  • Patents: Any patents possibly present on [parts of] the standard must be irrevocably made available on a royalty-free basis.
  • Trademarks: Any trademarks possibly present on identifier(s) must be used for non-discriminatory enforcement of compliance only.
  • Implementations: There must be multiple full, faithful, independent and interoperable implementations (for both client and server where applicable) and at least one such implementation must be licensed in its entirety under an Open Source Initiative (OSI) approved license or placed into the public domain.

Red Hat : An open cloud has the following characteristics:

  • Is open source. This allows adopters to control their particular implementation and doesn’t restrict them to the technology and business roadmap of a specific vendor. It lets them build and manage clouds that put them in control of their own destiny and provides them with visibility into the technology on which they’re basing their business. It provides them with the flexibility to run the workloads of their choice, including proprietary ones, in their cloud. Open source also lets them collaborate with other communities and companies to help drive innovation in the areas that are important to them.
  • Has a viable, independent community. Open source isn’t just about the code, its license, and how it can be used and extended. At least as important is the community associated with the code and how it’s governed. Realizing the collaborative potential of open source and the innovation it can deliver to everyone means having the structures and organization in place to tap it fully.
  • Is based on open standards, or protocols and formats that are moving toward standardization and that are independent of vendor and platform. Standardization in the sense of “official” cloud standards blessed by standards bodies is still in early days. That said, approaches to interoperability that aren’t under the control of individual vendors and that aren’t tied to specific platforms offer important flexibility. This allows the API specification to evolve beyond implementation constraints and creates the opportunity for communities and organizations to develop variants that meet their individual technical and commercial requirements.
  • Freedom to use IP.  Recent history has repeatedly shown that there are few guarantees that intellectual property (IP) assets will remain accessible to all from one day to the next.  To have confidence that you will continue to enjoy access to IP assets that you depend on under the terms that you depend on, permission needs to be given in ways that make that technology open and accessible to the user.  So-called “de facto standards,” which are often “standards” only insofar as they are promoted by a large vendor, often fail this test.
  • Is deployable on your choice of infrastructure. Hybrid cloud management should provide an additional layer of abstraction above virtualization, physical servers, storage, networking, and public cloud providers. This implies, or indeed requires, that cloud management be independent of virtualization and other foundational technologies. This is a fundamental reason that cloud is different from virtualization management and a fundamental enabler of hybrid clouds that span physical servers, multiple virtualization platforms, and a wide range of public cloud providers including top public clouds.
  • Is pluggable and extensible with an open API. This lets users add features, providers, and technologies from a variety of vendors or other sources. Critically, the API itself cannot be under the control of a specific vendor or tied to a specific implementation but must be under the auspices of a third-party organization that allows for contributions and extensions in an open and transparent manner. Deltacloud, an API that abstracts the differences between clouds, provides a good example. It is under the auspices of the Apache Software Foundation and is neither a Red Hat-controlled project nor tied to a particular implementation of cloud management.
  • Enables portability to other clouds. Implicit in a cloud approach that provides support for heterogeneous infrastructure is that investments made in developing for an open cloud must be portable to other such clouds. Portability takes a variety of forms including programming languages and frameworks, data, and the applications themselves. If you develop an application for one cloud, you shouldn’t need to rewrite it in a different language or use different APIs to move it somewhere else. Furthermore, a consistent runtime environment across clouds means that retesting and requalification isn’t needed every time you want to redeploy.

Open Cloud Manifesto : Open Cloud Principles

Rather, as cloud computing matures, there are several key principles that must be followed to ensure the cloud is open and delivers the choice, flexibility and agility organizations demand:
1. Cloud providers must work together to ensure that the challenges to cloud adoption (security, integration, portability, interoperability, governance/management, metering/monitoring) are addressed through open collaboration and the appropriate use of standards.
2. Cloud providers must not use their market position to lock customers into heir particular platforms and limit their choice of providers.
3. Cloud providers must use and adopt existing standards wherever appropriate. The IT industry has invested heavily in existing standards and standards organizations; there is no need to duplicate or reinvent them.
4. When new standards (or adjustments to existing standards) are needed, we must be judicious and pragmatic to avoid creating too many standards. We must ensure that standards promote innovation and do not inhibit it.
5. Any community effort around the open cloud should be driven by customer needs, not merely the technical needs of cloud providers, and should be tested or verified against real customer requirements.
6. Cloud computing standards organizations, advocacy groups, and communities should work together and stay coordinated, making sure that efforts do not conflict or overlap.
Leaders of the Open Cloud at OpenCloud

Richard Kaufmann, chief technologist, HP Cloud Services
July 3, 2012, „HP Public Cloud Aims to Boost OpenStack Customer Base.“ 

There are two important APIs out there, one is Amazon’s and the other is OpenStack. And OpenStack has Amazon compatibility. HP will continue to support those Amazon compatibility layers; We’re not trying to lead on a position about what customers should do with APIs… I (personally) believe there should be a popular cloud API for IaaS and it should not be Amazon. It could be anything else but it can’t float from above. It has to be based on popular usage.

Lew Moorman, Rackspace
July 10, 2012, „Open Cloud Key to Modern Networking.“

Some people seem to think that APIs are the cloud and one thing that made the cloud so revolutionary is it’s programmatically accessible by API.  But (Amazon) S3 is a really complex distributed system. The issue with a model that says “clone Amazon” is that, unless you have the core technology underneath it, you can’t have a cloud…

OpenStack is really setting out to build an open alternative from end to end. They say we’re going to do networking, not just set out to copy Amazon. We need to really innovate and build a visionary system that can power the future of computing. Amazon, VMware and Microsoft don’t have all the answers.

Christopher Brown, CTO, Opscode
July 17, 2012, „Chef Offers a Recipe for the Open Source Cloud.“ 

The open cloud lies both below and above the waterline of the API. At the beginning we all wanted to treat the cloud as an easier way to get the compute units that looked like the old thing we used to get buying a physical machine. But that’s not actually true. It’s not the same thing and it requires a different design underneath of a common substrate. If you look above the water line at the consumer, the way you build applications, the way they scale, etc., designing the cloud and for the cloud are different than what is now legacy.

Mark Hinkle, senior director of cloud computing community, Citrix
July 24, 2012, „Citrix’s Hinkle Proposes Linux Model for an Open Source Cloud.“ 

It’s first and foremost that the orchestration platform is open source. The data you store within the cloud is open to you as the end user in a format you can manipulate easily and it’s easily transferable. The API is also open and clearly documented.

Ross Turk, vice president of community, InkTank
July 31, 2012, „An Open Source Storage Solution for the Enterprise.“ 

It can mean a cloud stack that is built on open source like OpenStack or CloudStack and that reflects the economic and community advantages behind building something that’s akin to what Amazon has done, but built on commodity hardware. It’s an open alternative to AWS.

Another way to think of the open cloud doesn’t exclude AWS. It’s having cloud services with standardized APIs so applications written for one cloud can work on another cloud.

Imad Sousou, director of Intel’s Open Source Technology Center
Aug. 7, 2012, „Open Cloud Standards will Emerge With More Collaboration.“

The open cloud must meet these requirements:  Use Open Formats, where all user data and metadata must be represented in Open Standard formats. Use Open Interfaces, where functionality must be exposed through Open Standard interfaces.

In addition, in the open cloud, various open source technologies should be available to build the right solutions efficiently, and to drive innovation. These would include software stack and tools, such as the hypervisors or operating systems, middleware, such as databases and web servers, web content management systems, and development tools and languages. Such open source-based software solutions would reinforce interoperability of the open cloud.

Kyle MacDonald, vice president of cloud, Canonical
Aug. 14, 2012, „Canonical: Making the Open Cloud Seamless for Users.“ 

True power comes when the users can move from one cloud service to another. That’s the state of nirvana. The cool word is ‘interoperability.’ …  It will be almost required that if you’re a cloud service you publish an API that’s clear. And eventually there will be a common API or it becomes so simple the minor differences won’t be a big deal to end users. And then partners who define the services can use those same open source technologies and provide a good service.

Alan Clark, director of industry initiatives, emerging standards and open source, SUSE
Aug. 21, 2012, „SUSE Aims for One-Click Enterprise Deployment on OpenStack.“ 

Enterprise IT must deliver the most efficient, scalable and flexible services possible. The open cloud provides that through the ability to have a flexible infrastructure, quick and easy deployment, service management and complete life cycle management.

We’re working with partners — many are part of these open source projects – to build this together and that builds interoperability. It’s a collaboration of ideas as well as code. It accelerates bringing a solution to market that works across all the different partners.

Angel Diaz, vice president of software standards and cloud, IBM
Aug. 28, 2012, „3 Projects Creating User-Driven Standards for the Open Cloud.“ 

Our clients who use technology have a heterogeneous environment. They need to take existing systems, extend them and deal with it and they don’t want to be locked into a singe vendor solution. That is how (IBM) defines an open cloud: where end users want to have these points of interoperability.

Joe Brockmeier, open source cloud computing evangelist, Citrix
Sept. 6, 2012, „Defining the Open Cloud.“ 

Some folks will argue that a cloud service or offering is open if it has open APIs and open standards.  For my money, the best definition of the open cloud came from Red Hat’s Scott Crenshaw: It enables portability across clouds; Has a pluggable, extensible, and open API; Lets you deploy to your choice of infrastructure; It’s unencumbered by patents and other IP restrictions; It’s based on open standards; It has a viable and independent community; It is open source.

Having open APIs is necessary, but it’s not enough. If you depend on one vendor to provide your cloud, and you can’t pull it in-house for any reason, it’s just not open.

FOOTNOTES

[footnote1]  This is symptom and cause, concurrently, for the Zurich’s rise as the European Silicon Valley, as some already claim, based on its flourishing high-tech start-up scene that enjoys a wide array of support and attention (e.g. Zurich Start-Up PortalStartwerk PortalZurich Start-up Weekend, Zurich StartupsStartup.ch).

[footnote2] Apologies to one of my previous employers. I really didn’t mean to mention the evil.

 

Open- or NotSo-OpenStack? – Comments on the OpenStack Foundation

An analysis of the OpenStack Foundation and its technical governance approach. Originally published at the ICCLab

Contributions to Open Source Software (OSS) projects are an excellent means to foster broad uptake of  innovations and has therefore become indispensable for research and development in computer science.

With the Internet allowing ubiquitous collaboration (e.g. between OSS software developers, OSS community managers, OSS document editors, etc) of all sorts, across all backgrounds, and locations spread over the entire globe, some OSS projects are so successful that they reach sizes (and budgets) that are comparable to full-blown companies.

Contributing to OSS is also an unparalleled frank (and in times brutal) means for receiving feedback by an expert community. OSS communities are commonly governed by technical meritocracy, a term inherently subjective and thus reliable warrant of controversy. Contributions are in turn relentlessly scrutinized, the latter not seldom amplified by the fact that the motivation for OSS contribution is reward by community appreciation (instead of financial compensation), a principle that renders OSS environments highly competitive, in particular for highly popular projects like Linux, Apache, OpenStack and the likes.

We, at the ICCLab, consider it paramount to deliver our ideas and innovations as running code to a few carefully selected and relevant OSS projects to get receive feedback that validates our ideas and to ensure that our ideas and innovations gain support and uptake by the community. This is an inherent element of our impact-centric research methodology.

The powers of OSS is exemplified by the OpenStack project. It emerged out of a merger between NASA and Rackspace, who both have developed their own IaaS framework but decided to cooperate for the sake of creating a serious competitor to existing incumbents, like Amazon and VMWare. This initial motivation of the founders continues to materialize and the project enjoys comprehensive community support backed up by significant financial and organizational backing by some of the most influential industry incumbents.

OpenStack meanwhile became (supposedly) the largest OSS project since Linux and reached a size significantly larger than Linux. Such growth pushes organizational structures of any OSS project to the limits. If also imposes a hefty burden onto founding members, for OpenStack in particular onto Rackspace who managed the project from an administrative perspective.

A common way out of this is to transform the organisation into an foundation, like for instance the Apache Foundation  or the Linux Foundation, and this was applied to OpenStack too. With the beginning of September 2012 the OpenStack Foundation is in charge of the OpenStack project. The advantages are evident; professional structures, comprehensive governance, and financial management. All this fosters trust as it leads OpenStack out of a loosly coupled community project into a trustworthy company-style enterprise.

But such industry-grade and -oriented advantages come at a price. While native (pure) OSS projects share powers based on meritocracy, derived from technical  expertise and commitment to the project, foundations are characterized by a significant financial dimension, and the OpenStack Foundation does not make an exception. The difficult part of this is the balanace between professional structures, the financial backing required, and the value (influence) provided to those that are willing to invest cash on one hand, and on the other to preserve the drive and nature of the OSS movement, that is technical liberty and community recognition.

The OpenStack compromise to this issue is documented in the OpenStack Foundation Bylaws. This document lays out the general framework (not to say powers) and thus puts any person and institution committed to OpenStack – just like the ICCLab – in an unequivocal context. The question therefore is: What are the implications of the OpenStack Foundation?

Our initial analysis goes here ICCLab : The OpenStack Project and Foundation

Feedback much welcome!