Cloudscaling | Open Cloud Solutions

Vancouver OpenStack Summit – EMC Federation Presentations

Posted on February 18, 2015 by Randy Bias

Voting for presentations at the Vancouver OpenStack Summit is now open. Please help us by voting on the sessions submitted by EMC Federation speakers along with any other sessions that cover topics that might interest you. Please vote at your earliest convenience since each vote helps!

OpenStack community members are voting on presentations to be presented at the OpenStack Summit, May 18-22, in Vancouver, Canada. Hundreds of high-quality submissions were received, and your votes can help determine which ones to include in the schedule.

PDF containing all of the submissions by the EMC Federation:

https://my.syncplicity.com/share/6ndhnwrkpxkgodp/OpenStack%20Liberty%20Summit%20-%20Vancouver%20-%20EMC%20Federation%20Session%20Proposals%20v2

Here is a list you can click through to vote.

Cinder: Efforts in Cinder to provide quality as well as compatibility
- www.openstack.org/vote-vancouver/presentation/efforts-in-cinder-to-provide-quality-as-well-as-compatibility
Storage Use-Cases in OpenStack
- www.openstack.org/vote-vancouver/presentation/storage-use-cases-in-openstack
Highly Available OpenStack: From Theory to Reality
- www.openstack.org/vote-vancouver/presentation/highly-available-openstack-from-theory-to-reality
State of the Stack v4
- www.openstack.org/vote-vancouver/presentation/state-of-the-stack-v4
Highly scalable distributed block storage with ScaleIO
- www.openstack.org/vote-vancouver/presentation/highly-scalable-distributed-block-storage-with-scaleio
Software-defined Storage: the future of OpenStack storage systems
- www.openstack.org/vote-vancouver/presentation/software-defined-storage-the-future-of-openstack-storage-systems
Panel: Product Management Strategies for OpenStack
- www.openstack.org/vote-vancouver/presentation/bridging-the-gap-between-app-developers-operators-and-technology-product-management-in-the-openstack-ecosystem
Have your Cake and Eat it too! Monitoring OpenStack in Layers
- www.openstack.org/vote-vancouver/presentation/plugging-the-ceilometer-firehose-into-nagios
Things you need to know before you containerize your OpenStack deployment
- www.openstack.org/vote-vancouver/presentation/things-you-need-to-know-before-you-containerize-your-openstack-deployment
State of OpenStack Product Management
- www.openstack.org/vote-vancouver/presentation/state-of-openstack-product-management
Ask the Experts: Are Containers a Threat to OpenStack?
- www.openstack.org/vote-vancouver/presentation/ask-the-experts-are-containers-a-threat-to-openstack
What Your Customers Don’t Know About OpenStack Might Hurt You
- www.openstack.org/vote-vancouver/presentation/what-your-customers-dont-know-about-openstack-might-hurt-you
OpenStack Data Storage Deep Dive
- www.openstack.org/vote-vancouver/presentation/openstack-storage-deep-dive
OpenStack Operations
- www.openstack.org/vote-vancouver/presentation/operating-an-openstack-environment-a-focus-look-on-monitoring-logging-and-troubleshooting
Architecting OpenStack Monitoring for Next-Generation Functionality
- www.openstack.org/vote-vancouver/presentation/architecting-openstack-monitoring-for-next-generation-functionality
Helios Burn: A REST API Fault Injection Platform
- www.openstack.org/vote-vancouver/presentation/heliosburn-a-rest-fault-injection-platform
Modernizing Your Applications
- www.openstack.org/vote-vancouver/presentation/modernizing-your-applications
How to build a case for a ITaaS Business Case
- www.openstack.org/vote-vancouver/presentation/how-to-build-a-case-for-a-itaas-business-case
Using Cinder with EMC Block Storage Systems: Reference Architecture
- www.openstack.org/vote-vancouver/presentation/using-cinder-with-emc-block-storage-systems-reference-architecture
Building Scalable Networks for Fun and Profit
- www.openstack.org/vote-vancouver/presentation/building-scalable-networks-for-fun-and-profit
What’s Next in OpenStack: A Glimpse at the Roadmap
- www.openstack.org/vote-vancouver/presentation/whats-next-in-openstack-a-glimpse-at-the-roadmap
Meet the Ambassadors
- www.openstack.org/vote-vancouver/presentation/ambassadors-community-report-panel-talk
OpenStack for VMware Administrators
- www.openstack.org/vote-vancouver/presentation/openstack-for-vmware-administrators
Leveraging Your Existing DC Investments
- www.openstack.org/vote-vancouver/presentation/leveraging-your-existing-dc-investments
Leveraging Congress for Policy Management
- www.openstack.org/vote-vancouver/presentation/leveraging-congress-for-policy-management
DefCore and Me
- www.openstack.org/vote-vancouver/presentation/defcore-and-me
DefCore 2015
- www.openstack.org/vote-vancouver/presentation/defcore-2015
Community OpenStack Training Wants to Come a User Group Near You!
- www.openstack.org/vote-vancouver/presentation/community-openstack-training-wants-to-come-a-user-group-near-you
Leveraging vSphere Virtual Distributed Switches and NSX vSphere for Neutron
- www.openstack.org/vote-vancouver/presentation/leveraging-vsphere-virtual-distributed-switches-and-nsx-vsphere-for-neutron
Guided Lab for Learning All Aspects of OpenStack
- www.openstack.org/vote-vancouver/presentation/guided-lab-for-learning-all-aspects-of-openstack
404 Error: Unicorn Stack Not Found
- www.openstack.org/vote-vancouver/presentation/404-error-unicornstack-not-found
Openstack Networking Introduction Hands on Lab
- www.openstack.org/vote-vancouver/presentation/openstack-networking-introduction-hands-on-lab
Openstack Networking Advanced Hands on Lab
- www.openstack.org/vote-vancouver/presentation/openstack-networking-advanced-hands-on-lab
Congress HOL
- www.openstack.org/vote-vancouver/presentation/congress-hands-on-lab
Practical Lessons from real world Multi-Hypervisor deployments
- www.openstack.org/vote-vancouver/presentation/practical-lessons-learned-in-a-production-multi-hypervisor-openstack-deployment
How to run and upkeep a 3rd party Openstack community CI
- www.openstack.org/vote-vancouver/presentation/how-to-run-and-upkeep-a-3rd-party-openstack-community-ci
Governing (Murano) Application Deployment with (Congress) Policy
- www.openstack.org/vote-vancouver/presentation/governing-murano-application-deployment-with-congress-policy
Helping Telcos go Green and save OpEx via Policy
- www.openstack.org/vote-vancouver/presentation/helping-telcos-go-green-and-save-opex-via-policy
Panel: Defining Policy Frameworks for Openstack
- www.openstack.org/vote-vancouver/presentation/panel-defining-policy-frameworks-for-openstack
Panel: Neutron Considerations in Production environments
- www.openstack.org/vote-vancouver/presentation/user-panel-neutron-considerations-in-production-environments
Container Networking models with OpenStack Neutron
- www.openstack.org/vote-vancouver/presentation/secure-networking-models-for-containersdocker-and-vms-on-neutron-networks
Neutron – Past, Present & Future of Cloud Networking
- www.openstack.org/vote-vancouver/presentation/neutron-past-present-and-future-of-cloud-networking
OpenStack A La VMware: Made to Order
- www.openstack.org/vote-vancouver/presentation/openstack-a-la-vmware-made-to-order
Demystifying NSX: A Technology Deep Dive on Neutron with NSX
- www.openstack.org/vote-vancouver/presentation/demystifying-nsx-a-technology-deep-dive-on-neutron-with-nsx
Open Virtual Network (OVN) Native Virtual Networking for Open vSwitch
- www.openstack.org/vote-vancouver/presentation/ovn-native-virtual-networking-for-open-vswitch
Implementing zero-trust micro-segmentation architecture with Neutron
- www.openstack.org/vote-vancouver/presentation/implementing-zero-trust-micro-segmentation-with-neutron-and-nsx
Diving Deeper into Nova and VMware ESXi
- www.openstack.org/vote-vancouver/presentation/diving-deeper-into-nova-and-vmware-esxi
OpenStack VMware Enabling the Evolution of Enterprise Applications at Adobe
- www.openstack.org/vote-vancouver/presentation/openstack-vmware-enabling-the-evolution-of-enterprise-applications-at-adobe
Extending OpenStack Congress to the PaaS layer for Next-Gen App Policy Controls
- www.openstack.org/vote-vancouver/presentation/extending-openstack-congress-to-the-paas-layer-for-next-gen-app-policy-controls
Cloud Foundry on OpenStack Hands-on: It’s what on the Stack that matters!
- www.openstack.org/vote-vancouver/presentation/cloudfoundry-on-openstack-hands-on-its-what-on-the-stack-that-matters

Thank you so much for taking the time to vote on these sessions!

Posted in OpenStack | Leave a comment

The Future of OpenStack’s EC2 APIs

Posted on February 12, 2015 by Randy Bias

Recently, some more talk was had around the future of the EC2 APIs, beginning with some comments on the openstack-operators mailing list, followed by threads on the dev and foundation mailing list. This ultimately resulted in a suggested commit to officially “deprecate” the EC2 APIs from Nova. This commit was rejected, but make sure you read through the commentary if you have time. Some really great perspective. If you don’t here’s my basic summation:

Many people still very much care about the EC2 and AWS APIs and are quite concerned about their state and the lack of attention to keeping them current
Some people are adamant about deprecating and then removing them as expeditiously as possible
Others are interested in keeping them around, but moving them out of the default distribution and making sure they have a good home

As many people know, I’m passionate about this subject. If you missed the blog posting that caused a massive kerfuffle in summer of 2013, now is the time to take a look again: OpenStack’s Future Depends on Embracing Amazon. Now. There was a pretty massive response to that original article, including a very vibrant OpenStack meetup with a debate that was covered live between myself and Boris Renski, co-founder of Mirantis.

I am proud of driving that conversation, but one pushback that arose could be summarized as: “put your money where your mouth is”. At the time we were already working towards a goal that would have responded to this pushback but it’s taken alot longer than I would like to materialize.

We are finally there. Let me explain.

The StackForge Standalone EC2 API

It’s taken a while and the entire backstory and history isn’t really relevant for this article, but Cloudscaling (now part of EMC) has been working diligently to build a drop-in replacement for the existing Nova EC2 API. This standalone EC2 API can be found in StackForge. This re-implementation of the EC2 APIs is now ready for prime time and serendipitously you can see from the opening comments that the community is very interested in adopting it.

Some details on the status of the new EC2 API can be found in the initial documentation in the commit.

To summarize, the new standalone API:

Is feature complete and at parity with the existing Nova EC2 API
Has equivalent or better test coverage to the existing Nova EC2 API
Is configured by default on a different port (can be run in parallel to all existing APIs)
Included a new set of features in the form of full coverage for the VPC APIs (a subset of EC2)
Has been tested exhaustively with the AWS unified CLI tool, a python CLI for driving all of the AWS services
Calls the OpenStack REST APIs rather than any of the “internal API” or function calls for a clean separation of duties
AWS tempest tests have been expanded, tested against AWS itself as a baseline then used to validate this APIs behavior

This is very exciting and it’s what the community has been asking for. More importantly, to me, at least, is that the EC2 API could potentially stay in StackForge and become an optional plugin to OpenStack, letting those who care use it while also allowing the team who is maintaining it to iterate at a slightly different speed from the current 6-month integrated release cycle.

For those who are wondering, it’s EMC’s intention to continue to invest into and maintain this API bridge to OpenStack.

The EC2 API Still Matters to OpenStack

During the “debate” that occurred in 2013, I was frequently bemused by the attempts of community members to downplay the importance of the EC2 APIs. I think it’s all settled down now and generally accepted that we want the EC2 APIs to live on and succeed in OpenStack-land and hopefully we’ll even support other APIs down the road.

For those who are still holdouts though, I think the latest OpenStack User Survey data continues to reinforce how important the EC2 and other AWS APIs are:

What’s enlightening here is that in 2013 I was hearing the constant refrain of “the EC2 APIs are used by only a ‘fraction’ of the community”. That ‘fraction’ was *merely* ~30-35% at the time according to the user surveys. As you can see, usage of the EC2 APIs has actually increased since that time and now we’re at 44% for production deployments, a 25% increase in roughly 18 months. This is very important.

It means that usage of the EC2 APIs is increasing, fairly dramatically, over time.

I’ll reiterate again, since folks still sometimes get confused, I’m not advocating dropping the OpenStack APIs in favor of AWS. I’m advocating embracing the AWS APIs, making them a first class citizen, and viewing AWS as a partner, not an enemy. A partner in making cloud big for everyone.

This reality inside the OpenStack community is starting to materialize and I need your help.

The Game Plan

Awesome, we have a new set of improved EC2 APIs, a path towards supporting them and deprecating the old. Whether you love the EC2 APIs or hate them, it’s good for everyone to move them out of the default deployment, create greater isolation between these APIs and OpenStack internals, and to have a path forward where they can be maintained with love.

Everybody wins, even the detractors.

Well, to get this the rest of the way, we need to do the following:

Test, test, test: if you are using the existing EC2 APIs, please give these a try, break them, and file bugs
If you are a developer and want to help cover any gaps in functionality or bugs that have been found, then get involved now; this is a standard stackforge project, so anyone can get in the mix
There are some known challenges in the existing OpenStack APIs that need to be addressed for a more robust solution; these are documented in a new blueprint you can find here
Help update and maintain the documentation so people know that this capability is available for their OpenStack deployments/products, whether DIY or product based
Add a set of testing capabilities to RefStack to test for “AWS interoperability” alongside “OpenStack interoperability”

I really appreciate all of the supporters and also detractors who have been involved in this discussion. I believe that this kind of debate and action, like the Internet before it and the IETF mantra of old (“running code and rough consensus”), is what makes OpenStack great. Completing this project will also provide us a blueprint for how we support the public APIs of other public clouds in OpenStack-land.

Finally, a big thanks to Alex Levine, Feodor Tersin, and Andrey Pavlov, for being the tip of the spear on this work. Without them we wouldn’t have made it this far.

Posted in OpenStack | Leave a comment

“Vanilla OpenStack” Doesn’t Exist and Never Will

Posted on February 3, 2015 by Randy Bias

One of the biggest failures of OpenStack to date is expectation setting. New potential OpenStack users and customers come into OpenStack and expect to find:

A uniform, monolithic cloud operating system (like Linux)
Set of well-integrated and interoperable components
Interoperability with their own vendors of choice in hardware, software, and public cloud

Unfortunately, none of this exists and probably none of it should have ever been expected since OpenStack won’t ever become a unified cloud operating system.

The problem can be summed up by a request I still see regularly from customers:

I want ‘vanilla OpenStack’

Vanilla OpenStack does not exist, never has existed, and never will exist.

Examining The Request

First of all, it’s a reasonable request. The potential new OpenStack customer is indirectly asking for those things that led them to OpenStack in the first place. The bi-annual user survey has already told us what people care about:

The top reasons for OpenStack boil down to:

Help me reduce costs
Help me reduce or eliminate vendor lock-in

Hence the desire for “vanilla” OpenStack.

But what is “vanilla”? It could be a number of things:

Give me the “official” OpenStack release with no modifications
Give me OpenStack with all default settings
Give me an OpenStack that has no proprietary components

Which immediately leads into the next problem: what is “OpenStack”? Which could also be a number of things[1]:

Until recently, officially the principle trademark “OpenStack-powered” meant Nova + Swift *only*
The de facto “baseline” set of commonly deployed OpenStack services
1. Nova, Keystone, Glance, Cinder, Horizon, and Neutron
2. ~~There is no name or official stance on this arbitrary grouping~~
Use of DefCore + RefStack to test for OpenStack

UPDATE: to be more clear, the baseline set above *does* have a name. It is called “core” and called out in section 4.1 of the bylaws, which is below. I apologize for the confusion as “core” has been overloaded a fair bit in discussions on the board and at one point trademark rights were tied to “core”.

4.1 General Powers.

(a) The business and affairs of the Foundation shall be managed by or under the direction of a Board of Directors, who may exercise all of the powers of the Foundation except as otherwise provided by these Bylaws.
(b) The management of the technical matters relating to the OpenStack Project shall be managed by the Technical Committee. The management of the technical matters for the OpenStack Project is designed to be a technical meritocracy. The “OpenStack Project” shall consist of a “Core OpenStack Project,” library projects, gating projects and supporting projects. . The Core OpenStack Project means the software modules which are part of an integrated release and for which an OpenStack trademark may be used. The other modules which are part of the OpenStack Project, but not the Core OpenStack Project may not be identified using the OpenStack trademark except when distributed with the Core OpenStack Project. The role of the Board of Directors in the management of the OpenStack Project and the Core OpenStack Project are set forth in Section 4.13. On formation of the Foundation, the Core OpenStack Project is the Block Storage, Compute, Dashboard, Identity Service, Image Service, Networking, and Object Storage modules. The Secretary shall maintain a list of the modules in the Core OpenStack Project which shall be posted on the Foundation’s website.

So this is helpful, but still confusing. If, for example, you don’t ship Swift, which some OpenStack vendors do not, then technically you can’t call your product OpenStack-powered. HP’s public cloud and Rackspace’s public clouds, last I checked anyway, don’t use the identity service (Keystone), which also means that technically they can’t be “OpenStack-powered” either. A strict reading of this section also says that all projects that are in “integrated” status are also part of “core” and that you can’t identify “core” with an OpenStack trademark unless “core” is distributed together, which implies that if you don’t have Sahara, then you aren’t OpenStack. Which, of course makes no sense.

So my point still stands. There has been a disconnect between how OpenStack is packaged up by vendors, how the trademarks are used, and how integrated is defined and contrasted to “core”, etc.

This is why item #3 above is still in motion and is the intended replacement model for #1. You can find out more about DefCore’s work here.

Understanding The OpenStack Trademark and Its History

It’s not really a secret, but it’s deeply misunderstood: until the last few weeks, the Bylaws very specifically said that “OpenStack-powered” is Nova plus Swift. That’s it. No other projects were included in the definition. Technically, many folks who shipped an “OpenStack-powered” product without Swift were not actually legally allowed to use the trademark and brand. This was widely unenforced because the Board and Foundation knew the definitions were broken. Hence DefCore.

Also, the earliest deployments out there of OpenStack were Swift-only. Cloudscaling launched the first ever production deployment of OpenStack outside of Rackspace in January of 2011,