Canonical Voices

Posts tagged with 'ensemble'

Etherpad Lite is awesome. Have you ever organized a conference, large meeting, or foocamp and thought “wow, I could really have used an etherpad at this event.” But many times organizers don’t have the resources to get this kind of stuff running. Now, in an effort to make it trivial for anyone to set up an etherpad, James has implemented an etherpad lite formula for ensemble. 

Here are the steps. First install and configure Ensemble, this is just putting your AWS keys in an environment file and/or using a PPA if you’re not on 11.10. Then we bootstrap, snag James’ formula, and then deploy it in EC2.

ensemble bootstrap
bzr branch lp:~ensemble-composers/+junk/etherpad-lite
ensemble deploy --repository= . etherpad-lite

Right now we’re grabbing formulas from Launchpad but in the future that will be much more user friendly, where you can search for formulas right from the ensemble command (Like apt-cache search). It takes about 5 minutes to bootstrap, and then another 5-10 minutes to deploy, since it’s installing everything we need for Etherpad Lite and then installing it on EC2.

Then let’s check in how that’s going with:

ensemble status
2011-08-30 13:24:33,166 INFO Connecting to environment.
machines:
  0: {dns-name: ec2-107-20-19-203.compute-1.amazonaws.com, instance-id: i-6b706b0a}
  1: {dns-name: ec2-107-20-7-236.compute-1.amazonaws.com, instance-id: i-8f6b70ee}
services:
  etherpad-lite:
    formula: local:etherpad-lite-5
    relations: {}
    units:
      etherpad-lite/0:
        machine: 1
        relations: {}
        state: started
2011-08-30 13:24:35,883 INFO 'status' command finished successfully

Ok as we can see there the service has state:started, which means it’s running. Node 1 is our instance so let’s hit it up, we hit up port 9001 because that’s the Etherpad Lite’s default port:

http://ec2-50-17-151-64.compute-1.amazonaws.com:9001/

Neat huh? Ok so I don’t have a fancy URL like pad.socallinuxexpo.org (Hint hint Gareth!) but you get the idea. I’ll leave it running for a while so you can mess with it. When your conference is done, save your data and tear it all down with

ensemble shutdown

or keep it around, up to you.

Currently it’s only running in a single instance, there’s no logic in the formula for load balancing or anything yet, but James tells me it scales pretty nicely on just once instance. You’ll also note that I don’t have to know/care about building node.js or npm or any of those dependencies. The formula author basically does the work for everyone, ONCE, and then people like me who aren’t familiar with deploying this sort of app can just enjoy using Etherpad Lite.

Ok so now $your_favorite_event can have an etherpad very easily. We’ve already made it easy to get mediawiki, it’s my hope that eventually we’ll have formulas for every service a conference would need to get running so they can just fire it up.

This is just an example on the public cloud, later on we’ll show you how to deploy to bare metal using these exact commands. 

Read more
Dustin Kirkland

I've seen Ensemble evolve from a series of design-level conversations (Brussels May 2010), through a year of fast-paced Canonical-style development, and participated in Ensemble sprints (Cape Town March 2011, and Dublin June 2011).  I've observed Ensemble at first as an outsider, then provided feedback as a stake-holder, and have now contributed code as a developer to Ensemble and authored Formulas.


Think about bzr or git circa 2004/2005, or apt circa 1998/1999, or even dpkg circa 1993/1994...  That's where we are today with Ensemble circa 2011. 

Ensemble is a radical, outside-of-the-box approach to a problem that the Cloud ecosystem is just starting to grok: Service Orchestration.  I'm quite confident that in a few years, we're going to look back at 2011 and the work we're doing with Ensemble and Ubuntu and see an clear inflection point in the efficiency of workload management in The Cloud.

From my perspective as the leader of Canonical's Systems Integration Team, Ensemble is now the most important tool in our software tool belt when building complex cloud solutions.

Period.

Juan, Marc, Brian, and I are using Ensemble to generate modern solutions around new service deployments to the cloud.  We have contributed many formulas already to Ensemble's collection, and continue to do so every day.

There's a number of novel ideas and unique approaches in Ensemble.  You can deep dive into the technical details here.  For me, there's one broad concept in Ensemble that just rocks my world...  Ensemble deals in individual service units, with the ability to replicate, associate, and scale those units quite dynamically.  Service units in practice are cloud instances (or if you're using Orchestra + Ensemble, bare metal systems!).  Service units are federated together to deliver a (perhaps large and complicated) user facing service.

Okay, that's a lot of words, and at a very high level.  Let to me try to break that down into something a bit more digestable...

I've been around Red Hat and Debian packaging for many years now.  Debian packaging is particularly amazing at defining prerequisites packages, pre- and post- installation procedures, and are just phenomenal at rolling upgrades.  I've worked with hundreds (thousands?) of packages at this point, including some mind bogglingly complex ones!

It's truly impressive how much can be accomplished within traditional Debian packaging.  But it has its limits.  These limits really start to bare their teeth when you need to install packages on multiple separate systems, and then federate those services together.  It's one thing if you need to install a web application on a single, local system:  depend on Apache, depend on MySQL, install, configure, restart the services...

sudo apt-get install your-web-app

...

Profit!

That's great.  But what if you need to install MySQL on two different nodes, set them up in a replicating configuration, install your web app and Apache on a third node, and put a caching reverse proxy on a fourth?  Oh, and maybe you want to do that a few times over.  And then scale them out.  Ummmm.....

sudo apt-get errrrrrr....yeah, not gonna work :-(

But these are exactly the type(s) of problems that Ensemble solves!  And quite elegantly in fact.

Once you've written your Formula, you'd simply:

ensemble bootstrap

ensemble deploy your-web-app
...
Profit!

Stay tuned here and I'll actually show some real Ensemble examples in a series of upcoming posts.  I'll also write a bit about how Ensemble and Orchestra work together.

In the mean time, get primed on the Ensemble design and usage details here, and definitely check out some of Juan's awesome Ensemble how-to posts!

After that, grab the nearest terminal and come help out!

We are quite literally at the edge of something amazing here, and we welcome your contributions!  All of Ensemble and our Formula Repository are entirely free software, building on years of best practice open source development on Ubuntu at Canonical.  Drop into the #ubuntu-ensemble channel in irc.freenode.net, introduce yourself, and catch one of the earliest waves of something big.  Really, really big.

:-Dustin

Read more

Since every one else is doing it, I thought I’d play with more Ensemble tonight, but instead of firing something up I started working on a formula for summit, the tool we used to schedule UDS.

First Chris Johnston and Michael Hall started an etherpad with the instructions for installing summit (and we’re doing one for the LoCo directory too since we like biting off more than we can chew).

Here’s the first cut of the install script based on those instructions, then I went ahead and ran it in a VM to make sure it worked non-interactively. The documentation recommends that you have a plan before you start. Basically you are scripting an install on a brand new OS installation so you have to think of things you might normally take for granted, like remembering to install bzr or git before you pull something, heh:

When attempting to write a formula, it is beneficial to have a mental plan of what it takes to deploy the software. In our case, you should deploy drupal manually, understand where its configuration information is written, how the first node is deployed, and how further nodes are configured. With respect to this formula, this is the plan.

I did ok until I got to the python manage.py syncdb part of summit, which asked me a question, but not bad for the first shot. 

Of course, had I picked something packaged it wouldn’t be so complicated, my install script would just be an apt-get command but I think it’s useful to be able to just fire off an instance of summit right from trunk. 

The ability to just grab whatever you want right from trunk and fire off an instance is pretty powerful, I’m looking forward to seeing James Page’s etherpad-lite formula be ready so anyone can just fire one up for $your-favorite-conference.

Read more

People are talking a lot about Hadoop and Big Data. This is an area where a tool like Ensemble can really help out. Juan Negron has distilled the installation process for Hadoop on Ubuntu into a formula. 

Check out the instructions (1,2) that Mark was reading, and then you can see how Mark took all that Hadoop expertise and just used the Ensemble Formula to dramatically simplify the installation.

There is work going on to make the formulas easier to find and share, right now you pull it from a bzr branch, but the project is moving along at a fast rate, you can find out more about Ensemble on cloud.ubuntu.com.

Read more

Ahmed and I have tagged some Ensemble formula requests as bitesize. The PHP ones will be pretty straight forward, so if you’re deploying in EC2 and want to dive in and share your expertise (and snag some existing formulas for yourself!)

If you’re looking for a Formula for something you’d like to see to be made easy to deploy in EC2 please feel free to file a new bug.

You can find the docs for writing formulas here. If you’re writing an awesome app you’d like to see made easy to deploy in EC2, then let me know and we’ll get started.

Read more