Canonical Voices

Posts tagged with 'cloud'

Michael Hall

Sweet Chorus

Juju is revolutionizing the way web services are deployed in the cloud, taking what was either a labor-intensive manual task, or a very labor-intensive re-invention of the wheel  (or deployment automation in this case), and distilling it into a collection of reusable components called “Charms” that let anybody deploy multiple inter-connected services in the cloud with ease.

There are currently 84 Juju charms written for everything from game backends to WordPress sites, with databases and cache servers that work with them.  Charms are great when you can deploy the same service the same way, regardless of it’s intended use.  Wordpress is a good use case, since the process of deploying WordPress is going to be the same from one blog to the next.

Django’s Blues

But when you go a little lower in the stack, to web frameworks, it’s not quite so simple.  Take Django, for instance.  While much of the process of deploying a Django service will be the same, there is going to be a lot that is specific to the project.  A Django site can have any number of dependencies, both common additions like South and Celery, as well as many custom modules.  It might use MySQL, or PostgreSQL, or Oracle (even SQLite for development and testing).  Still more things will depend on the development process, while WordPress is available in a DEB package, or a tarball from the upstream site, a Django project may be anywhere, and most frequently in a source control branch specific to that project.  All of this makes writing a single Django charm nearly impossible.

There have been some attempts at making a generic, reusable Django charm.  Michael Nelson made one that uses Puppet and a custom config.yaml for each project.  While this works, it has two drawbacks: 1) It requires Puppet, which isn’t natural for a Python project, and 2) It required so many options in the config.yaml that you still had to do a lot by hand to make it work.  The first of these was done because ISD (where Michael was at the time) was using Puppet to deploy and configure their Django services, and could easily have been done another way.  The second, however, is the necessary consequence of trying to make a reusable Django charm.

Just for Fun

Given the problems detailed above, and not liking the idea of making config options for every possible variation of a Django project, I recently took a different approach.  Instead of making one Django Charm to rule them all, I wrote a small Django App that would generate a customized Charm for any given project.  My goal is to gather enough information from the project and it’s environment to produce a charm that is very nearly complete for that project.  I named this charming code “Naguine” after Django Reinhardt’s second wife, Sophie “Naguine” Ziegler.  It seemed fitting, since this project would be charming Django webapps.

Naguine is very much a JFDI project, so it’s not highly architected or even internally consistent at this point, but with a little bit of hacking I was able to get a significant return. For starters, using Naguine is about as simple as can be, you simply install it on your PYTHONPATH and run:

python manage.py charm --settings naguine

The –settings naguine will inject the naguine django app into your INSTALLED_APPS, which makes the charm command available.

This Kind of Friend

The charm command makes use of your Django settings to learn about your other INSTALLED_APPS as well as your database settings.  It will also look for a requirements.txt and setup.py, inspecting each to learn more about your project’s dependencies.  From there it will try to locate system packages that will provide those dependencies and add them to the install hook in the Juju  charm.

The charm command also looks to see if your project is currently in a bzr branch, and if it is it will use the remote branch to pull down your  project’s code during the install.  In  the future I hope to also support git and hg deployments.

Finally the command will write hooks for linking to a database instance on another server, including running syncdb to create the tables for your models, adding a superuser account with a randomly generated password and, if you are using South, running any migration scripts as well. It also writes some metadata about your charm and a short README explaining how to use it.

All that is left for you to do is review the generated charm, manually add any dependencies Naguine couldn’t find a matching package for, and manually add any install or database initialization that is specific to your project.  The amount of custom work needed to get a charm working is extremely minor, even for moderately complex projects.

Are you in the Mood

To try Naguine with your Django project, use the following steps:

  1. cd to your django project root (where your manage.py is)
  2. bzr branch lp:naguine
  3. python manage.py charm –settings naguine

That’s all you need.  If your django project lives in a bzr branch, and if it normally uses settings.py, you should have a directory called ./charms/precise/ that contains an almost working Juju charm for your project.

I’ve only tested this on a few Django project, all of which followed the same general conventions when it came to development, so don’t be surprised if you run into problems.  This is still a very early-stage project after all.  But you already have the code (if you followed step #2 above), so you can poke around and try to get it working or working better for your project.  Then submit your changes back to me on Launchpad, and I’ll merge them in.  You can also find me on IRC (mhall119 on freenode) if you get stuck and I will help you get it working.

(For those who are interested, each of the headers in this post is the name of a Django Reinhardt song)

Read more
Michael Hall

Starting today at 1500 UTC, we’ll be conducting a series of online classes for Ubuntu Developer Week.  Whether you are interest in developing new applications for Ubuntu, or want to make an existing app take advantage of all of Ubuntu’s features, this is definitely something you should attend.

This cycle Daniel Holbach will kick things off with a overview of Ubuntu development, using Bazaar and Launchpad to collaborate both online and off with teams of developers all over the world.

After that I will be giving an overview of the unique collection of technologies and services that Ubuntu offers application developers, including Unity integration, Ubuntu One cloud storage, and the Software Center.  Then I will be joined by Micha? Sawicz to talk about Ubuntu TV, and how you can get a development environment setup and start hacking on it yourself

Later, David Callé and Michal Hruby will be showing you how to integrate with the Unity Dash by writing custom lenses and scopes for your content.  And if you are interested in that, be sure to come back Thursday for my session on writing simple lenses and scopes in Python using the Singlet library.

Mark Mims and Dustin Kirland will both by presenting on different ways Ubuntu lets you take advantage of the latest cloud technology to improve the development, testing and deployment of your application and stack.  And Stuart Langridge will be talking about the latest developments in the Ubuntu One Database (U1DB), and then showing how you can integrate our file and data syncing infrastructure into your own application.

You will also learn how to work upstream with Debian (both pulling changes in and sending them back), how to properly and easily package your application for distribution, and of course how to work on contributing changes back to Ubuntu itself.

Read more
Michael Hall

It’s late, I’m tired, so this is going to be brief.  But if I didn’t put something up now, chances are I’d procrastinate to the point where it didn’t matter anymore, so something is better than nothing.

JuJu

So the buzz all week was about Juju and Charms.  It’s a very cool technology that I think is really going to highlight the potential of cloud computing.  Until now I always had people comparing the cloud to virtual machines, telling me they already automate deploying VMs, but with Juju you don’t think about machines anymore, virtual of otherwise.  It’s all about services, which is really what you want, a service that is doing something for you.  You don’t need to care where, or on what, or in combination with some other thing, Juju handles all that automatically.  It’s really neat, and I’m looking forward to using it more.

Summit

Summit worked this week.  In fact, this is the first time in my memory where there wasn’t a problem with the code during UDS.  And that’s not because we left it alone either.  IS actually moved the entire site to a new server the day before UDS started.  We landed several fixes during the week to fix minor inconveniences experienced by IS or the admins.  And that’s not even taking into consideration all the last-minute features that were added by our Linaro developers the week prior.  But through it all, Summit kept working.  That, more than anything else, is testament to the work the Summit developers put in over the last cycle to improve the code quality and development processes, and I am very, very proud that.  But we’re not taking a break this cycle.  In fact, we had two separate sessions this week about ways to improve the user experience, and will be joined by some professional designers to help us towards that goal.

Ubuntu One eBook syncing

So what started off as an casual question to Stuart Langridge turned into a full blown session about how to sync ebook data using Ubuntu One.  We brainstormed several options of what we can sync, including reading position, bookmarks, highlights and notes, as well as ways to sync them in an application agnostic manner.  I missed the session on the upcoming Ubuntu One Database (U1DB), but we settled on that being the ideal way of handling this project, and that this project was an ideal test case for the U1DB.  For reasons I still can’t explain, I volunteered to develop this functionality, at some point during the next cycle.  It’s certainly going to be a learning experience.

Friends

Friends!  It sure was good to catch up with all of you.  Both friends from far-away lands, and those closer to home.  Even though we chat on IRC almost constantly, there’s still nothing quite like being face to face.  I greatly enjoyed working in the same room with the Canonical ISD team, which has some of the smartest people I’ve ever had the pleasure of working with.  It was also wonderful to catch up with all my friends from the community.  I don’t know of any other product or project that brings people together the way Ubuntu does, and I’m amazed and overjoyed that I get to be a part of it.

Read more
Michael Hall

If you’ve been doing anything with Ubuntu lately, chances are you’ve been hearing a lot of buzz about Juju.  If you’re attending UDS, then there’s also a good chance that you’ve been to one or more sessions about Juju.  But do you know it?

The building blocks for Juju are it’s “charms”, which detail exactly how to deploy and configure services in the Cloud.  Writing charms is how you harness the awesome power of Juju.  Tomorrow (Friday) there will be a 2 hour session all about writing charms, everything from what they do and how they work, to helping you get started writing your own.  Questions will be answers, minds will be inspired, things will be made, so don’t miss out.

http://summit.ubuntu.com/uds-p/meeting/19875/juju-charm-school/

(Photo courtesy of http://www.flickr.com/photos/slightlynorth/3977607387/)

Read more
Michael Hall

Are you both an Ubuntu user and a bibliophile?  Want to keep your ebooks synced between all your connected devices, including bookmarks and reading position?   If so, join us for this UDS session Thursday, Nov 3rd, where we’ll be talking about how to add that functionality to Ubuntu One.

http://summit.ubuntu.com/uds-p/meeting/19820/other-p-u1-book-sync/

Read more
Michael Hall

In a previous post a commentator was explaining his typical web-stack deployment, and boasting about how he “can roll this out on Debian in less than 4 hours”.  Now he’s talking about provisioning, installing the OS, installing the services, and configuring everything.  That’s easily a day’s work for a capable sysadmin in a corporate environment. At least, it is in my experience.  So 4 hours sounds pretty good, doesn’t it?  I tell you what, go find your nearest admin and ask them how long it would take for them to get a new WordPress site up and ready for you to start posting to.  No really, I mean it, go ask.  I’ll wait.

Back?  Good.  I’m betting that, for many of you, the answer was a day or less, depending on their backlog of work.  A lucky few would be able to get their new site in an hour or two.  Not bad. But if you were using Ubuntu and cloud technology, you would have had your new site ready in the time that it took you to get that answer.

You think I’m exaggerating?  You can get a new world-accessible WordPress site up and running in less than 10 minutes.  That’s not marketing hype, Canonical is quite literally putting it’s money where it’s mouth is, by paying for an hour of Ubuntu Server on Amazon’s cloud service.  Go to http://try.cloud.ubuntu.com and follow along, and you’ll have a WordPress site up and running in less time than it’ll take you to finish reading this article.

Welcome to the Try Ubuntu project.  Yes, we really are footing the bill for this, it won’t cost you a dime.  Click that large inviting “Let’s go to the cloud” button to start your adventure in cloud computing.

You will need to sign into Ubuntu SSO, because while we’re perfectly happy to pay for one hour, we do need to prevent abuse.  Requiring a login lets us limit this to once per user.  If you don’t have an SSO account you can create one now, and really if you’re going to spend any time at all with Ubuntu or the community, you’re going to want one soon or later anyway.

Ubuntu SSO lets you decide what detailed information to send back to any requesting web service.  You don’t have to send any of these details back to use this service, but if you have a Launchpad profile and uploaded SSH keys, you’ll get a better experience if you send at least your username.

Next you’ll get to choose what you want running on your trial instance.  You’ll also have to agree not to be abusive with your instance, do anything illegal, or generally cause other people problems as a result of our generosity.  Seriously, just don’t do it.

Currently we offer a base Ubuntu Server, running just the default installation, as well as the base server plus WordPress, Drupal or MoinMoin.  These aren’t pre-made images, they’ll be installed after the new server is provisioned, just like you would do manually, only thanks to cloud-init, we have it all automated.  You will use the same Ubuntu Server AMI regardless of which service you choose.

That’s all you need to do!  Now click the “Launch” button and the website will ask for a new m1.small instance through Amazon’s EC2 API.  This API is available to anyone, by the way, so you can script your own cloud deployments in exactly the same way.  We are using the boto python library to access the API from within Django.

After we get an instance reserved, we have to wait for Amazon to start it.  It only takes about a minute for Amazon to start up your new instance. In the mean time, this page will periodically refresh itself, and will let you know once your instance has started.  Did you notice that “View Cloud-Config” link?

This is what will be run on your new instance as soon as it’s ready.  You can copy this script, and use it later to start up your own permanent Ubuntu Server instances on AWS or any other EC2 compatible cloud host.  It is this script that will install Apache, MySQL and WordPress, configure them all properly, and get them all running.

Now your instance has started, and so has your countdown clock. At this point Ubuntu is fully booted, and your cloud-init script is busy installing all the packages and dependencies needed to run WordPress, all you have to do is sit back and wait.  Deploying is hard work huh?.  Don’t worry, I won’t tell your boss.

Hope you didn’t get too comfy, because 3 minutes later and we’re done!  Here you’ve given the command to SSH into your new instance.  If you’re running this from Windows, you’ll need to get something like PuTTY, because last time I checked Redmond still thinks that unencrypted Telnet is a good idea.  Trust me, you want SSH.

So fire up a terminal (yes, a terminal, you’re in server-land now buddy) and copy/paste the ssh command to get connected to your new instance.

Next you will be prompted for your password, it’s back on the web page, just copy/paste it into the terminal.  Remember how I said earlier that having a Launchpad profile with uploaded SSH keys would give a better experience?  Well if you’ve got all that, then you won’t see this step.  You see, part of the initialization that happened when your new instance started was to download your public SSH keys from Launchpad, allowing you to use your private SSH keys for authentication.  Nice huh?

So now you’re connected to your new instance.  But what’s all that stuff at the bottom of your terminal now?  That my friend, is Byobu, Ubuntu’s highly customized profile for GNU Screen.   Describing all of it’s wonderful goodness would take another full blog post, so I’ll just point out the highlights.  F2 creates a new “tab”, and you can switch between them with F3/F4.  Down at the bottom are some system monitoring widgets for things like load and memory usage.  In green characters is a special widget for Amazon EC2 that gives you an estimate of how much your instance is costing you (a whopping $0.09 USD at this point), and in blue characters is a clock showing how long your trial has been running (so you’ll know how much of your 55 minutes you have used).

Alright, your instance is running and configured, now what?  Well, did you  see that link on the web page?

That one, next to “Try going to”.  Click on it.

Well look at that, it’s your new WordPress site, just waiting for you to give it a name (and also username and password).  No unzipping, copying, apache configs, database setup, nothing.  We literally can’t make it any easier.

So pick a name, give it a password and email, and you’re all set!  Yes, I know my password was weak.  Actually my password was ‘password’.  Hey, it was only up for 55 minutes, I’m not going to spend extra time thinking of a secure password.  Come on, we’ve got a WordPress site to play with!

 

There you are, your new WordPress site is deployed.  And how are we doing on time?  Well if you hadn’t spent so much time reading along with this article, you’d have at least 45 minutes left out of the original 55.  Heck, I wasted a bunch of time taking screenshots along the way, and I still had more than 40 remaining.  How far do you think your sysadmin would have gotten in this same amount of time?  He probably just got back form refilling his coffee (which, to his credit, really is necessary before attempting a deployment the old fashioned way).

After a while, as you get close to the end of your trial period, you’ll get these helpful messages in your terminal session, letting you know how much longer you have.  And before you start thinking that you can use your mighty sudo powers to stop your instance’s termination, sorry pal, but we keep track of them on our end too, and your instance will be killed through the same EC2 API that launched it.  But I sure hope you had fun.

 

So now you’ve seen how fast and easy it is to deploy not just Ubuntu Server in the cloud, but actual, useful services running on top of it.  We offer you three popular software packages for websites, but those are only the tip of the iceberg.  You can write cloud-init scripts for anything you want to deploy on Ubuntu, even your own in-house build applications.  Then you too can deploy into the cloud with the click of a button.

What’s that you say?  You don’t have a handy dandy webapp for one-click deployments into the cloud?  Oh but you do!  You see, everything you just saw is open source, you can download it from lp:awstrial on Launchpad.  Use it to run your own trials, or just to learn how we did it so you can write your own internal provisioning service.  It defaults to using Amazon’s EC2 cloud, but you can point it to any EC2 compatible cloud.  We ran it internally against an OpenStack EC2 cloud during development and testing.

Did you enjoy your trial?  Leave us some feedback on what you liked, what you didn’t, and what you want us to offer in the future.

Read more