Canonical Voices

Posts tagged with 'utlemming'

Ben Howard

Ubuntu Server 11.04 has proven to be a venerable server platform, it has nonetheless, reached its end of life on October 28, 2012. Whether or not you are blissfully unaware, end-of-life means that updates and security patches have been discontinued.

As part of the EOL'ing of a release, the mirrors are retired over time. Last week the mirrors for 11.04 were retired from archive.ubuntu.com, which in turn propagated through to the S3 EC2 mirrors. Any person using Ubuntu 11.04 and the S3 mirrors or archive.ubuntu.com will be unable to install software.

Over the last week, the Cloud Image team has fielded several questions from distraught users caused by the continued use of 11.04. We strongly suggest that those running Ubuntu Server 11.04 and the recently expired 11.10 and 8.04 LTS upgrade to a supported release to prevent any disruptions to their infrastructure. The current supported LTS is 12.04 with 13.04 being the latest stable release. Ubuntu 10.04 LTS is supported until April of 2015.

Those who continue to run expired Ubuntu releases may experience issues and may be required to mitigate the movement of mirrors from the S3 and main archive servers to old-releases.ubuntu.com/ubuntu [2]. While Ubuntu 8.04 LTS and 11.10 are currently available at the main archive and S3 locations, they will be removed anytime, per policy [3]

The Symptom

For those who are still running 11.04 and are running the 11.04 Cloud Image, you mostly likely have encountered or will encounter some ugly error messages when you try to access the S3 archives for EC2 images in the form of 404 and 403 errors. For example:

Err http://security.ubuntu.com natty-security/main Sources
  404  Not Found [IP: 91.189.92.201 80]
Err http://security.ubuntu.com natty-security/universe Sources
  404  Not Found [IP: 91.189.92.201 80]
Err http://security.ubuntu.com natty-security/main amd64 Packages
  404  Not Found [IP: 91.189.92.201 80]
Err http://security.ubuntu.com natty-security/universe amd64 Packages
  404  Not Found [IP: 91.189.92.201 80]
Ign http://security.ubuntu.com natty-security/main Translation-en_US

And....

Err http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ natty-updates/main libapr1 amd64 1.4.2-7ubuntu2.1
  403  Forbidden
Err http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ natty/main libaprutil1 amd64 1.3.9+dfsg-5ubuntu3
  403  Forbidden

The reason for this is that when the archives were expired the S3 mirrors themselves replicated the expiration. This is a friendly way to let you know that you should upgrade [3]  to the Ubuntu Server 12.04 LTS.



If, however, you are unable to upgrade, there are two options. 

Fix Option 1:

sudo sed -i 's,http://.*ubuntu.com,http://old-releases.ubuntu.com,g' /etc/apt/sources.list
sudo apt-get -y update

If you intend on rebundling the image, you will need to run the following commands to ensure that bundled images retain the settings:

dpkg-divert --local --rename --add /etc/cloud/templates/sources.list.tmpl
sed -i  's,http://.*ubuntu.com,http://old-releases.ubuntu.com,g' \
     /etc/cloud/templates/sources.list.tmpl.distrib \
    | sudo tee /etc/cloud/templates/sources.list.tmpl

Fix Option 2: Starting new instances

If you are starting a new instance, you can configure your cloud-config to default to the new mirrors. The relevent option is the 

#cloud-config
bootcmd: 
- sed -i 's,\$mirror,http://old-releases.ubuntu.com/ubuntu,g' -e 's,http://security.ubuntu.com.,http://old-releases.ubuntu.com,g' /etc/cloud/templates/sources.list.tmpl.distrib > /etc/cloud/templates/sources.list.tmpl

References

[1] http://fridge.ubuntu.com/2012/10/28/ubuntu-11-04-natty-narwhal-end-of-life-reached-on-october-28-2012/ 
[2] http://old-releases.ubuntu.com/ubuntu/dists/
[3] https://help.ubuntu.com/community/PreciseUpgrades


Read more
Ben Howard

Over the course of working with Microsoft on Windows Azure, we have had the goal of bringing the same experience on Azure as our users have on EC2. As part of our QA process, we publish daily images (http://cloud-images.ubuntu.com) for EC2 and OpenStack users.

Today, I am pleased to announce that Windows Azure Cloud Image dailies are now being published for Ubuntu Server 12.04 LTS, 12.10, 13.04 and the current development version 13.10. Due to the way that Windows Azure image publication works, these images will appear with in a three or four hours of the EC2 images and will be published to all Windows Azure regions.

However, the daily images will not be available in the Windows Azure Gallery; these images are published to API users. In the coming weeks, we'll throw up some pages to help our API users find the current images, but for now, you can use the API Query tools to find the images.

The initial daily images are:

  • Ubuntu_DAILY_BUILD-precise-12_04_2-LTS-amd64-server-20130502-en-us-30GB
  • Ubuntu_DAILY_BUILD-raring-13_04-amd64-server-20130501-en-us-30GB
  • Ubuntu_DAILY_BUILD-saucy-13_10-amd64-server-20130502-en-us-30GB  
As you can see, daily builds are clearly marked as "DAILY_BUILD" and include both the code name and the version number. Canonical provided images are all prefixed with our publisher GUUID of "b39f27a8b8c64d52b05eac6a62ebad85__"

While we make every effort at maintaining quality, daily images are not officially supported and may have issues as they are not rigorously QA'ed. As part of our release process, we take a daily, put it through QA and then promote the image.  If you see any problems with any of the daily builds, please head over to Launchpad.net and file us a bug.

For those who need a primer on using the Azure CLI Tool, our friends over at Microsoft have a really good explanation here. After you get it all setup, you should see all versions of the released Ubuntu Cloud Images and the dailies.

Finally, our daily image publishing will be restricted to the last five images for any one series. Like on EC2, all versions of Ubuntu Server released Cloud Images will remain indefinitely, with the exception of the pre-Windows Azure GA images (i.e. images with a serial of less than 20130414).

Read more
Ben Howard

We are pleased to announce that Canonical has stood up official mirrors in HP Cloud's AZ-1, 2, and 3 regions.

If you are using Ubuntu Server 12.10 Cloud Images, there is no action to take; 12.10 images are by default configured to use the new mirror address.

For Ubuntu 12.04 instances, the default Ubuntu image does not automatically use the in-HP Cloud mirrors. We are currently working with HP to publish a new image that defaults to the local mirrors. If you would like to switch to the new in-HP mirrors, simply run:
          
    $ sudo sed -i -e \
            's,^archive.ubuntu.com/ubuntu,nova.clouds.archive.ubuntu.com/ubuntu,g'  \
             /etc/apt/sources.list 

    $ sudo apt-get -y update

Note: *.clouds.archive.ubuntu.com is configured using split-horizon DNS. This means that the DNS answer to queries is based on the askering IP address; only queries originating within HP Cloud are answered with the HP Cloud mirror addresses. If your DNS resolver[s] is not based in HP Cloud, then you will be unable to benefit from these new mirrors. 
 

Read more
Ben Howard

Shortly after introducing the Vagrant images, a number of users provided very valuable feedback. The general gist was "sure this is a nice, but useless." For Cloud Images, we definitely take our user feedback to heart. 


The 12.10 and 13.04 images now include Chef, Puppet and Juju clients. Also, the 13.04 images work now that the annoying Virtualbox installation error has been fixed. Users report that Chef and Chef Solo provisioning work with out any problems.

For 12.04, providing Chef support is somewhat more difficult as there are no official Ubuntu provided versions of Chef. Policy restricts us from providing third-party software on any image hosted on ubuntu.com. However, 12.04 does include Puppet and Juju.

The inclusion of Juju was added at the request of Juju charmers. In a future blog post, I'll illustrate how to use Vagrant for Juju charm development. 

Finally, a common query that I get is about this particular error message:

[default] No guest additions were detected on the base box for this VM! Guest
additions are required for forwarded ports, shared folders, host only
networking, and more. If SSH fails on this machine, please install
the guest additions and repackage the box to continue.

This is not an error message; everything may continue to work properly,
in which case you may ignore this message.

I came up a nice long explanation as to the root cause (tl;dr: the Vagrant Cloud Images are _never_ booted and therefore the agent doesn't report to VirtualBox its information). And then the engineer in me started to think that this might be a trivial fix. Anyhow, in the next few days, this ugly error message will disappear for our daily builds (for Raring the message is gone as of today). 

In conclusion, I wanted to say thank you to all the people who have dropped me an email for feature requests, rants and feedback. As always please feel to drop me a line, and I'll take a look at making these better.  




Read more
Ben Howard

Earlier we announced[1] that Canonical had worked this cycle to enable more frequent releases to the Ubuntu Cloud Images stable and long term releases. As of today, we are pleased to announce that Ubuntu Server 10.04 LTS, 11.10, 12.04 LTS and 12.10 are now fully enabled to follow the kernel SRU schedule with automated update releases. This means that within 24 hours of most SRU kernel releases, a new Ubuntu Cloud Image will be published.

Please note: with this change, the release notes have been moved the http://cloud-images.ubuntu.com/releases website. You can find them under <SUITE>/release/unpacked/release-notes.txt. Effective today, all emails announcing these new updates are discontinued. 

However, at this time, 12.04 LTS and 12.10 Cloud Images are not yet being promoted automatically to Windows Azure. We expect that as Windows Azure moves closer to General Availability (i.e. moves out of preview status) that automatic promotion will be enabled.

Please use either Cloud-Images[2], the AMI Finder[3], the RSS feed[4], or "ubuntu-cloudimg-query" from the Cloud-Utils packages to find the latest released images.

[1] http://blog.utlemming.org/2013/01/ubuntu-cloud-images-automated-release.html
     https://lists.ubuntu.com/archives/ubuntu-cloud-announce/2013-January/000045.html
     https://lists.ubuntu.com/archives/ubuntu-cloud/2013-January/000879.html
     https://groups.google.com/forum/?fromgroups=#!topic/ec2ubuntu/Mg-qpfguE10
[2] http://cloud-images.ubuntu.com/releases
[3] http://cloud-images.ubuntu.com/locator/ec2/
[4] http://cloud-images.ubuntu.com/rss/

Read more
Ben Howard



Traditionally, updates for the stable release and long term stable release Cloud Images have been on an ad-hoc basis; reasons for releasing new images were generally restricted to security, critical bugs, and stale images. This ad-hoc update cycle meant that updated images were only released every three months or so, and for older releases, as often as six months.

While quality has always been a concern and top priority, during this cycle, Canonical has worked to vastly improve the QA infrastructure to support our Cloud Images. For example, when a new kernel is released, the daily build for that image is now put through the complete QA process. This change in process has allowed us to identify and automatically evaluate whether or not an image is a good candidate for update release.


As such, we are pleased to announce in the next few weeks, we will be turning on automated updates for Ubuntu Server 10.04 LTS, 11.10, 12.04 LTS, and 12.10. This means that approximately every three to four weeks, a new, freshened image will be released. The release cadence will follow the kernel SRU process.

The first updated image to be released under this process was 10.04 LTS[1].

There are a variety of ways to find the released Cloud Images. The two easiest ways are to go the AMI Finder[2] or use http://cloud-images.ubuntu.com/releases/<SUITE>/release. For example, http://cloud-images.ubuntu.com/releases/lucid/release would bring you to the last AMI's for Ubuntu Server 10.04 LTS.

Due to this change, we will discontinuing the email notifications of updated images to the various email lists for updated images. At UDS-R in Copenhagen[3], we discussed email notifications and the decision was reached to discontinue them. Replacing email notification is the RSS feed[4] and release notes (example from 10.04 LTS)[5].

As Cloud Image suites are migrated to automated releases, we will follow up on this announcement.

Finally, for 12.04 LTS and later, this change will introduce lock-step update releases with Windows Azure. As Windows Azure moves towards GA, we have been working to have the same releases for the Ubuntu Server Cloud Images on both EC2 and Windows Azure.

As always, your feedback is most appreciated. Please feel free to follow on either this post or to email concerns direct to me.

[1] http://cloud-images.ubuntu.com/releases/lucid/release-20130124/
[2] http://cloud-images.ubuntu.com/locator/ec2/
[3] http://blueprints.launchpad.net/ubuntu/+spec/servercloud-r-cloudtesting
[4] http://cloud-images.ubuntu.com/rss/
[5] http://cloud-images.ubuntu.com/releases/lucid/release-20130124/unpacked/release_notes.txt

Read more
Ben Howard


We are pleased to announce the availability of beta Vagrant Cloud Images. These images have been customized to work with the Vagrant development environment, and are based on the Ubuntu Cloud Images. As such, these are vanilla images. They do, however, have the Virtualboxguest additions found in the Universe archive (required for Vagrant integration). 

For those who use Vagrant, your feedback is essential. Please feel free to send feedback via the ubuntu-cloud@lists.ubuntu.com mailing list.

The images are approximately 256M in size, and are configured for 512MB of RAM. They use a custom cloud-init user-data to drive the first boot. And of course, they have the vagrant user with vagrant insecure SSH key pre-installed. During the beta period, we will not be promoting any of the Vagrant boxes with the regular releases of the Ubuntu Cloud Images and will only publish the daily image builds; after the beta period these images will be promoted with the releases.

To kick the tires on the Vagrant images, take a look at http://cloud-images.ubuntu.com/vagrant. I will be working with the fine folks at Vagrant to get the official Ubuntu Vagrant images listed at  http://www.vagrantbox.es/

If you are interested in learning about the Vagrant development environment, head on over to Vagrant for more information.

Read more
Ben Howard


For sometime people have been asking me "when will Cloud Images sport a Twitter account?" Well, wait no longer, because the Ubuntu Cloud Image Builder now has a Twitter Account.

The Cloud Image process will now Tweet when a new image is build and published -- dailies, new release updates and new versions being releases. For right we're only Tweeting EC2 information, but once Windows Azure goes GA, we'll start Tweeting that too.

So in the meantime, you can follow our faithful Cloud Builder as it tweets merrily its build progress at @UbuCloudImages. But I'll have to warn you, the Cloud Builder won't response to tweets, so we're not snubbing you if there is no response.


Read more
Ben Howard

If Twitter isn't your cup of tea, or coffee, or <insert liquid refreshment here>, how about RSS. A while back in November, we introduced RSS feeds for the Ubuntu Cloud Images.

There is a feed for both released images and dailies. The feeds are really simple: they show all the builds that are available. This is a great way to track new releases of the Ubuntu Cloud Images if you don't want want to follow Twitter, hate checking your email or you don't care much for reading our announcement emails.

Anyway, enjoy the RSS.

Read more
Ben Howard

We are pleased to announce the immediate availability of the latest 12.04.1 (Precise Pangolin) and 12.10 (Quantal Quetzal) Cloud Images on Windows Azure, Microsoft's public cloud. These images are named:
  • 12.04.1: Ubuntu-12_04_1-LTS-amd64-20121218-en-us-30GB
  • 12.10: Ubuntu-12_10-amd64-20121218-1-en-us-30GB
Please note, that due to the way that Windows Azure Gallery works, we will not be able to maintain older images on the gallery. As new images become available, we will be expiring old images. If you need a specific version/serial of a Ubuntu Cloud Image, we strongly suggest that you snapshot and use that version.
Action required: Updates for Existing Images

ACTION REQUIRED: Update existing 12.04.1 or 12.10 images

Over the last year, it has been our pleasure to work with Microsoft to build the Ubuntu Cloud Images for Windows Azure. Microsoft is making updates to Windows Azure to further increase performance and stability of Virtual Machines, currently in preview. These updates require an action - by January 15, 2013 - on custom Linux virtual machine images that fit the definition below:

  • Any gallery Linux images captured into storage accounts from a virtual machine originally created on or before December 21, 2012
  • Any of your personal Linux images uploaded to Windows Azure on or before December 21, 2012.
To learn more about the changes that Microsoft has implemented, please see Microsoft’s announcement.
All Ubuntu users who have instances running on Windows Azure need to make sure that they take action BEFORE January 15, 2013 (else, they might find the Mayan prophecy 30 days late, as far as their instances are concerned).
If, however, you want to update existing images, then we present to you, utlemming’s tl;dr update guide (or you can use Microsoft’s guide, which does the same stuff, just with more explicit commands).

utlemming’s tl;dr update guide

From a Ubuntu perspective the changes required for 12.04 and 12.10 are:
  • CHANGE the current archive mirrors to the on-site Azure mirrors. Canonical has worked to build fast and stable regional mirrors which are co-located in each Azure region. To reduce your bandwidth costs and improve the experience of installing software, all users should update to the new mirrors.
  • CHANGE bootloader configuration to include kernel parameters of “rootdelay=300 console=ttyS0? and drop the kernel parameter "ata_piix.disable_driver" as it is no longer beneficial and will cause harm after January 15, 2013
  • ADD hv-kvp-daemon-init, which facilitates the start of the hv-kvp-daemon and supporting scripts. The hv-kvp-daemon handles hyper-visor-to-Ubuntu communication channels.
  • ADD linux-backports-hv-{precise,quantal}-virtual (lbm module). The lbm module backports the 3.7 HV stack to support new hypervisor features, as well as increase performance and stability.
  • UPDATE walinuxagent to version 1.2. Microsoft has introduced some bug fixes to the way that provisioning of Linux images work. This updated agent will reduce provisioning failures.

   
Update Ubuntu 12.04 and 12.04.1

For those running 12.04 and 12.04.1, the following steps are needed to fully update Ubuntu to Windows Azure compatibility. Complete all eight steps to update the mirror, the kernel, and the Azure agent.
  1. sudo sed -i “s,archive.ubuntu.com,azure.archive.ubuntu.com,g” /etc/apt/sources.list
    • This step updates the mirrors to point to an Azure hosted mirror.
  2. sudo apt-add-repository ‘http://archive.canonical.com/ubuntu precise-backports main’
    • This step adds the repository needed to get the kernel and agent changes.
  3. sudo apt-get update
  4. sudo apt-get install linux-backports-modules-hv-precise-virtual
    • This step adds the update kernel and associated modules.
  5. sudo apt-get install hv-kvp-daemon-init walinuxagent
    • This step adds the updated agent.
  6. Perform the GRUB_CMDLINE_LINUX_DEFAULT steps outlined below to adjust the boot commandline options prior to your next boot.
  7. (recommended) sudo apt-get dist-upgrade
  8. sudo reboot

Update Ubuntu 12.10

For those who have already upgraded their images and area already running 12.10,  the following steps are needed to fully update Ubuntu to Windows Azure compatibility. Complete all eight steps to update the mirror, the kernel, and the Azure agent.
  1. sudo sed -i “s,archive.ubuntu.com,azure.archive.ubuntu.com,g” /etc/apt/sources.list
    • This step updates the mirrors to point to an Azure hosted mirror.
  2. sudo apt-add-repository ‘http://archive.canonical.com/ubuntu precise-backports main’
    • This step adds the repository needed to get the kernel and agent changes.
  3. sudo apt-get update
  4. sudo apt-get install linux-backports-modules-hv-precise-virtual
    • This step adds the update kernel and associated modules.
  5. sudo apt-get install hv-kvp-daemon-init walinuxagent
    • This step adds the updated agent.
  6. Perform the GRUB_CMDLINE_LINUX_DEFAULT steps outlined below to adjust the boot commandline options prior to your next boot.
  7. (recommended) sudo apt-get dist-upgrade
  8. sudo reboot

Update the Boot Loader Configuration

Ubuntu instances running on Windows Azure need to be configured for a long root delay (how long Ubuntu will wait for the root device to appear) and to output kernel messages to the serial console.

Edit /etc/default/grub and make sure that the GRUB_CMDLINE_LINUX_DEFAULT has “console=ttyS0 rootdelay=300” in it.  Remove any reference to "ata_piix.disable_driver". For example: 

GRUB_CMDLINE_LINUX_DEFAULT="console=ttyS0 rootdelay=300”Ubuntu instances running on Windows Azure should no longer be configured to disable the ata_piix driver as it is now used for simulating a CD-ROM.


An alternative to using an editor on /etc/default/grub is to just run these commands:
  1. sudo sed -i 's/GRUB_CMDLINE_LINUX_DEFAULT="/GRUB_CMDLINE_LINUX_DEFAULT="console=ttyS0 rootdelay=300 /g' /etc/default/grub
  2. sudo sed -i 's/atapiix.disable_driver//g' /etc/default/grub
Then run the following to process these grub linux command line changes:
  1. sudo update-grub
  2. (optional) sudo reboot

Bring-your-own-Ubuntu (BYOU)

Obviously, we would love for you to use the images that Canonical builds. After all, we have a team that has put in countless hours building, perfecting and QA’ing. But, if for some reason you can’t, then can we suggest that you start with our base images? You can find them here:
These VHD files are the exact base-bits that have been uploaded and registered in the Windows Azure environment. Even better, these are based on the 20121218 Amazon AWS EC2 official images with the same package version and set (except there are a few more packages to support Windows Azure).
TIP: If you use the VHD files for BYOU, cloud-init is installed. Cloud-init is the magic sauce in the Ubuntu Cloud Images that gives each instance of Ubuntu running in the cloud a personality; for Windows Azure, Cloud-Init and WALinuxAgent work side-by-side to offer the best Ubuntu experience. We have configured cloud-init for NoDataSource, which means that it will look for user-data in /var/lib/cloud/seed/nocloud-net/user-data. Simply drop your user-data script in there, and at boot time, it will be run. Also, you can edit /etc/cloud/cloud.cfg as well to use different mirrors, add SSH keys, etc. You can read more about cloud-init here: https://help.ubuntu.com/community/CloudInit
For the adventurous that like to spin their own bits, please make sure that you have the following packages installed.
  • hv-kvp-daemon-init: handles hypervisor communication with Ubuntu
  • walinuxagent: Windows Azure Linux provisioning agent and Azure fabric registration agent
  • A kernel based on 3.7 or backports of the HV stack Ubuntu 13.04 kernels support the HV stack normally.
  • Install the linux-tools-common package.
  • 12.04 and 12.10 Ubuntu kernels will need the linux-backports-hv-{precise,quantal}-virtual package installed. This package is a complete backport of the _entire_ HV stack from Ubuntu 13.04.
  • Set /etc/apt/sources.list to use the hosted-in-Azure Ubuntu mirrors (http://azure.archive.ubuntu.com), which will use the speedy mirrors local to your Azure virtual machine. WARNING: these mirrors are dreadfully slow outside Azure.
Follow the Windows Azure recommendations for publication.
NOTE: It is very hard to generate a VHD file that is compatible with Windows Azure using open-source tools without playing a very annoying and disk-space intensive dance. For this reason we strongly recommend using the VHD files above, or using the in-Azure images and taking a snapshot.

Special thanks

The Ubuntu Cloud Image team recognizes that the success on Windows Azure would not be possible if not for the amazing help and special talents of Adam Conrad (infinity), Andy Whitcroft (apw) and the QA and Certification staff, as well as our fine colleagues at Microsoft.

Read more