Canonical Voices

Posts tagged with 'ubuntu-desktop'

Dustin Kirkland

Introducting the Canonical Livepatch Service

Ubuntu 16.04 LTS’s 4.4 Linux kernel includes an important new security capability in Ubuntu -- the ability to modify the running Linux kernel code, without rebooting, through a mechanism called kernel livepatch.

Today, Canonical has publicly launched the Canonical Livepatch Service -- an authenticated, encrypted, signed stream of Linux livepatches that apply to the 64-bit Intel/AMD architecture of the Ubuntu 16.04 LTS (Xenial) Linux 4.4 kernel, addressing the highest and most critical security vulnerabilities, without requiring a reboot in order to take effect.  This is particularly amazing for Container hosts -- Docker, LXD, etc. -- as all of the containers share the same kernel, and thus all instances benefit.

I’ve tried to answer below some questions that you might have. As you have others, you’re welcome
to add them to the comments below or on Twitter with hastag #Livepatch.

Retrieve your token from

Q: How do I enable the Canonical Livepatch Service?

A: Three easy steps, on a fully up-to-date 64-bit Ubuntu 16.04 LTS system.
  1. Go to and retrieve your livepatch token
    1. Install the canonical-livepatch snap
      $ sudo snap install canonical-livepatch 
    2. Enable the service with your token
      $ sudo canonical-livepatch enable [TOKEN] 
    And you’re done! You can check the status at any time using:

    $ canonical-livepatch status --verbose

      Q: What are the system requirements?

      A: The Canonical Livepatch Service is available for the generic and low latency flavors of the 64-bit Intel/AMD (aka, x86_64, amd64) builds of the Ubuntu 16.04 LTS (Xenial) kernel, which is a Linux 4.4 kernel. Canonical livepatches work on Ubuntu 16.04 LTS Servers and Desktops, on physical machines, virtual machines, and in the cloud. The safety, security, and stability firmly depends on unmodified Ubuntu kernels and network access to the Canonical Livepatch Service (  You also will need to apt update/upgrade to the latest version of snapd (at least 2.15).

      Q: What about other architectures?

      A: The upstream Linux livepatch functionality is currently limited to the 64-bit x86 architecture, at this time. IBM is working on support for POWER8 and s390x (LinuxOne mainframe), and there’s also active upstream development on ARM64, so we do plan to support these eventually. The livepatch plumbing for 32-bit ARM and 32-bit x86 are not under upstream development at this time.

      Q: What about other flavors?

      A: We are providing the Canonical Livepatch Service for the generic and low latency (telco) flavors of the the Linux kernel at this time.

      Q: What about other releases of Ubuntu?

      A: The Canonical Livepatch Service is provided for Ubuntu 16.04 LTS’s Linux 4.4 kernel. Older releases of Ubuntu will not work, because they’re missing the Linux kernel support. Interim releases of Ubuntu (e.g. Ubuntu 16.10) are targeted at developers and early adopters, rather than Long Term Support users or systems that require maximum uptime.  We will consider providing livepatches for the HWE kernels in 2017.

      Q: What about derivatives of Ubuntu?

      A: Canonical livepatches are fully supported on the 64-bit Ubuntu 16.04 LTS Desktop, Cloud, and Server operating systems. On other Ubuntu derivatives, your mileage may vary! These are not part of our automated continuous integration quality assurance testing framework for Canonical Livepatches. Canonical Livepatch safety, security, and stability will firmly depend on unmodified Ubuntu generic kernels and network access to the Canonical Livepatch Service.

      Q: How does Canonical test livepatches?

      A: Every livepatch is rigorously tested in Canonical's in-house CI/CD (Continuous Integration / Continuous Delivery) quality assurance system, which tests hundreds of combinations of livepatches, kernels, hardware, physical machines, and virtual machines.  Once a livepatch passes CI/CD and regression tests, it's rolled out on a canary testing basis, first to a tiny percentage of the Ubuntu Community users of the Canonical Livepatch Service. Based on the success of that microscopic rollout, a moderate rollout follows.  And assuming those also succeed, the livepatch is delivered to all free Ubuntu Community and paid Ubuntu Advantage users of the service.  Systemic failures are automatically detected and raised for inspection by Canonical engineers.  Ubuntu Community users of the Canonical Livepatch Service who want to eliminate the small chance of being randomly chosen as a canary should enroll in the Ubuntu Advantage program (starting at $12/month).

      Q: What kinds of updates will be provided by the Canonical Livepatch Service?

      A: The Canonical Livepatch Service is intended to address high and critical severity Linux kernel security vulnerabilities, as identified by Ubuntu Security Notices and the CVE database. Note that there are some limitations to the kernel livepatch technology -- some Linux kernel code paths cannot be safely patched while running. We will do our best to supply Canonical Livepatches for high and critical vulnerabilities in a timely fashion whenever possible. There may be occasions when the traditional kernel upgrade and reboot might still be necessary. We’ll communicate that clearly through the usual mechanisms -- USNs, Landscape, Desktop Notifications, Byobu, /etc/motd, etc.

      Q: What about non-security bug fixes, stability, performance, or hardware enablement updates?

      A: Canonical will continue to provide Linux kernel updates addressing bugs, stability issues, performance problems, and hardware compatibility on our usual cadence -- about every 3 weeks. These updates can be easily applied using ‘sudo apt update; sudo apt upgrade -y’, using the Desktop “Software Updates” application, or Landscape systems management. These standard (non-security) updates will still require a reboot, as they always have.

      Q: Can I rollback a Canonical Livepatch?

      A: Currently rolling-back/removing an already inserted livepatch module is disabled in Linux 4.4. This is because we need a way to determine if we are currently executing inside a patched function before safely removing it. We can, however, safely apply new livepatches on top of each other and even repatch functions over and over.

      Q: What about low and medium severity CVEs?

      A: We’re currently focusing our Canonical Livepatch development and testing resources on high and critical security vulnerabilities, as determined by the Ubuntu Security Team.  We'll livepatch other CVEs opportunistically.

      Q: Why are Canonical Livepatches provided as a subscription service?

      A: The Canonical Livepatch Service provides a secure, encrypted, authenticated connection, to ensure that only properly signed livepatch kernel modules -- and most importantly, the right modules -- are delivered directly to your system, with extremely high quality testing wrapped around it.

      Q: But I don’t want to buy UA support!

      A: You don’t have to! Canonical is providing the Canonical Livepatch Service to community users of Ubuntu, at no charge for up to 3 machines (desktop, server, virtual machines, or cloud instances). A randomly chosen subset of the free users of Canonical Livepatches will receive their Canonical Livepatches slightly earlier than the rest of the free users or UA users, as a lightweight canary testing mechanism, benefiting all Canonical Livepatch users (free and UA). Once those canary livepatches apply safely, all Canonical Livepatch users will receive their live updates.

      Q: But I don’t have an Ubuntu SSO account!

      A: An Ubuntu SSO account is free, and provides services similar to Google, Microsoft, and Apple for Android/Windows/Mac devices, respectively. You can create your Ubuntu SSO account here.

      Q: But I don’t want login to!

      A: You don’t have to! Canonical Livepatch is absolutely not required maintain the security of any Ubuntu desktop or server! You may continue to freely and anonymously ‘sudo apt update; sudo apt upgrade; sudo reboot’ as often as you like, and receive all of the same updates, and simply reboot after kernel updates, as you always have with Ubuntu.

      Q: But I don't have Internet access to!

      A: You should think of the Canonical Livepatch Service much like you think of Netflix, Pandora, or Dropbox.  It's an Internet streaming service for security hotfixes for your kernel.  You have access to the stream of bits when you can connect to the service over the Internet.  On the flip side, your machines are already thoroughly secured, since they're so heavily firewalled off from the rest of the world!

      Q: Where’s the source code?

      A: The source code of livepatch modules can be found here.  The source code of the canonical-livepatch client is part of Canonical's Landscape system management product and is commercial software.

      Q: What about Ubuntu Core?

      A: Canonical Livepatches for Ubuntu Core are on the roadmap, and may be available in late 2016, for 64-bit Intel/AMD architectures. Canonical Livepatches for ARM-based IoT devices depend on upstream support for livepatches.

      Q: How does this compare to Oracle Ksplice, RHEL Live Patching and SUSE Live Patching?

      A: While the concepts are largely the same, the technical implementations and the commercial terms are very different:

      • Oracle Ksplice uses it’s own technology which is not in upstream Linux.
      • RHEL and SUSE currently use their own homegrown kpatch/kgraft implementations, respectively.
      • Canonical Livepatching uses the upstream Linux Kernel Live Patching technology.
      • Ksplice is free, but unsupported, for Ubuntu Desktops, and only available for Oracle Linux and RHEL servers with an Oracle Linux Premier Support license ($2299/node/year).
      • It’s a little unclear how to subscribe to RHEL Kernel Live Patching, but it appears that you need to first be a RHEL customer, and then enroll in the SIG (Special Interests Group) through your TAM (Technical Account Manager), which requires Red Hat Enterprise Linux Server Premium Subscription at $1299/node/year.  (I'm happy to be corrected and update this post)
      • SUSE Live Patching is available as an add-on to SUSE Linux Enterprise Server 12 Priority Support subscription at $1,499/node/year, but does come with a free music video.
      • Canonical Livepatching is available for every Ubuntu Advantage customer, starting at our entry level UA Essential for $150/node/year, and available for free to community users of Ubuntu.

      Q: What happens if I run into problems/bugs with Canonical Livepatches?

      A: Ubuntu Advantage customers will file a support request at where it will be serviced according to their UA service level agreement (Essential, Standard, or Advanced). Ubuntu community users will file a bug report on Launchpad and we'll service it on a best effort basis.

      Q: Why does canonical-livepatch client/server have a proprietary license?

      A: The canonical-livepatch client is part of the Landscape family of tools available to Canonical support customers. We are enabling free access to the Canonical Livepatch Service for Ubuntu community users as a mark of our appreciation for the broader Ubuntu community, and in exchange for occasional, automatic canary testing.

      Q: How do I build my own livepatches?

      A: It’s certainly possible for you to build your own Linux kernel live patches, but it requires considerable skill, time, computing power to produce, and even more effort to comprehensively test. Rest assured that this is the real value of using the Canonical Livepatch Service! That said, Chris Arges has blogged a howto for the curious a while back:

      Q: How do I get notifications of which CVEs are livepatched and which are not?

      A: You can, at any time, query the status of the canonical-livepatch daemon using: ‘canonical-livepatch status --verbose’. This command will show any livepatches successfully applied, any outstanding/unapplied livepatches, and any error conditions. Moreover, you can monitor the Ubuntu Security Notices RSS feed and the ubuntu-security-announce mailing list.

      Q: Isn't livepatching just a big ole rootkit?

      A: Canonical Livepatches inject kernel modules to replace sections of binary code in the running kernel. This requires the CAP_SYS_MODULE capability. This is required to modprobe any module into the Linux kernel. If you already have that capability (root does, by default, on Ubuntu), then you already have the ability to arbitrarily modify the kernel, with or without Canonical Livepatches. If you’re an Ubuntu sysadmin and you want to disable module loading (and thereby also disable Canonical Livepatches), simply ‘echo 1 | sudo tee /proc/sys/kernel/modules_disabled’.

      Keep the uptime!

      Read more
      Dustin Kirkland

      If you haven't heard about last week's Dirty COW vulnerability, I hope all of your Linux systems are automatically patching themselves...

      Why?  Because every single Linux-based phone, router, modem, tablet, desktop, PC, server, virtual machine, and absolutely everything in between -- including all versions of Ubuntu since 2007 -- was vulnerable to this face-palming critical security vulnerability.

      Any non-root local user of a vulnerable system can easily exploit the vulnerability and become the root user in a matter of a few seconds.  Watch...

      Coincidentally, just before the vulnerability was published, we released the Canonical Livepatch Service for Ubuntu 16.04 LTS.  The thousands of users who enabled canonical-livepatch on their Ubuntu 16.04 LTS systems with those first few hours received and applied the fix to Dirty COW, automatically, in the background, and without rebooting!

      If you haven't already enabled the Canonical Livepatch Service on your Ubuntu 16.04 LTS systems, you should really consider doing so, with 3 easy steps:
      1. Go to and retrieve your livepatch token
      2. Install the canonical-livepatch snap
        $ sudo snap install canonical-livepatch 
      3. Enable the service with your token
        $ sudo canonical-livepatch enable [TOKEN]
      And you’re done! You can check the status at any time using:

      $ canonical-livepatch status --verbose

      Let's retry that same vulnerability, on the same system, but this time, having been livepatched...

      Aha!  Thwarted!

      So that's the Ubuntu 16.04 LTS kernel space...  What about userspace?  Most of the other recent, branded vulnerabilities (Heartbleed, ShellShock, CRIME, BEAST) have been critical vulnerabilities in userspace packages.

      As of Ubuntu 16.04 LTS, the unattended-upgrades package is now part of the default package set, so you should already have it installed on your Ubuntu desktops and servers.  If you don't already have it installed, you can install it with:

      $ sudo apt install unattended-upgrades

      And moreover, as of Ubuntu 16.04 LTS, the unattended-upgrades package automatically downloads and installs important security updates once per day, automatically patching critical security vulnerabilities and keeping your Ubuntu systems safe by default.  Older versions of Ubuntu (or Ubuntu systems that upgraded to 16.04) might need to enable this behavior using:

      $ sudo dpkg-reconfigure unattended-upgrades

      With that combination enabled -- (1) automatic livepatches to your kernel, plus (2) automatic application of security package updates -- Ubuntu 16.04 LTS is the most secure Linux distribution to date.  Period.


      Read more
      Dustin Kirkland

      I happen to have a full mirror of the entire Ubuntu Xenial archive here on a local SSD, and I took the opportunity to run a few numbers...
      • 6: This is our 6th Ubuntu LTS
        • 6.06, 8.04, 10.04, 12.04, 14.04, 16.04
      • 7: With Ubuntu 16.04 LTS, we're supporting 7 CPU architectures
        • armhf, arm64, i386, amd64, powerpc, ppc64el, s390x
      • 25,671: Ubuntu 16.04 LTS is comprised of 25,671 source packages
        • main, universe, restricted, multiverse
      • 150,562+: Over 150,562 (and counting!) cloud instances of Xenial have launched to date
        • and we haven't even officially released yet!
      • 216,475: A complete archive of all binary .deb packages in Ubuntu 16.04 LTS consists of 216,475 debs.
        • 24,803 arch independent
        • 27,159 armhf
        • 26,845 arm64
        • 28,730 i386
        • 28,902 amd64
        • 27,061 powerpc
        • 26,837 ppc64el
        • 26,138 s390x
      • 1,426,792,926: A total line count of all source packages in Ubuntu 16.04 LTS using cloc yields 1,426,792,926 total lines of source code
      • 250,478,341,568: A complete archive all debs, all architectures in Ubuntu 16.04 LTS requires 250GB of disk space
      Yes, that's 1.4 billion lines of source code comprising the entire Ubuntu 16.04 LTS archive.  What an amazing achievement of open source development!

      Perhaps my fellow nerds here might be interested in a breakdown of all 1.4 billion lines across 25K source packages, and throughout 176 different programming languages, as measured by Al Danial's cloc utility.  Interesting data!

      You can see the full list here.  What further insight can you glean?


      Read more
      Dustin Kirkland

      This makes me so incredibly happy!

      Here's how...

      First, start with a fully up-to-date Ubuntu 16.04 LTS desktop.

      sudo apt update
      sudo apt dist-upgrade -y

      Then, install dconf-editor.

      sudo apt install -y dconf-editor

      Launch dconf-editor and find the "launcher" key and change it to "bottom".


      For good measure, I triggered a reboot, to make sure my changes stuck.  And voila!  Beauty!


      Read more
      Dustin Kirkland


      • Put /tmp on tmpfs and you'll improve your Linux system's I/O, reduce your carbon foot print and electricity usage, stretch the battery life of your laptop, extend the longevity of your SSDs, and provide stronger security.
      • In fact, we should do that by default on Ubuntu servers and cloud images.
      • Having tested 502 physical and virtual servers in production at Canonical, 96.6% of them could immediately fit all of /tmp in half of the free memory available and 99.2% could fit all of /tmp in (free memory + free swap).

      Try /tmp on tmpfs Yourself

      $ echo "tmpfs /tmp tmpfs rw,nosuid,nodev" | sudo tee -a /etc/fstab
      $ sudo reboot


      In April 2009, I proposed putting /tmp on tmpfs (an in memory filesystem) on Ubuntu servers by default -- under certain conditions, like, well, having enough memory. The proposal was "approved", but got hung up for various reasons.  Now, again in 2016, I proposed the same improvement to Ubuntu here in a bug, and there's a lively discussion on the ubuntu-cloud and ubuntu-devel mailing lists.

      The benefits of /tmp on tmpfs are:
      • Performance: reads, writes, and seeks are insanely fast in a tmpfs; as fast as accessing RAM
      • Security: data leaks to disk are prevented (especially when swap is disabled), and since /tmp is its own mount point, we should add the nosuid and nodev options (and motivated sysadmins could add noexec, if they desire).
      • Energy efficiency: disk wake-ups are avoided
      • Reliability: fewer NAND writes to SSD disks
      In the interest of transparency, I'll summarize the downsides:
      • There's sometimes less space available in memory, than in your root filesystem where /tmp may traditionally reside
      • Writing to tmpfs could evict other information from memory to make space
      You can learn more about Linux tmpfs here.

      Not Exactly Uncharted Territory...

      Fedora proposed and implemented this in Fedora 18 a few years ago, citing that Solaris has been doing this since 1994. I just installed Fedora 23 into a VM and confirmed that /tmp is a tmpfs in the default installation, and ArchLinux does the same. Debian debated doing so, in this thread, which starts with all the reasons not to put /tmp on a tmpfs; do make sure you read the whole thread, though, and digest both the pros and cons, as both are represented throughout the thread.

      Full Data Treatment

      In the current thread on ubuntu-cloud and ubuntu-devel, I was asked for some "real data"...

      In fact, across the many debates for and against this feature in Ubuntu, Debian, Fedora, ArchLinux, and others, there is plenty of supposition, conjecture, guesswork, and presumption.  But seeing as we're talking about data, let's look at some real data!

      Here's an analysis of a (non-exhaustive) set of 502 of Canonical's production servers that run,, and hundreds of related services, including OpenStack, dozens of websites, code hosting, databases, and more. These servers sampled are slightly biased with more physical machines than virtual machines, but both are present in the survey, and a wide variety of uptime is represented, from less than a day of uptime, to 1306 days of uptime (with live patched kernels, of course).  Note that this is not an exhaustive survey of all servers at Canonical.

      I humbly invite further study and analysis of the raw, tab-separated data, which you can find at:
      The column headers are:
      • Column 1: The host names have been anonymized to sequential index numbers
      • Column 2: `du -s /tmp` disk usage of /tmp as of 2016-01-17 (ie, this is one snapshot in time)
      • Column 3-8: The output of the `free` command, memory in KB for each server
      • Column 9-11: The output of the `free` command, sway in KB for each server
      • Column 12: The number of inodes in /tmp
      I have imported it into a Google Spreadsheet to do some data treatment. You're welcome to do the same, or use the spreadsheet of your choice.

      For the numbers below, 1 MB = 1000 KB, and 1 GB = 1000 MB, per Wikipedia. (Let's argue MB and MiB elsewhere, shall we?)  The mean is the arithmetic average.  The median is the middle value in a sorted list of numbers.  The mode is the number that occurs most often.  If you're confused, this article might help.  All calculations are accurate to at least 2 significant digits.

      Statistical summary of /tmp usage:

      • Max: 101 GB
      • Min: 4.0 KB
      • Mean: 453 MB
      • Median: 16 KB
      • Mode: 4.0 KB
      Looking at all 502 servers, there are two extreme outliers in terms of /tmp usage. One server has 101 GB of data in /tmp, and the other has 42 GB. The latter is a very noisy django.log. There are 4 more severs using between 10 GB and 12 GB of /tmp. The remaining 496 severs surveyed (98.8%) are using less than 4.8 GB of /tmp. In fact, 483 of the servers surveyed (96.2%) use less than 1 GB of /tmp. 454 of the servers surveyed (90.4%) use less than 100 MB of /tmp. 414 of the servers surveyed (82.5%) use less than 10 MB of /tmp. And actually, 370 of the servers surveyed (73.7%) -- the overwhelming majority -- use less than 1MB of /tmp.

      Statistical summary of total memory available:

      • Max: 255 GB
      • Min: 1.0 GB
      • Mean: 24 GB
      • Median: 10.2 GB
      • Mode: 4.1 GB
      All of the machines surveyed (100%) have at least 1 GB of RAM.  495 of the machines surveyed (98.6%) have at least 2GB of RAM.   437 of the machines surveyed (87%) have at least 4 GB of RAM.   255 of the machines surveyed (50.8%) have at least 10GB of RAM.    157 of the machines surveyed (31.3%) have more than 24 GB of RAM.  74 of the machines surveyed (14.7%) have at least 64 GB of RAM.

      Statistical summary of total swap available:

      • Max: 201 GB
      • Min: 0.0 KB
      • Mean: 13 GB
      • Median: 6.3 GB
      • Mode: 2.96 GB
      485 of the machines surveyed (96.6%) have at least some swap enabled, while 17 of the machines surveyed (3.4%) have zero swap configured. One of these swap-less machines is using 415 MB of /tmp; that machine happens to have 32 GB of RAM. All of the rest of the swap-less machines are using between 4 KB and 52 KB (inconsequential) /tmp, and have between 2 GB and 28 GB of RAM.  5 machines (1.0%) have over 100 GB of swap space.

      Statistical summary of swap usage:

      • Max: 19 GB
      • Min: 0.0 KB
      • Mean: 657 MB
      • Median: 18 MB
      • Mode: 0.0 KB
      476 of the machines surveyed (94.8%) are using less than 4 GB of swap. 463 of the machines surveyed (92.2%) are using less than 1 GB of swap. And 366 of the machines surveyed (72.9%) are using less than 100 MB of swap.  There are 18 "swappy" machines (3.6%), using 10 GB or more swap.

      Modeling /tmp on tmpfs usage

      Next, I took the total memory (RAM) in each machine, and divided it in half which is the default allocation to /tmp on tmpfs, and subtracted the total /tmp usage on each system, to determine "if" all of that system's /tmp could actually fit into its tmpfs using free memory alone (ie, without swap or without evicting anything from memory).

      485 of the machines surveyed (96.6%) could store all of their /tmp in a tmpfs, in free memory alone -- i.e. without evicting anything from cache.

      Now, if we take each machine, and sum each system's "Free memory" and "Free swap", and check its /tmp usage, we'll see that 498 of the systems surveyed (99.2%) could store the entire contents of /tmp in tmpfs free memory + swap available. The remaining 4 are our extreme outliers identified earlier, with /tmp usages of [101 GB, 42 GB, 13 GB, 10 GB].

      Performance of tmpfs versus ext4-on-SSD

      Finally, let's look at some raw (albeit rough) read and write performance numbers, using a simple dd model.

      My /tmp is on a tmpfs:
      kirkland@x250:/tmp⟫ df -h .
      Filesystem Size Used Avail Use% Mounted on
      tmpfs 7.7G 2.6M 7.7G 1% /tmp

      Let's write 2 GB of data:
      kirkland@x250:/tmp⟫ dd if=/dev/zero of=/tmp/zero bs=2G count=1
      0+1 records in
      0+1 records out
      2147479552 bytes (2.1 GB) copied, 1.56469 s, 1.4 GB/s

      And let's write it completely synchronously:
      kirkland@x250:/tmp⟫ dd if=/dev/zero of=./zero bs=2G count=1 oflag=dsync
      0+1 records in
      0+1 records out
      2147479552 bytes (2.1 GB) copied, 2.47235 s, 869 MB/s

      Let's try the same thing to my Intel SSD:
      kirkland@x250:/local⟫ df -h .
      Filesystem Size Used Avail Use% Mounted on
      /dev/dm-0 217G 106G 100G 52% /

      And write 2 GB of data:
      kirkland@x250:/local⟫ dd if=/dev/zero of=./zero bs=2G count=1
      0+1 records in
      0+1 records out
      2147479552 bytes (2.1 GB) copied, 7.52918 s, 285 MB/s

      And let's redo it completely synchronously:
      kirkland@x250:/local⟫ dd if=/dev/zero of=./zero bs=2G count=1 oflag=dsync
      0+1 records in
      0+1 records out
      2147479552 bytes (2.1 GB) copied, 11.9599 s, 180 MB/s

      Let's go back and read the tmpfs data:
      kirkland@x250:~⟫ dd if=/tmp/zero of=/dev/null bs=2G count=1
      0+1 records in
      0+1 records out
      2147479552 bytes (2.1 GB) copied, 1.94799 s, 1.1 GB/s

      And let's read the SSD data:
      kirkland@x250:~⟫ dd if=/local/zero of=/dev/null bs=2G count=1
      0+1 records in
      0+1 records out
      2147479552 bytes (2.1 GB) copied, 2.55302 s, 841 MB/s

      Now, let's create 10,000 small files (1 KB) in tmpfs:
      kirkland@x250:/tmp/foo⟫ time for i in $(seq 1 10000); do dd if=/dev/zero of=$i bs=1K count=1 oflag=dsync ; done
      real 0m15.518s
      user 0m1.592s
      sys 0m7.596s

      And let's do the same on the SSD:
      kirkland@x250:/local/foo⟫ time for i in $(seq 1 10000); do dd if=/dev/zero of=$i bs=1K count=1 oflag=dsync ; done
      real 0m26.713s
      user 0m2.928s
      sys 0m7.540s

      For better or worse, I don't have any spinning disks, so I couldn't repeat the tests there.

      So on these rudimentary read/write tests via dd, I got 869 MB/s - 1.4 GB/s write to tmpfs and 1.1 GB/s read from tmps, and 180 MB/s - 285 MB/s write to SSD and 841 MB/s read from SSD.

      Surely there are more scientific ways of measuring I/O to tmpfs and physical storage, but I'm confident that, by any measure, you'll find tmpfs extremely fast when tested against even the fastest disks and filesystems.


      • /tmp usage
        • 98.8% of the servers surveyed use less than 4.8 GB of /tmp
        • 96.2% use less than 1.0 GB of /tmp
        • 73.7% use less than 1.0 MB of /tmp
        • The mean/median/mode are [453 MB / 16 KB / 4 KB]
      • Total memory available
        • 98.6% of the servers surveyed have at least 2.0 GB of RAM
        • 88.0% have least 4.0 GB of RAM
        • 57.4% have at least 8.0 GB of RAM
        • The mean/median/mode are [24 GB / 10 GB / 4 GB]
      • Swap available
        • 96.6% of the servers surveyed have some swap space available
        • The mean/median/mode are [13 GB / 6.3 GB / 3 GB]
      • Swap used
        • 94.8% of the servers surveyed are using less than 4 GB of swap
        • 92.2% are using less than 1 GB of swap
        • 72.9% are using less than 100 MB of swap
        • The mean/median/mode are [657 MB / 18 MB / 0 KB]
      • Modeling /tmp on tmpfs
        • 96.6% of the machines surveyed could store all of the data they currently have stored in /tmp, in free memory alone, without evicting anything from cache
        • 99.2% of the machines surveyed could store all of the data they currently have stored in /tmp in free memory + free swap
        • 4 of the 502 machines surveyed (0.8%) would need special handling, reconfiguration, or more swap


      • Can /tmp be mounted as a tmpfs always, everywhere?
        • No, we did identify a few systems (4 out of 502 surveyed, 0.8% of total) consuming inordinately large amounts of data in /tmp (101 GB, 42 GB), and with insufficient available memory and/or swap.
        • But those were very much the exception, not the rule.  In fact, 96.6% of the systems surveyed could fit all of /tmp in half of the freely available memory in the system.
      • Is this the first time anyone has suggested or tried this as a Linux/UNIX system default?
        • Not even remotely.  Solaris has used tmpfs for /tmp for 22 years, and Fedora and ArchLinux for at least the last 4 years.
      • Is tmpfs really that much faster, more efficient, more secure?
        • Damn skippy.  Try it yourself!

      Read more
      Dustin Kirkland

      With the recent introduction of Snappy Ubuntu, there are now several different ways to extend and update (apt-get vs. snappy) multiple flavors of Ubuntu (Core, Desktop, and Server).

      We've put together this matrix with a few examples of where we think Traditional Ubuntu (apt-get) and Transactional Ubuntu (snappy) might make sense in your environment.  Note that this is, of course, not a comprehensive list.

      Ubuntu Core
      Ubuntu Desktop
      Ubuntu Server
      Traditional apt-get
      Minimal Docker and LXC imagesDesktop, Laptop, Personal WorkstationsBaremetal, MAAS, OpenStack, General Purpose Cloud Images
      Transactional snappy
      Minimal IoT Devices and Micro-Services Architecture Cloud ImagesTouch, Phones, TabletsComfy, Human Developer Interaction (over SSH) in an atomically updated environment

      I've presupposed a few of the questions you might ask, while you're digesting this new landscape...

      Q: I'm looking for the smallest possible Ubuntu image that still supports apt-get...
      A: You want our Traditional Ubuntu Core. This is often useful in building Docker and LXC containers.

      Q: I'm building the next wearable IoT device/drone/robot, and perhaps deploying a fleet of atomically updated micro-services to the cloud...
      A: You want Snappy Ubuntu Core.

      Q: I want to install the best damn Linux on my laptop, desktop, or personal workstation, with industry best security practices, 30K+ freely available open source packages, freely available, with extensive support for hardware devices and proprietary add-ons...
      A: You want the same Ubuntu Desktop that we've been shipping for 10+ years, on time, every time ;-)

      Q: I want that same converged, tasteful Ubuntu experience on your personal, smart devices like my Phones and Tablets...
      A: You want Ubuntu Touch, which is a very graphical human interface focused expression of Snappy Ubuntu.

      Q: I'm deploying Linux onto bare metal servers at scale in the data center, perhaps building IaaS clouds using OpenStack or PaaS cloud using CloudFoundry? And I'm launching general purpose Linux server instances in public clouds (like AWS, Azure, or GCE) and private clouds...
      A: You want the traditional apt-get Ubuntu Server.

      Q: I'm developing and debugging applications, services, or frameworks for Snappy Ubuntu devices or cloud instances?
      A: You want Comfy Ubuntu Server, which is a command line human interface extension of Snappy Ubuntu, with a number of conveniences and amenities (ssh, byobu, manpages, editors, etc.) that won't be typically included in the minimal Snappy Ubuntu Core build. [*Note that the Comfy images will be available very soon]


      Read more
      Dustin Kirkland

      Last week, I posed a question on Google+, looking for suggestions on a minimal physical format, x86 machine.  I was looking for something like a Raspberry Pi (of which I already have one), but really it had to be x86.

      I was aware of a few options out there, but I was very fortunately introduced to one spectacular little box...the Intel NUC!

      The unboxing experience is nothing short of pure marketing genius!

      The "NUC" stands for Intel's Next Unit of Computing.  It's a compact little device, that ships barebones.  You need to add DDR3 memory (up to 16GB), an mSATA hard drive (if you want to boot locally), and an mSATA WiFi card (if you want wireless networking).

      The physical form factor of all models is identical:

      • 4.6" x 4.4" x 1.6"
      • 11.7cm x 11.2cm x 4.1cm

      There are 3 different processor options:

      And there are three different peripheral setups:

      • HDMI 1.4a (x2) + USB 2.0 (x3) + Gigabit ethernet
      • HDMI 1.4a (x1) + Thunderbolt supporting DisplayPort 1.1a (x1) + USB 2.0 (x3)
      • HDMI 1.4a (x1) + Mini DisplayPort 1.1a (x2) + USB 2.0 (x2); USB 3.0 (x1)
      I ended up buying 3 of these last week, and reworked my audio/video and baby monitoring setup in the house last week.  I bought 2 of these (i3 + Ethernet) , and 1 of these (i3 + Thunderbolt)

      Quite simply, I couldn't be happier with these little devices!

      I used one of these to replace the dedicated audio/video PC (an x201 Thinkpad) hooked up in my theater.  The x201 was a beefy machine, with plenty of CPU and video capability.  But it was pretty bulky, rather noisy, and drew too much power.

      And the other two are Baby-buntu baby monitors, as previously blogged here, replacing a real piece-of-crap Lenovo Q100 (Atom + SiS307DV and all the horror maligned with that sick chip set).

      All 3 are now running Ubuntu 13.10, spectacularly I might add!  All of the hardware cooperated perfectly.

      Here are the two views that I really wanted Amazon to show me, as I was buying the device...what the inside looks like!  You can see two mSATA ports and red/black WiFi antenna leads on the left, and two DDR3 slots on the right.

      On the left, you can now see a 24GB mSATA SSD, and beneath it (not visible) is an Intel Centrino Advanced-N 6235 WiFi adapter.  On the right, I have two 8GB DDR3 memory modules.

      Note, to get wireless working properly I did have to:

      echo "options iwlwifi 11n_disable=1" | sudo tee -a /etc/modprobe.d/iwlwifi.conf

      The BIOS is really super fancy :-)  There's a mouse and everything.  I made a few minor tweaks, to the boot order, assigned 512MB of memory to the display adapter, and configured it to power itself back on at any power loss.

      Speaking of power, it sustains about 10 watts of power, at idle, which costs me about $11/year in electricity.

      Some of you might be interested in some rough disk IO statistics...

      kirkland@living:~⟫ sudo hdparm -Tt /dev/sda
      Timing cached reads: 11306 MB in 2.00 seconds = 5657.65 MB/sec
      Timing buffered disk reads: 1478 MB in 3.00 seconds = 492.32 MB/sec

      And the lshw output...

          description: Desktop Computer
      product: (To be filled by O.E.M.)
      width: 64 bits
      capabilities: smbios-2.7 dmi-2.7 vsyscall32
      configuration: boot=normal chassis=desktop family=To be filled by O.E.M. sku=To be filled by O.E.M. uuid=[redacted]
      description: Motherboard
      product: D33217CK
      vendor: Intel Corporation
      physical id: 0
      version: G76541-300
      serial: [redacted]
      description: BIOS
      vendor: Intel Corp.
      physical id: 0
      version: GKPPT10H.86A.0025.2012.1011.1534
      date: 10/11/2012
      size: 64KiB
      capacity: 6336KiB
      capabilities: pci upgrade shadowing cdboot bootselect socketedrom edd int13floppy1200 int13floppy720 int13floppy2880 int5printscreen int14serial int17printer acpi usb biosbootspecification uefi
      width: 32 bits
      clock: 66MHz
      capabilities: storage msi pm ahci_1.0 bus_master cap_list
      configuration: driver=ahci latency=0
      resources: irq:40 ioport:f0b0(size=8) ioport:f0a0(size=4) ioport:f090(size=8) ioport:f080(size=4) ioport:f060(size=32) memory:f6906000-f69067ff
      *-serial UNCLAIMED
      description: SMBus
      product: 7 Series/C210 Series Chipset Family SMBus Controller
      vendor: Intel Corporation
      physical id: 1f.3
      bus info: pci@0000:00:1f.3
      version: 04
      width: 64 bits
      clock: 33MHz
      configuration: latency=0
      resources: memory:f6905000-f69050ff ioport:f040(size=32)
      physical id: 1
      logical name: scsi0
      capabilities: emulated
      description: ATA Disk
      product: BP4 mSATA SSD
      physical id: 0.0.0
      bus info: scsi@0:0.0.0
      logical name: /dev/sda
      version: S8FM
      serial: [redacted]
      size: 29GiB (32GB)
      capabilities: gpt-1.00 partitioned partitioned:gpt
      configuration: ansiversion=5 guid=be0ab026-45c1-4bd5-a023-1182fe75194e sectorsize=512
      description: Windows FAT volume
      vendor: mkdosfs
      physical id: 1
      bus info: scsi@0:0.0.0,1
      logical name: /dev/sda1
      logical name: /boot/efi
      version: FAT32
      serial: 2252-bc3f
      size: 486MiB
      capacity: 486MiB
      capabilities: boot fat initialized
      configuration: FATs=2 filesystem=fat mount.fstype=vfat mount.options=rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro state=mounted
      description: EXT4 volume
      vendor: Linux
      physical id: 2
      bus info: scsi@0:0.0.0,2
      logical name: /dev/sda2
      logical name: /
      version: 1.0
      serial: [redacted]
      size: 25GiB
      capabilities: journaled extended_attributes large_files huge_files dir_nlink recover extents ext4 ext2 initialized
      configuration: created=2013-11-06 13:01:57 filesystem=ext4 lastmountpoint=/ modified=2013-11-12 15:38:33 mount.fstype=ext4 mount.options=rw,relatime,errors=remount-ro,data=ordered mounted=2013-11-12 15:38:33 state=mounted
      description: Linux swap volume
      vendor: Linux
      physical id: 3
      bus info: scsi@0:0.0.0,3
      logical name: /dev/sda3
      version: 1
      serial: [redacted]
      size: 3994MiB
      capacity: 3994MiB
      capabilities: nofs swap initialized
      configuration: filesystem=swap pagesize=4095

      It also supports: virtualization technology, S3/S4/S5 sleep states, Wake-on-LAN, and PXE boot.  Sadly, it does not support IPMI :-(

      Finally, it's worth noting that I bought the model with the i3 for a specific purpose...  These three machines all have full virtualization capabilities (KVM).  Which means these little boxes, with their dual-core hyper-threaded CPUs and 16GB of RAM are about to become Nova compute nodes in my local OpenStack cluster ;-)  That will be a separate blog post ;-)


      Read more
      Dustin Kirkland


      Thanks for all the positive feedback to my last post!  I have made a couple of updates to the 5.1 channel Ubuntu login sound, namely:
      1. Remastered based on the original wav files, since my previous version was based on the lossy, compressed ogg files.
      2. Adjusted a couple of levels, having actually tested it on as many different 5.1 and 2-channel stereo environments I could find.
      3. Updated the ubuntu-sounds package and pushed to bzr and a PPA for easier installation on lucid, maverick, natty, or oneiric!
      So now, you can install the 5.1 channel Ubuntu login sound easily from this PPA to any supported Ubuntu release with:

      sudo apt-add-repository ppa:kirkland/sound
      sudo apt-get update
      sudo apt-get install ubuntu-sounds

      Log out, and then log back in.  If your Ubuntu system is hooked up (correctly) to a 5.1 stereo receiver, then you should hear the login sound start in the center speaker, then spread outwards to the front left and right channels, with the sound moving from the front to the rear for the whoosh and crickets at the end.  Oh, and the bongos should be bumpin' in your sub woofer :-)

      If you're interested in the sources, they're in bzr too:

      bzr branch lp:~kirkland/ubuntu-sounds/834802

      Finally, if you'd like to see this land in Ubuntu, mark bug #834802 as "affects me too"!

      I'll embed the audio here, but it sounds really different in the various browsers I've tested (Firefox, Chromium, Chrome).  Sounds like the the multi-channel OGG is being correctly passed to Pulse Audio for proper downmixing/discrete playback in Firefox, but not in Chrome/Chromium.  So your mileage may vary! :-)

      Doh! Your browser doesn't support embedded audio :-(


      Read more
      Dustin Kirkland

      It was way too hot here in Austin, Texas this weekend, as it hit 110F on Sunday!  So I spent most of the heat of the day inside, toying with something that I think is pretty cool :-)  I couldn't find any OS today (Mac, Windows, or Linux) that has a 5.1 channel login sound...  I'm hoping that Ubuntu might be the first!

      I have 7.1 channel surround sound in my home theater, which is great for watching movies.  Hooked up to my projector is (of course) an Ubuntu nettop, which I use to stream and serve most of my media content.

      I thought it would be neat to remix the Ubuntu login sound in 5.1 channels, to exercise my theater's surround sound at boot.

      So I grabbed the familiar "drums and crickets" OGG file, which you can find at /usr/share/sounds/ubuntu/stereo/desktop-login.ogg, and opened it in audacity, a phenomenal open source mixer.  I split that stereo track into two mono tracks, and then added four more blank tracks.

      The first two tracks are the Left and Right channels, respectively, followed by the Center channel, the Sub woofer channel, and then the Surround Left and Surround Right channels.  I copied the Left and Right channels to the Surround Left and Surround Right channels.

      Then, I opened the original desktop-login.ogg again, and mixed that stereo track to a single mono track.  I took that mono track and copied it to the Center and Sub woofer channels.

      Okay, now I had 6 tracks ... time to start playing with them!

      I decided that I wanted the "crickets and wind" at the end of the clip to be exclusively in my rear, surround channels.  So I silenced the Surround Left and Surround Right tracks until about the 3.85 second mark, and then faded in from 3.85 seconds to 5.43 seconds, and faded out from 5.43 seconds until the end of the clip.  Since I wanted that sound exclusively in the rear channels, I silenced each of the Left, Right, Center, and Sub woofer channels from the 5.0 second mark, until the end of the clip.  Next, I smoothly faded out the Left and Right channels from about 2.21 until the 4.54 second marks.

      For the intro, I wanted the first few drum beats to emanate from the center channel, and then spread wide to the Left and Right channels, right up to the big cymbal crash and the crescendo of the clip.  So I took the Center channel and added a very long fade, from the 0.30 second mark until about 3.97 seconds.  And then I set the Left and Right channels to slowly fade in, from 0 seconds to about 1.48 seconds.

      Finally, I took the bass track and de-amplified it way down.  And then I applied a low-pass bass boost filter several times, until the lowest hits of the bass drum are the only audible parts of the track.

      Want to hear it for yourself?  Well, you'll have to have 5.1 speakers in a true Surround Sound setup...  If so, grab the [flacogg, or wav] file, and open it in smplayer, ensuring that you have 5.1 channel sound enabled in smplayer.

      With the right equipment, you should be in for a treat!  The first few drum beats you'll hear in your Center channel along with some solid, thumping bass hits.  The sound should spread quickly from the Center, fanning outward toward your Right and Left channels right up to the big crashing cymbal!  And with that crescendo, the Left, Right, Center, and Sub should all gracefully fall silent, while the crickets and the wooshing wind sweep back to your Rear Left and Rear Right channels!

      Don't have 5.1 sound?  Well, you can still listen to each track individually.  Grab the [flac, ogg, or wav] file, and open it in audacity.  You should see 6 channels vertically down your screen.  You can click the Solo button next to each track, and listen to each track one-by-one.  Make sure you un-click the Solo button between plays.  This might give you a decent idea of how each of the channels come together.

      Fancy yourself a sound producer?  Remix it again and share :-)  I have the wav sources up at lp:~kirkland/ubuntu-sounds/834802. Better yet, how about creating a whole new Ubuntu login sound?  :-)  Maybe one day....

      From the right side of my brain,

      Read more
      Dustin Kirkland

      I was pretty stoked to read earlier this month that ChromeOS and Chrome/Chromium was getting a Netflix app in the Chrome Webstore.  I installed it earlier tonight, but sadly it's not working on Chrome or Chromium on Ubuntu.  I installed it on my Cr-48, and it worked there.  Reports on the page indicate that it's working on Chrome/Windows.  But Chrome/Chromium on Linux -- no dice :-(

      So the Netflix-on-Linux blues continue, unfortunately :-(

      In looking for workarounds, I came across this web petition for Netflix-on-Linux support:

      So I signed the petition and was impressed to see 16,518 other signatures!

      In fact, I downloaded all of the signatures and did a little (far from scientific) grepping of my own to see where Ubuntu stood among the other desktops in the signature list.  Ubuntu lands at nearly 70%.  Impressive!

      Ubuntu 11433 69.2%
      Fedora/RH/CentOS 1600 9.7%
      Mint 1092 6.6%
      Arch 891 5.4%
      Debian 856 5.2%
      SuSE 596 3.6%
      Other 50 0.3%



      Read more