Canonical Voices

Posts tagged with 'ubuntu'

Leo Arias

Here at Ubuntu we are working hard on the future of free software distribution. We want developers to release their software to any Linux distro in a way that's safe, simple and flexible. You can read more about this at snapcraft.io.

This work is extremely fun because we have to work constantly with a wild variety of free software projects to make sure that the tools we write are usable and that the workflow we are proposing makes sense to developers and gives them a lot of value in return. Today I want to talk about one of those projects: IPFS.

IPFS is the permanent and decentralized web. How cool is that? You get a peer-to-peer distributed file system where you store and retrieve files. They have a nice demo in their website, and you can give it a try on Ubuntu Trusty, Xenial or later by running:

$ sudo snap install ipfs

screenshot of the IPFS peers

So, here's one of the problems we are trying to solve. We have millions of users on the Trusty version of Ubuntu, released during 2014. We also have millions of users on the Xenial version, released during 2016. Those two versions are stable now, and following the Ubuntu policies, they will get only security updates for 5 years. That means that it's very hard, almost impossible, for a young project like IPFS to get into the Ubuntu archives for those releases. There will be no simple way for all those users to enjoy IPFS, they would have to use a Personal Package Archive or install the software from a tarball. Both methods are complex with high security risks, and both require the users to put a lot of trust on the developers, more than what they should ever trust anybody.

We are closing the Zesty release cycle which will go out in April, so it's too late there too. IPFS could make a deb, put it into Debian, wait for it to sync to Ubuntu, and then it's likely that it will be ready for the October release. Aside from the fact that we have to wait until October, there are a few other problems. First, making a deb is not simple. It's not too hard either, but it requires quite some time to learn to do it right. Second, I mentioned that IPFS is young, they are on the 0.4.6 version. So, it's very unlikely that they will want to support this early version for such a long time as Debian and Ubuntu require. And they are not only young, they are also fast. They add new features and bug fixes every day and make new releases almost every week, so they need a feedback loop that's just as fast. A 6 months release cycle is way too slow. That works nicely for some kinds of free software projects, but not for one like IPFS.

They have been kind enough to let me play with their project and use it as a test subject to verify our end-to-end workflow. My passion is testing, so I have been focusing on continuous delivery to get happy early adopters and constant feedback about the most recent changes in the project.

I started by making a snapcraft.yaml file that contains all the metadata required for the snap package. The file is pretty simple and to make the first version it took me just a couple of minutes, true story. Since then I've been slowly improving and updating it with small changes. If you are interested in doing the same for your project, you can read the tutorial to create a snap.

I built and tested this snap locally on my machines. It worked nicely, so I pushed it to the edge channel of the Ubuntu Store. Here, the snap is not visible on user searches, only the people who know about the snap will be able to install it. I told a couple of my friends to give it a try, and they came back telling me how cool IPFS was. Great choice for my first test subject, no doubt.

At this point, following the pace of the project by manually building and pushing new versions to the store was too demanding, they go too fast. So, I started working on continuous delivery by translating everything I did manually into scripts and hooking them to travis-ci. After a few days, it got pretty fancy, take a look at the github repo of the IPFS snap if you are curious. Every day, a new version is packaged from the latest state of the master branch of IPFS and it is pushed to the edge channel, so we have a constant flow of new releases for hardcore early adopters. After they install IPFS from the edge channel once, the package will be automatically updated in their machines every day, so they don't have to do anything else, just use IPFS as they normally would.

Now with this constant stream of updates, me and my two friends were not enough to validate all the new features. We could never be sure if the project was stable enough to be pushed to the stable channel and make it available to the millions and millions of Ubuntu users out there.

Luckily, the Ubuntu community is huge, and they are very nice people. It was time to use the wisdom of the crowds. I invited the most brave of them to keep the snap installed from edge and I defined a simple pipeline that leads to the stable release using the four available channels in the Ubuntu store:

  • When a revision is tagged in the IPFS master repo, it is automatically pushed to edge channel from travis, just as with any other revision.
  • Travis notifies me about this revision.
  • I install this tagged revision from edge, and run a super quick test to make sure that the IPFS server starts.
  • If it starts, I push the snap to the beta channel.
  • With a couple of my friends, we run a suite of smoke tests.
  • If everything goes well, I push the snap to the candidate channel.
  • I notify the community of Ubuntu testers about a new version in the candidate channel. This is were the magic of crowd testing happens.
  • The Ubuntu testers run the smoke tests in all their machines, which gives us the confidence we need because we are confirming that the new version works on different platforms, distros, distro releases, countries, network topologies, you name it.
  • This candidate release is left for some time in this channel, to let the community run thorough exploratory tests, trying to find weird usage combinations that could break the software.
  • If the tag was for a final upstream release, the community also runs update tests to make sure that the users with the stable snap installed will get this new version without issues.
  • After all the problems found by the community have been resolved or at least acknowledged and triaged as not blockers, I move the snap from candidate to the stable channel.
  • All the users following the stable channel will automatically get a very well tested version, thanks to the community who contributed with the testing and accepted a higher level of risk.
  • And we start again, the never-ending cycle of making free software :)

Now, let's go back to the discussion about trust. Debian and Ubuntu, and most of the other distros, rely on maintainers and distro developers to package and review every change on the software that they put in their archives. That is a lot of work, and it slows down the feedback loop a lot, as we have seen. In here we automated most of the tasks of a distro maintainer, and the new revisions can be delivered directly to the users without any reviews. So the users are trusting directly their upstream developers without intermediaries, but it's very different from the previously existing and unsafe methods. The code of snaps is installed read-only, very well constrained with access only to their own safe space. Any other access needs to be declared by the snap, and the user is always in control of which access is permitted to the application.

This way upstream developers can go faster but without exposing their users to unnecessary risks. And they just need a simple snapcraft.yaml file and to define their own continuous delivery pipeline, on their own timeline.

By removing the distro as the intermediary between the developers and their users, we are also making a new world full of possibilities for the Ubuntu community. Now they can collaborate constantly and directly with upstream developers, closing this quick feedback loop. In the future we will tell our children of the good old days when we had to report a bug in Ubuntu, which would be copied to Debian, then sent upstream to the developers, and after 6 months, the fix would arrive. It was fun, and it lead us to where we are today, but I will not miss it at all.

Finally, what's next for IPFS? After this experiment we got more than 200 unique testers and almost 300 test installs. I now have great confidence on this workflow, new revisions were delivered on time, existing Ubuntu testers became new IPFS contributors and I now can safely recommend IPFS users to install the stable snap. But there's still plenty of work ahead. There are still manual steps in the pipeline that can be scripted, the smoke tests can be automated to leave more free time for exploratory testing, we can release also to armhf and arm64 architectures to get IPFS into the IoT world, and well, of course the developers are not stopping, they keep releasing new interesting features. As I said, plenty of opportunities for us as distro contributors.

screenshot of the IPFS snap stats

I'd like to thank everybody who tested the IPFS snap, specially the following people for their help and feedback:

  • freekvh
  • urcminister
  • Carla Sella
  • casept
  • Colin Law
  • ventrical
  • cariboo
  • howefield

<3

If you want to release your project to the Ubuntu store, take a look at the snapcraft docs, the Ubuntu tutorials, and come talk to us in Rocket Chat.

Read more
Dustin Kirkland

Mobile World Congress is simply one of the biggest trade shows in the entire world.

It's also, perhaps, the best place in the world to see how encompassing the Ubuntu ecosystem actually is.

Canonical and our partners demonstrated Ubuntu running on dozens of devices -- from robots, to augmented reality headsets, digital signs, vending machines, IoT Gateways, cell tower base stations, phones, tablets, servers, from super computers to tiny, battery powered embedded controllers.

But that was only a tiny fraction of the Ubuntu running at MWC!

We saw Ubuntu at the heart of demos from Dell, AMD, Intel, IBM, Deutsche Telekom, DJI, and hundreds of other booths, running autonomous drones, national telephone networks, self driving cars, smart safety helmets, inflight entertainment systems, and so, so, so much more.

Among the thousands of customers, prospects, fans, competitors, students, and industry executives, we even received a visit from (the somewhat controversial?) King of Spain!

It was an incredible week, with no fewer than 12 hours per day, on our feet, telling the Ubuntu story.
And what a story it is... I hope you enjoy.

Cheers,
Dustin





































Read more
Dustin Kirkland



Yesterday, I delivered a talk to a lively audience at ContainerWorld in Santa Clara, California.

If I measured "the most interesting slides" by counting "the number of people who took a picture of the slide", then by far "the most interesting slides" are slides 8-11, which pose an answer the question:
"Should I run my PaaS on top of my IaaS, or my IaaS on top of my PaaS"?
In the Ubuntu world, that answer is super easy -- however you like!  At Canonical, we're happy to support:
  1. Kubernetes running on top of Ubuntu OpenStack
  2. OpenStack running on top of Canonical Kubernetes
  3. Kubernetes running along side OpenStack
In all cases, the underlying substrate is perfectly consistent:
  • you've got 1 to N physical or virtual machines
  • which are dynamically provisioned by MAAS or your cloud provider
  • running stable, minimal, secure Ubuntu server image
  • carved up into fast, efficient, independently addressable LXD machine containers
With that as your base, we'll easily to conjure-up a Kubernetes, an OpenStack, or both.  And once you have a Kubernetes or OpenStack, we'll gladly conjure-up one inside the other.


As always, I'm happy to share my slides with you here.  You're welcome to download the PDF, or flip through the embedded slides below.



Cheers,
Dustin

Read more
Leo Arias

Call for testing: MySQL

I promised that more interesting things were going to be available soon for testing in Ubuntu. There's plenty coming, but today here is one of the greatest:

$ sudo snap install mysql --channel=8.0/beta

screenshot of mysql snap running

Lars Tangvald and other people at MySQL have been working on this snap for some time, and now they are ready to give it to the community for crowd testing. If you have some minutes, please give them a hand.

We have a testing guide to help you getting started.

Remember that this should run in trusty, xenial, yakkety, zesty and in all flavours of Ubuntu. It would be great to get a diverse pool of platforms and test it everywhere.

In here we are introducing a new concept: tracks. Notice that we are using --channel=8.0/beta, instead of only --beta as we used to do before. That's because mysql has two different major versions currently active. In order to try the other one:

$ sudo snap install mysql --channel=5.7/beta

Please report back your results. Any kind of feedback will be highly appreciated, and if you have doubts or need a hand to get started, I'm hanging around in Rocket Chat.

Read more
Dustin Kirkland

Introducting the Canonical Livepatch Service
Howdy!

Ubuntu 16.04 LTS’s 4.4 Linux kernel includes an important new security capability in Ubuntu -- the ability to modify the running Linux kernel code, without rebooting, through a mechanism called kernel livepatch.

Today, Canonical has publicly launched the Canonical Livepatch Service -- an authenticated, encrypted, signed stream of Linux livepatches that apply to the 64-bit Intel/AMD architecture of the Ubuntu 16.04 LTS (Xenial) Linux 4.4 kernel, addressing the highest and most critical security vulnerabilities, without requiring a reboot in order to take effect.  This is particularly amazing for Container hosts -- Docker, LXD, etc. -- as all of the containers share the same kernel, and thus all instances benefit.



I’ve tried to answer below some questions that you might have. As you have others, you’re welcome
to add them to the comments below or on Twitter with hastag #Livepatch.

Retrieve your token from ubuntu.com/livepatch

Q: How do I enable the Canonical Livepatch Service?

A: Three easy steps, on a fully up-to-date 64-bit Ubuntu 16.04 LTS system.
  1. Go to https://ubuntu.com/livepatch and retrieve your livepatch token
    1. Install the canonical-livepatch snap
      $ sudo snap install canonical-livepatch 
    2. Enable the service with your token
      $ sudo canonical-livepatch enable [TOKEN] 
    And you’re done! You can check the status at any time using:

    $ canonical-livepatch status --verbose

      Q: What are the system requirements?

      A: The Canonical Livepatch Service is available for the generic and low latency flavors of the 64-bit Intel/AMD (aka, x86_64, amd64) builds of the Ubuntu 16.04 LTS (Xenial) kernel, which is a Linux 4.4 kernel. Canonical livepatches work on Ubuntu 16.04 LTS Servers and Desktops, on physical machines, virtual machines, and in the cloud. The safety, security, and stability firmly depends on unmodified Ubuntu kernels and network access to the Canonical Livepatch Service (https://livepatch.canonical.com:443).  You also will need to apt update/upgrade to the latest version of snapd (at least 2.15).

      Q: What about other architectures?

      A: The upstream Linux livepatch functionality is currently limited to the 64-bit x86 architecture, at this time. IBM is working on support for POWER8 and s390x (LinuxOne mainframe), and there’s also active upstream development on ARM64, so we do plan to support these eventually. The livepatch plumbing for 32-bit ARM and 32-bit x86 are not under upstream development at this time.

      Q: What about other flavors?

      A: We are providing the Canonical Livepatch Service for the generic and low latency (telco) flavors of the the Linux kernel at this time.

      Q: What about other releases of Ubuntu?

      A: The Canonical Livepatch Service is provided for Ubuntu 16.04 LTS’s Linux 4.4 kernel. Older releases of Ubuntu will not work, because they’re missing the Linux kernel support. Interim releases of Ubuntu (e.g. Ubuntu 16.10) are targeted at developers and early adopters, rather than Long Term Support users or systems that require maximum uptime.  We will consider providing livepatches for the HWE kernels in 2017.

      Q: What about derivatives of Ubuntu?

      A: Canonical livepatches are fully supported on the 64-bit Ubuntu 16.04 LTS Desktop, Cloud, and Server operating systems. On other Ubuntu derivatives, your mileage may vary! These are not part of our automated continuous integration quality assurance testing framework for Canonical Livepatches. Canonical Livepatch safety, security, and stability will firmly depend on unmodified Ubuntu generic kernels and network access to the Canonical Livepatch Service.

      Q: How does Canonical test livepatches?

      A: Every livepatch is rigorously tested in Canonical's in-house CI/CD (Continuous Integration / Continuous Delivery) quality assurance system, which tests hundreds of combinations of livepatches, kernels, hardware, physical machines, and virtual machines.  Once a livepatch passes CI/CD and regression tests, it's rolled out on a canary testing basis, first to a tiny percentage of the Ubuntu Community users of the Canonical Livepatch Service. Based on the success of that microscopic rollout, a moderate rollout follows.  And assuming those also succeed, the livepatch is delivered to all free Ubuntu Community and paid Ubuntu Advantage users of the service.  Systemic failures are automatically detected and raised for inspection by Canonical engineers.  Ubuntu Community users of the Canonical Livepatch Service who want to eliminate the small chance of being randomly chosen as a canary should enroll in the Ubuntu Advantage program (starting at $12/month).

      Q: What kinds of updates will be provided by the Canonical Livepatch Service?

      A: The Canonical Livepatch Service is intended to address high and critical severity Linux kernel security vulnerabilities, as identified by Ubuntu Security Notices and the CVE database. Note that there are some limitations to the kernel livepatch technology -- some Linux kernel code paths cannot be safely patched while running. We will do our best to supply Canonical Livepatches for high and critical vulnerabilities in a timely fashion whenever possible. There may be occasions when the traditional kernel upgrade and reboot might still be necessary. We’ll communicate that clearly through the usual mechanisms -- USNs, Landscape, Desktop Notifications, Byobu, /etc/motd, etc.

      Q: What about non-security bug fixes, stability, performance, or hardware enablement updates?

      A: Canonical will continue to provide Linux kernel updates addressing bugs, stability issues, performance problems, and hardware compatibility on our usual cadence -- about every 3 weeks. These updates can be easily applied using ‘sudo apt update; sudo apt upgrade -y’, using the Desktop “Software Updates” application, or Landscape systems management. These standard (non-security) updates will still require a reboot, as they always have.

      Q: Can I rollback a Canonical Livepatch?

      A: Currently rolling-back/removing an already inserted livepatch module is disabled in Linux 4.4. This is because we need a way to determine if we are currently executing inside a patched function before safely removing it. We can, however, safely apply new livepatches on top of each other and even repatch functions over and over.

      Q: What about low and medium severity CVEs?

      A: We’re currently focusing our Canonical Livepatch development and testing resources on high and critical security vulnerabilities, as determined by the Ubuntu Security Team.  We'll livepatch other CVEs opportunistically.

      Q: Why are Canonical Livepatches provided as a subscription service?

      A: The Canonical Livepatch Service provides a secure, encrypted, authenticated connection, to ensure that only properly signed livepatch kernel modules -- and most importantly, the right modules -- are delivered directly to your system, with extremely high quality testing wrapped around it.

      Q: But I don’t want to buy UA support!

      A: You don’t have to! Canonical is providing the Canonical Livepatch Service to community users of Ubuntu, at no charge for up to 3 machines (desktop, server, virtual machines, or cloud instances). A randomly chosen subset of the free users of Canonical Livepatches will receive their Canonical Livepatches slightly earlier than the rest of the free users or UA users, as a lightweight canary testing mechanism, benefiting all Canonical Livepatch users (free and UA). Once those canary livepatches apply safely, all Canonical Livepatch users will receive their live updates.

      Q: But I don’t have an Ubuntu SSO account!

      A: An Ubuntu SSO account is free, and provides services similar to Google, Microsoft, and Apple for Android/Windows/Mac devices, respectively. You can create your Ubuntu SSO account here.

      Q: But I don’t want login to ubuntu.com!

      A: You don’t have to! Canonical Livepatch is absolutely not required maintain the security of any Ubuntu desktop or server! You may continue to freely and anonymously ‘sudo apt update; sudo apt upgrade; sudo reboot’ as often as you like, and receive all of the same updates, and simply reboot after kernel updates, as you always have with Ubuntu.

      Q: But I don't have Internet access to livepatch.canonical.com:443!

      A: You should think of the Canonical Livepatch Service much like you think of Netflix, Pandora, or Dropbox.  It's an Internet streaming service for security hotfixes for your kernel.  You have access to the stream of bits when you can connect to the service over the Internet.  On the flip side, your machines are already thoroughly secured, since they're so heavily firewalled off from the rest of the world!

      Q: Where’s the source code?

      A: The source code of livepatch modules can be found here.  The source code of the canonical-livepatch client is part of Canonical's Landscape system management product and is commercial software.

      Q: What about Ubuntu Core?

      A: Canonical Livepatches for Ubuntu Core are on the roadmap, and may be available in late 2016, for 64-bit Intel/AMD architectures. Canonical Livepatches for ARM-based IoT devices depend on upstream support for livepatches.

      Q: How does this compare to Oracle Ksplice, RHEL Live Patching and SUSE Live Patching?

      A: While the concepts are largely the same, the technical implementations and the commercial terms are very different:

      • Oracle Ksplice uses it’s own technology which is not in upstream Linux.
      • RHEL and SUSE currently use their own homegrown kpatch/kgraft implementations, respectively.
      • Canonical Livepatching uses the upstream Linux Kernel Live Patching technology.
      • Ksplice is free, but unsupported, for Ubuntu Desktops, and only available for Oracle Linux and RHEL servers with an Oracle Linux Premier Support license ($2299/node/year).
      • It’s a little unclear how to subscribe to RHEL Kernel Live Patching, but it appears that you need to first be a RHEL customer, and then enroll in the SIG (Special Interests Group) through your TAM (Technical Account Manager), which requires Red Hat Enterprise Linux Server Premium Subscription at $1299/node/year.  (I'm happy to be corrected and update this post)
      • SUSE Live Patching is available as an add-on to SUSE Linux Enterprise Server 12 Priority Support subscription at $1,499/node/year, but does come with a free music video.
      • Canonical Livepatching is available for every Ubuntu Advantage customer, starting at our entry level UA Essential for $150/node/year, and available for free to community users of Ubuntu.

      Q: What happens if I run into problems/bugs with Canonical Livepatches?

      A: Ubuntu Advantage customers will file a support request at support.canonical.com where it will be serviced according to their UA service level agreement (Essential, Standard, or Advanced). Ubuntu community users will file a bug report on Launchpad and we'll service it on a best effort basis.

      Q: Why does canonical-livepatch client/server have a proprietary license?

      A: The canonical-livepatch client is part of the Landscape family of tools available to Canonical support customers. We are enabling free access to the Canonical Livepatch Service for Ubuntu community users as a mark of our appreciation for the broader Ubuntu community, and in exchange for occasional, automatic canary testing.

      Q: How do I build my own livepatches?

      A: It’s certainly possible for you to build your own Linux kernel live patches, but it requires considerable skill, time, computing power to produce, and even more effort to comprehensively test. Rest assured that this is the real value of using the Canonical Livepatch Service! That said, Chris Arges has blogged a howto for the curious a while back:

      http://chrisarges.net/2015/09/21/livepatch-on-ubuntu.html

      Q: How do I get notifications of which CVEs are livepatched and which are not?

      A: You can, at any time, query the status of the canonical-livepatch daemon using: ‘canonical-livepatch status --verbose’. This command will show any livepatches successfully applied, any outstanding/unapplied livepatches, and any error conditions. Moreover, you can monitor the Ubuntu Security Notices RSS feed and the ubuntu-security-announce mailing list.

      Q: Isn't livepatching just a big ole rootkit?

      A: Canonical Livepatches inject kernel modules to replace sections of binary code in the running kernel. This requires the CAP_SYS_MODULE capability. This is required to modprobe any module into the Linux kernel. If you already have that capability (root does, by default, on Ubuntu), then you already have the ability to arbitrarily modify the kernel, with or without Canonical Livepatches. If you’re an Ubuntu sysadmin and you want to disable module loading (and thereby also disable Canonical Livepatches), simply ‘echo 1 | sudo tee /proc/sys/kernel/modules_disabled’.

      Keep the uptime!
      :-Dustin

      Read more
      Leo Arias

      There is a huge announcement coming: snaps now run in Ubuntu 14.04 Trusty Tahr.

      Take a moment to note how big this is. Ubuntu 14.04 is a long-term release that will be supported until 2019. Ubuntu 16.04 is also a long-term release that will be supported until 2021. We have many many many users in both releases, some of which will stay there until we drop the support. Before this snappy new world, all those users were stuck with the versions of all their programs released in 2014 or 2016, getting only updates for security and critical issues. Just try to remember how your favorite program looked 5 years ago; maybe it didn't even exist. We were used to choose between stability and cool new features.

      Well, a new world is possible. With snaps you can have a stable base system with frequent updates for every program, without the risk of breaking your machine. And now if you are a Trusty user, you can just start taking advantage of all this. If you are a developer, you have to prepare only one release and it will just work in all the supported Ubuntu releases.

      Awesome, right? The Ubuntu devs have been doing a great job. snapd has already landed in the Trusty archive, and we have been running many manual and automated tests on it. So we would like now to invite the community to test it, explore weird paths, try to break it. We will appreciate it very much, but all of those Trusty users out there will love it, when they receive loads of new high quality free software on their oldie machines.

      So, how to get started?

      If you are already running Trusty, you will just have to install snapd:

      $ sudo apt update && sudo apt install snapd
      

      Reboot your system after that in case you had a kernel update pending, and to get the paths for the new snap binaries set up.

      If you are running a different Ubuntu release, you can Install Ubuntu in a virtual machine. Just make sure that you install the http://releases.ubuntu.com/14.04/ubuntu-14.04.5-desktop-amd64.iso.

      Once you have Trusty with snapd ready, try a few commands:

      $ snap list
      $ sudo snap install hello-world
      $ hello-world
      $ snap find something
      

      screenshot of snaps running in Trusty

      Keep searching for snaps until you find one that's interesting. Install it, try it, and let us know how it goes.

      If you find something wrong, please report a bug with the trusty tag. If you are new to the Ubuntu community or get lost on the way, come and join us in Rocket Chat.

      And after a good session of testing, sit down, relax, and get ohmygiraffe. With love from popey:

      $ sudo snap install ohmygiraffe
      $ ohmygiraffe
      

      screenshot of ohmygiraffe

      Read more
      Leo Arias

      After a little break, on the first Friday of February we resumed the Ubuntu Testing Days.

      This session was pretty interesting, because after setting some of the bases last year we are now ready to dig deep into the most important projects that will define the future of Ubuntu.

      We talked about Ubuntu Core, a snap package that is the base of the operating system. Because it is a snap, it gets the same benefits as all the other snaps: automatic updates, rollbacks in case of error during installation, read-only mount of the code, isolation from other snaps, multiple channels on the store for different levels of stability, etc.

      The features, philosophy and future of Core were presented by Michael Vogt and Zygmunt Krynicki, and then Federico Giménez did a great demo of how to create an image and test it in QEMU.

      Click the image below to watch the full session.

      Alt text

      There are plenty of resources in the Ubuntu websites related to Ubuntu Core.

      To get started, we recommend to follow this guide to run the operating system in a virtual machine.

      After that, and if you are feeling brave and want to help Michael, Zygmund and Federico, you can download the candidate image instead, from http://cdimage.ubuntu.com/ubuntu-core/16/candidate/pending/ubuntu-core-16-amd64.img.xz This is the image that's being currently tested, so if you find something wrong or weird, please report a bug in Launchpad.

      Finally, if you want to learn more about the snaps that compose the image and take a peek at the things that we'll cover in the following testing days, you can follow the tutorial to create your own Core image.

      On this session we were also accompanied by Robert Wolff who works on 96boards at Linaro. He has an awesome show every Thursday called Open Hours. At 96boards they are building open Linux boards for prototyping and embedded computing. Anybody can jump into the Open Hours to learn more about this cool work.

      The great news that Robert brought is that both Open Hours and Ubuntu Testing Days will be focused on Ubuntu Core this month. He will be our guest again next Friday, February 10th, where he will be talking about the DragonBoard 410c. Also my good friend Oliver Grawert will be with us, and he will talk about the work he has been doing to enable Ubuntu in this board.

      Great topics ahead, and a full new world of possiblities now that we are mixing free software with open hardware and affordable prototyping tools. Remember, every Friday in http://ubuntuonair.com/, no se lo pierda.

      Read more
      deviceguy

      Movin' on...

      A year has gone by since I started work with Canonical. As it turns out, I must be on my way. Where to? Not real sure at this moment, there seems plenty of companies using Qt & QML these days. \0/


      But saying that, I am open to suggestions. LinkedIn
       
      Plenty of IoT and devices using sensors around. Heck, even Moto Z phone has some great uses for sensor gestures similar to what I wrote for QtSensors while I was at Nokia.

      But a lack of companies that allow freelance or remote work. The last few years I have worked remotely doing work for Jolla and Canonical. Both fantastic companies to work for, which really have it together for working remotely.

      I am still surprised that only a handful of companies regularly allow remote work. I do not miss the stuffy non window opening offices and the long daily commute, which sometimes means riding a motorcycle through hail! (I do not suggest this for anyone)

      Of course, I am still maintainer for QtSensors, QtSystemInfo for the Qt Project, and Sensor Framework for Mer, and always dreaming up new ways to use sensors. Still keeping tabs on QtNetwork bearer classes.

      Although I had to send back the Canonical devices, I still have Ubuntu on my Nexus 4. I still have my Jolla phones and tablet.

      That said, I still have this blog here, and besides spending my time looking for a new programming gig, I am (always) preparing to release a new album. http://llornkcor.com
      and always willing to work with anyone needing music/audio/soundtrack work.

      Read more
      Leo Arias

      Happy new year Ubunteros and Ubunteras!

      If you have been following our testing days, you will know by now that our intention is to get more people contributing to Ubuntu and free software projects, and to help them getting started through testing and related tasks. So, we will be making frequent calls for testing where you can contribute and learn. Educational AND fun ^_^

      To start the year, I would like to invite you to test the IPFS candidate snap. IPFS is a really interesting free project for distributed storage. You can read more about it and watch a demo in the IPFS website.

      We have pushed a nice snap with their latest stable version to the candidate channel in the store. But before we publish it to the stable channel we would like to get more people testing it.

      You can get a clean and safe environment to test following some of the guides you'll find on the summaries of the past testing days.

      Or, if you want to use your current system, you can just do:

      $ sudo snap install ipfs --candidate
      

      I have written a gist with a simple guide to get started testing it

      If you finish that successfully and still have more time, or are curious about ipfs, please continue with an exploratory testing session. The idea here is just to execute random commands, try unusual inputs and just play around.

      You can get ideas from the IPFS docs.

      When you are done, please send me an email with your results and any comments. And if you get stuck or have any kind of question, please don't hesitate to ask. Remember that we welcome everybody.

      Read more
      Dustin Kirkland


      What's yours?

      Happy 2017!
      :-Dustin

      Read more
      Leo Arias

      Today we had the last Ubuntu Testing Day of the year.

      We invited Sergio Schvezov and Joe Talbott, to join Kyle and myself. Together we have been working on Snapcraft the whole year.

      Sergio did a great introduction of snapcraft, and showed some of the new features that will land next week in Ubuntu. And because it was the last day of work for everybody (except Kyle), we threw some beers into the hang out and made it our team end of year party.

      You can watch the full recording by clicking the image below.

      Alt text

      Snapcraft is one of the few projects that have an exception to land new features into released versions of Ubuntu. So every week we are landing new things in Xenial and Yakkety. This means that we need to constantly test that we are not breaking anything for all the people using stable Ubuntu releases; and it means that we would love to have many more hands helping us with those tests.

      If you would like to help, all you have to do is set up a virtual machine and enable the proposed pocket in there.

      This is the active bug for the Stable Release Update of snapcraft 2.24: bug #1650632

      Before I shut down my computer and start my holidays, I would like to thank all the Ubuntu community for one more year, it has been quite a ride. And I would like to thank Sergio, Kyle and Joe in particular. They are the best team a QA Engineer could ask for <3.

      See you next year for more testing days.

      Read more
      Leo Arias

      Ubuntu has a six month cycle for releases. This means that we work for six months updating software, adding new features and making sure that it all works consistently together when it’s integrated into the operating system, and then we release it. Once we make a release, it is considered stable and then we almost exclusively add to that release critical bug fixes and patches for security vulnerabilities. The exceptions are just a few, and are for software that’s important for Ubuntu and that we want to keep up-to-date with the latest features even in stable releases.

      These bug fixes, security patches and exceptional new features require a lot of testing, because right after they are published they will reach all the Ubuntu users that are in the stable release. And we want the release to remain stable, never ever introduce a regression that will make those users unhappy.

      We call all of them Stable Release Updates, and we test them in the proposed pocket of the Ubuntu archive. This is obviously not enabled by default, so the brave souls that want to help us testing the changes in proposed need to enable it.

      Before we go on, I would recommend to test SRUs in a virtual machine. Once you enable proposed following this guide you will get constant and untested updates from many packages, and these updates will break parts of your system every now and then. It’s not likely to be critical, but it can bother you if it happens on the machine you need to do your work, or other stuff. And if somebody makes a mistake, you might need to reinstall the system.

      You will also have to find a package that needs testing. Snapcraft is one of the few exceptions allowed to land in a stable release every week. So lets use it as an example. Lets say you want to help us testing the upcoming release of snapcraft in Ubuntu 16.04.

      With your machine ready and before enabling proposed, install the version already released of the package you want to test. This way you’ll test later an update to the newer version, just what a normal user would get once the update is approved and lands in the archive. So in a terminal, write:

        $ sudo apt update
        $ sudo apt install snapcraft

      Or replace snapcraft with whatever package you are testing. If you are doing it just during the weekend after I am writing this, the released version of snapcraft will be 2.23. You can check it with:

        $ dpkg -s snapcraft | grep Version
        Version: 2.23

      Now, to enable proposed, open the Ubuntu Software application, and select Software & Updates from the menu in the top of the window.

      image

      From the Software & Updates window, select the Developer Options tab. And check the item that says Pre-released updates.

      image

      This will prompt for your password. Enter it, and then click the Close button. You will be asked if you want to reload your sources, so yes, click the Reload button.

      image

      Finally try to upgrade. If there is a newer version available in the proposed pocket for the packet you are testing, now you will get it.

        $ sudo apt install snapcraft
        $ dpkg -s snapcraft | grep Version
        Version: 2.24

      Every time there is a proposed update, the package will have corresponding SRU bugs with the tag “verification-needed”. In the case of snapcraft this weekend, this is the bug for the 2.24 proposed update: https://bugs.launchpad.net/ubuntu/+source/snapcraft/+bug/1650632

      The SRU bugs will have instructions about how to test that the fix or the new features released are correct. Follow those steps, and if you are confident that they work as expected, change the tag to “verification-done”. If you find that the fix is not correct, change the tag to “verification-failed”. In case of doubt, you can leave a comment in the bug explaining what you tried and what you found.

      You can read more about SRUs in the Stable Release Updates wiki page, and also in the wiki page explaining how to perform verifications. This last page includes a section to find packages and bugs that need verification. If you want to help the Ubuntu community, you can just jump in and start verifying some of the pending bugs. It will be highly appreciated.

      If you have questions or find a problem, join us in the Ubuntu Rocket Chat.

      Read more
      Daniel Holbach

      Taking a break

      It’s a bit strange to write this blog post in the same week as Martin Pitt is announcing moving on from Canonical. I remember many moments of Martin’s post very vividly and he was one of the first I ran into on my flight to Sydney for Ubuntu Down Under in 2005.

      Fast forward to today: 2016 was a year full of change – my personal life was no exception there. In the last weeks I had to realise more and more that I need a long break from everything. I therefore decided to move on from Canonical, take some time off, wander the world, recharge my batteries, come back and surprise you all with what’s next.

      I’m very much leaving on good terms and I could imagine I won’t be too far away (I’d miss all you great people who became good friends way too much). Having been with Canonical for 11 years and 12 years in the Ubuntu community, it has been an incredibly hard decision to take. Still it’s necessary now and it’ll be good open myself up again to new challenges, new ways of working and new sets of problems.

      It was a great privilege to work with you all and be able to add my humble contribution to this crazy undertaking called Ubuntu. I’m extremely grateful for the great moments with you all, the opportunities to learn, your guidance, the friends I made around the world, the laughs, the discussions, the excellent work we did together. This was a very important time of my life.

      In the coming weeks I will be without internet, I haven’t quite decided yet, which part of the world I’m going to go to, but maybe I’ll post a picture or two somewhere. </p>
            <a href=Read more

      Leo Arias

      For the third session of the Ubuntu Testing Days, Kevin Gunn joined us to talk about the Unity8 snap.

      This is a thriving project with lots of things to do and bugs to uncover, so it's the perfect candidate for newcomers eager to help Ubuntu.

      If you are interested and didn't attend our meeting on Friday, click the image below to watch the recording.

      Alt text

      To install Unity 8 in the Ubuntu 16.04 cloned virtual machine that we prepared on the past session, enter the following commands in the terminal:

      $ sudo add-apt-repository -u ppa:ci-train-ppa-service/stable-phone-overlay
      $ sudo apt install unity8-session-snap
      $ unity8-snap-install
      $ sudo reboot
      

      After reboot, on the login screen click the Ubuntu icon next to your user name and change the selection to Unity 8.

      Kevin and his team have a google doc with the instructions to build and install the snap, and the current status. If you find a bug, you can talk to the developers joining the #ubuntu-touch IRC channel on freenode

      While preparing the testing environment on this session we had a crash, and explained how easy it is to send the report to the Ubuntu developers by just clicking the Report problem... button. The reports from all our users are in the error tracker, along with a frequency graph and the links to bug reports where those crashes are being fixed.

      We also showed two tricks to make faster and more pleasant the testing session in a virtual machine:

      Thanks to Julia, Kyle and Kevin for the nice session.

      Join us again next Friday at Ubuntu On-Air.

      Read more
      pitti

      I’ve had the pleasure of working on Ubuntu for 12½ years now, and during that time used up an entire Latin alphabet of release names! (Well, A and C are still free, but we used H and W twice, so on average.. ☺ ) This has for sure been the most exciting time in my life with tons of good memories! Very few highlights:

      • Getting some spam mail from a South African multi-millionaire about a GREAT OPPORTUNITY
      • Joining #warthogs (my first IRC experience) and collecting my first bounties for “derooting” Debian (i. e. drop privileges from root daemons and suid binaries)
      • Getting invited to Oxford to meet a bunch of people for which I had absolutely zero proof of existence, and tossing myself into debts for buying a laptop for that occasion
      • Once being there, looking into my fellows’ stern and serious faces and being amazed by their professionalism:
      • The excitement and hype around going public with Warty Warthogs Beta
      • Meeting lots of good folks at many UDSes, with great ideas and lots of enthusiasm, and sometimes “Bags of Death”. Group photo from Ubuntu Down Under:
      • Organizing UDSes without Launchpad or other electronic help:
         
      • Playing “Wish you were Here” with Bill, Tony, Jono, and the other All Stars
      • Seeing bug #1 getting closed, and watching the transformation of Microsoft from being TEH EVIL of the FOSS world to our business partner
      • Getting to know lots of great places around the world. My favourite: luring a few colleagues for a “short walk through San Francisco” but ruining their feet with a 9 hour hike throughout the city, Golden Gate Park and dipping toes into the Pacific.
      • Seeing Ubuntu grow from that crazy idea into one of the main pillars of the free software world
      • ITZ GTK BUG!
      • Getting really excited when Milbank and the Canonical office appeared in the Harry Potter movie
      • Moving between and getting to know many different teams from the inside (security, desktop, OEM, QA, CI, Foundations, Release, SRU, Tech Board, …) to appreciate and understand the value of different perspectives
      • Breaking burning wood boards, making great and silly videos, and team games in the forest (that was La Mola) at various All Hands

      But all good things must come to an end — after tossing and turning this idea for a long time, I will leave Canonical at the end of the year. One major reason for me leaving is that after that long time I am simply in need for a “reboot”: I’ve piled up so many little and large things that I can hardly spend one day on developing something new without hopelessly falling behind in responding to pings about fixing low-level stuff, debugging weird things, handholding infrastructure, explaining how things (should) work, do urgent archive/SRU/maintenance tasks, and whatnot (“it’s related to boot, it probably has systemd in the name, let’s hand it to pitti”). I’ve repeatedly tried to rid myself of some of those or at least find someone else to share the load with, but it’s too sticky :-/ So I spent the last few weeks with finishing some lose ends and handing over some of my main responsibilities.

      Today is my last day at work, which I spend mostly on unsubscribing from package bugs, leaving Launchpad teams, and catching up with emails and bugs, i. e. “clean up my office desk”. From tomorrow on I’ll enjoy some longer EOY holidays, before starting my new job in January.

      I got offered to work on Cockpit, on the product itself and its ties into the Linux plumbing stack (storaged/udisks, systemd, and the like). So from next year on I’ll change my Hat to become Red instead of orange. I’m curious to seeing for myself how that other side of the fence looks like!

      This won’t be a personal good-bye. I will continue to see a lot of you Ubuntu folks on FOSDEMs, debconfs, Plumber’s, or on IRC. But certainly much less often, and that’s the part that I regret most — many of you have become close friends, and Canonical feels much more like a family than a company. So, thanks to all lof you for being on that journey with me, and of course a special and big Thank You to Mark Shuttleworth for coming up with this great Ubuntu vision and making all of this possible!

      Read more
      deviceguy

      Like it says on the Intel IoT developer site, "Without sensors, there's no IoT".

      Because I am the maintainer of QtSensors, I like to inquire about  people's use of sensors and if they use QtSensors. Over the years, I have heard quite often something like, 'Qt is thought of as a UI framework'. *sigh*
       
      But Qt is more than just a UI framework and it's use is not dependent on widgets or declarative wizardry. It is used in quite a few middleware components without UI elements. One of those middleware frameworks is Sensor Framework.

      Sensor framework is a daemon that uses a plugin system written using Qt for reading various sensors such as accelerometer or light sensors. It was originally developed by Nokia for Harmattan and ran on the N9. It was also used in MeeGo and later included in the Mer Project and on Jolla phones and the ill fated tablet. So it has been released onto a few commercial products.

      We looked at it when I was working at Nokia on the project that I still cannot name, but we had decided we would come up with our own solution. Looking back, this was the wrong decision, we should have taken the already proven sensor framework and ran with that. Why? Because it existed and works.

      I started maintaining it when I was a privateer (contractor) developer for Jolla. No one else had touched it for some time so I grabbed the few not yet merged bug fixes and added support for libhybris/android libhardware adaptors.

      Sensor Framework has support for multiple clients with down sampling for different data rates. It uses dbus for control lines (to start and stop, etc) but sends data through a socket. It also has a working backend in QtSensors.

      I noticed that Ubuntu's Unity does nothing to respond when I put this into "tablet mode". I have to manually open the virtual keyboard among other things.

      So I thought I could use sensorfw on my Dell 2 in 1. It's one of those converged laptop/tablet devices. It has a few sensors - accelerometer, gyroscope, magnetometer, and lid sensors. One problem... sensorfw does not support lid sensors, or a few other sensors that are around today in IoT (which I will add a bit later). Lid "sensor" might be a bit of a misnomer, as they could be switches but I'd like to think it is more like a hal effect sensor that uses magnets. In any case there are event nodes to use.

      First one I chose is to add the lid sensor - to detect when this machine is put into tablet mode, so the UI can better deal with it.

      I also noticed that this kernel has support for iio sensor interface for the accel and gyro. Sensorfw only supports sysfs, evdev and hybris interfaces, so I also wanted to add support for that.

      I worked on adding iio support first. Well... really just wrote a sensor adaptor plugin. My plugin supports accelerometer, gyroscope and magnetometer, which this device seems to have. I will expand this to support other sensors later, as well as clean it up a bit.

      Thanks to QtSensors sensor framework backend, I can make a UI app change with the orientation and lid changes. Better yet, I can create a game that uses accelerometer data like a marble maze game. Or I can upload the data to one of those Node.js data visualization web apps.

      And since sensor framework is opensource, others can as well.


      Read more
      Leo Arias

      All Linux distributions are constantly updating the versions of the packages in their archives. That’s what makes them great, lots of people working in a distributed way to let you easily update your software and get the latest features or critical bug fixes.

      And you should constatly update your operating system. Otherwise you’ll become an easy target for criminals exploiting known vulnerabilities.

      The problem, at least for me, is that I have many many Ubuntu machines in the house and my badwidth is really bad. So keeping all my real machines, virtual machines and various devices up-to-date every day has become a slow problem.

      The solution is to cache the downloaded deb packages. So only one machine has to make the downloads from the internet, and they will be kept in my local network to make much faster to get the packages in the other machines.

      So let me introduce you to Apt-Cacher NG.

      Setting it up is simple. First, choose a machine to run the cacher and store the packages. Ideally, this machine should be running all the time, and should have a good amount of storage space. I’m using my desktop as the cacher; but as soon as I update my router to one that runs Ubuntu, I will make that one the cacher.

      On that machine, install apt-cacher-ng:

        sudo apt install apt-cacher-ng

      And that’s it. The cacher is installed and configured. Now we need the name of this machine to use it on the other ones:

        $ hostname
        calchas

      In this example, calchas is the name of the machine I’m using as the cacher. Take note of the name of your machine, and now, in all the other machines:

        $ sudo gedit /etc/apt/apt.conf.d/02proxy

      That will create a new empty configuration file for apt, and open it to be edited with gedit, the default graphical editor in Ubuntu. In the editor, write this:

        Acquire::http::proxy "http://calchas.lan:3142";

      replacing calchas with the name of your cacher machine, collected above. The .lan part is really only needed when you are setting it up in a virtual machine and the host is the same as the cacher, but it doesn’t hurt to add it on real machines. That number, 3142, is the network port where the caching service is running, leave it unchanged.

      After that, the first time you update a package in your network it will be slow just as before. But all the other machines updating the same package will be very fast. I have to thank apt-cacher-ng for saving me many hours during my updates of the past years.

      Read more
      ssweeny

      My team at work has been focused on snaps this year and one thing we’ve tried to do internally is establish a set of best practices for snap packaging software. Toward that end I’ve been working on a little tool I’m calling snaplint to encode those practices and verify that we’re following them.

      Right now you can run snaplint against your snapcraft project directory
      and it will scan the prime subdirectory for the following things:

      • copyright (basically that you included usr/share/doc/*copyright*) for
        any stage-packages
      • developer cruft (things like header and object files or static libs
        that might have made their way into your snap)
      • libraries (examine the ELF files in your snap and look for libraries
        which aren’t used)

      The next things I’m planning on adding are:

      • checking for copyright info from apps/parts themselves.
      • checking for mixing of incompatible licenses

      I would love to hear suggestions on further improvements.

      You can find the source at https://github.com/ssweeny/snaplint

      And, of course if you’re running Ubuntu 16.04 or later you can try it on your own machine with:
      $ snap install snaplint
      $ snaplint path/to/your/project

      Read more
      Dustin Kirkland


      From Linux kernel livepatches to encryption to ASLR to compiler optimizations and configuration hardening, we strive to ensure that Ubuntu 16.04 LTS is the most secure Linux distribution out of the box.

      These slides try to briefly explain:

      • what we do to secure Ubuntu
      • how the underlying technology works
      • when the features took effect in Ubuntu

      I hope you find this slide deck informative and useful!  The information herein is largely collected from the Ubuntu Security Features wiki page, where you can always find up to date information.



      Cheers,
      Dustin

      Read more
      Leo Arias

      We have survived two testing days, and now we can safely say that it will become a Friday tradition :)

      Last Friday our nice guest was Aaron Ogle, from Rocket Chat. He gave us a tour on the Rocket Chat UI and we discussed about how they packaged it as a snap.

      If you missed it, click the image below to watch it.

      Alt text

      Building from what we saw on the first session, we tested the snap using a virtual machine again. But this time, we cloned it to keep a pristine machine and make following testing sessions faster. If you want to help the Ubuntu and Rocket Chat communities, this is an easy way to prepare your environment:

      Once you have your clone ready, install the most recent and bleeding edge version of rocket chat with:

      $ sudo snap install rocketchat-server --edge
      

      Then you can follow this gist with the initial steps to start testing the Rocket Chat snap

      Also you can test a real installation of Rocket Chat, joining our community channel, where we are available all day, every day. If you have a question, just ask. I am elopio in there.

      During the session we took a look at the GitHub website, where many free software communities do their development in the open. They have a great guide to start contributing to open source projects. Go on and spread your love for free software in the form of bug reports :)

      The gratitude this week goes to our newly acquired staff members Julia and Kyle, and of course to Aaron for letting us have a funny Friday evening. Make sure to take a look at the cool things he and his teammates are doing; and if you have some free time and want to join an exciting, open and nice community, give them a hand. Also try the Jitsi integration for video conference, it's mind-blowing that there are no closed components anywhere.

      See you next Friday at Ubuntu On-Air.

      Read more