On Monday this week, I was afforded the distinct privilege to deliver the opening keynote at the OpenZFS Developer Summit in San Francisco. It was a beautiful little event, with a full day of informative presentations and lots of networking during lunch and breaks.
Hopefully you'll enjoy the demo -- especially the most interesting raw tracing system new in the Ubuntu 16.04 LTS Linux 4.4 kernel, something called The Berkeley Packet Filter, or "BPF" for short. I used a series of open source utilities from Brendan Gregg (from Netflix), called iovisor/bcc. Quoting the README.md on Github:
BCC is a toolkit for creating efficient kernel tracing and manipulation programs, and includes several useful tools and examples. It makes use of extended BPF (Berkeley Packet Filters), formally known as eBPF, a new feature that was first added to Linux 3.15. Much of what BCC uses requires Linux 4.1 and above.
I'll follow up this post with another one, formally introducing BPF and how to install and use bcc in Ubuntu 16.04 LTS, if anyone is interested...
Ok, ok.. sorry for the click-bait headline – but It is mainly true.. I recently got a Nextcloud box , it was pretty easy to set up and here are some great instructions.
But this box is not just a Nextcloud box, it is a box of unlimited possibilities. In just a few hours I added to my personal cloud a WIFI access point and chat server. So here are some amazing facts you should know about Ubuntu and snaps:
Amazing fact #1 – One box, many apps
With snaps you can transform you single function device, into a box of tricks. You can add software to extend its functionality after you have made it. In this case I created an WIFI access point and added a Rocketchat server to it.
You can release a drone without autonomous capabilities, and once you are sure that you have nailed, you can publish a new app for it… or even sale a pro-version autopilot snap.
You can add an inexpensive Zigbee and Bluetooth module to your home router, and partner with a security firm to provide home surveillance services.. The possibilities are endless.
Amazing fact #2 – Many boxes, One heart
Maybe an infinite box of tricks is attractive to a geek like me, but what it is interesting is product makers is :make one hardware, ship many products.
Compute parts (cpu,memory,storage) make a large part of bill of materials of any smart device. So does validation and integration of this components with your software base… and then you need to provide updates for the OS and the kernel for years to come.
What if I told you could build (or buy) a single multi-function core – pre-integrated with a Linux OS and use it to make drones, home routers, digital advertisement signs, industrial and home automation hubs, base stations, DSLAMs, top-of-rack switches,…
This is the real power of Ubuntu Core, with the OS and kernel being their own snaps – you can be sure the nothing has changes in them across these devices, and that you can reliably update of them. You not only are able to share validation and maintenance cost across multiple projects, you would be able to increase the volume of your part order and get a better price.
Hey just having more fun snapping on dragonboard. I’ve updated the mir-client snap to use a Qt demo that is probably a bit more like what a kiosk style application might be. It’s the photoviewer on dragonboard as an example. Which improved not only the demo experience but provides developers a better guide since it actually uses the qmake plugin of snapcraft to build the demo from source. I can’t emphasize how easy it was to modify my snapcraft project to add this to the demo. Again, good ‘ol mir-snaps wiki can be used as a guide. And if you don’t want to build, you can grab my personal builds of these snaps for arm64 for mir-server snap and mir-client-snap respectively.
So September, 28th 2016 is the 6th birthday of LibreOffice and at the recent conference, we took a picture of those who were there on day zero:
As you might notice, I am not on that picture — on day zero I was working at Oracle, and were surprised by the news — like many others in this picture:
This is everyone at this years LibreOffice conference who used to work on the OpenOffice.org codebase at StarDivision, Sun or Oracle. A few people are in both pictures: Caolán McNamara and Thorsten Behrens were with LibreOffice from the start, but also worked at the OpenOffice.org team in Hamburg at some point in time. Of those working on OpenOffice.org still when LibreOffice started, I was the first to join LibreOffice — I had quit my job for that. It was an exciting time.
Looking back, both of these groups appear small — mostly because merging them, the LibreOffice community became so much more that the sum of its parts:
And of course, while a lot of people were at the conference, not everyone could
join, so there are more contributors from each of these groups than are in the
pictures. This years “state of the project” presentation showed again that the members of The Document Foundation are a truly worldwide community:
Whether I’m adding dependencies, updating package names, or creating new package spins, I always have issues testing my debian packages. Something will work locally, only to fail on jenkins under a clean environment. Fortunately, there’s a nifty tool called pbuilder that exists to help out in these situations. pbuilder uses a chroot to set up a clean environment to build packages, and can even be used to build packages for systems with architectures different from your own.
Note: All code samples were originally written from a machine running Ubuntu 16.10 64-bit. Your mileage may vary.
Clean builds for current distro
Given a typical debian-packaged project with a debian directory (control, rules, .install), you can use debuild to build a package from your local environment:
$ cd my-project
$ ls ../*.deb
This works pretty well for sanity checks, but sometimes knowing your sane just isn’t quite enough. My development environment is filled with libraries and files installed in all kinds of weird ways and in all kinds of strange places, so there’s a good chance packages built successfully on my machine may not work on everyone’s machine. To solve this, I can install pbuilder and set up my first chroot:
$ # install pbuilder and its dependencies$ sudo apt-get install pbuilder debootstrap devscripts
$ # create a chroot for your current distro with build-essential pre-installed$ sudo pbuilder create --debootstrapopts --variant=buildd
Since I use debuild pretty frequently, I also rely on pdebuild which performs debuild inside of the clean chroot environment, temporarily installing the needed dependencies listed in the control file.
$ cd my-project
$ ls /var/cache/pbuilder/result/*.deb
Alternatively, I could create the .dsc file and then use pbuilder to create the package from there:
$ # generate a dsc file however you like$ cd my-project
$ bzr-builddeb -- -us -uc
$ cd ..
$ # use pbuilder to create package$ sudo pbuilder build my-project.dsc
$ ls /var/cache/pbuilder/result/*.deb
Clean cross builds
Let’s say that you need to build for an older distribution of Ubuntu on a weird architecture. For this example, let’s say vivid with armhf. We can use pbuilder-dist to verify and build our packages for other distros and architectures:
$ # create the chroot, once again with build-essential pre-installed$ pbuilder-dist vivid armhf create --debootstrapopts --variant=buildd
$ # the above command could take a while, but once it's finished$ # we can attempt to build our package using a .dsc file$ pbuilder-dist vivid armhf build my-project-dsc
$ ls ~/pbuilder/vivid-armhf_result/*.deb
Custom, persistent chroot changes
In some cases, you may need to enable other archives or install custom software in your chroot. In the case of our vivid-armhf chroot, let’s add the stable-overlay ppa which updates the outdated vivid with some more modern versions of packages.
$ # login to our vivid-armhf chroot, and save state when we're finished$ # if --save-after-login is omitted, a throwaway chroot will be used$ pbuilder vivid armhf login --save-after-login
(chroot)$ # install the package container add-apt-repository for convenience(chroot)$ apt install software-properties-common
(chroot)$ add-apt-repository ppa:ci-train-ppa-service/stable-phone-overlay
(chroot)$ exit$ # update packages in the chroot$ pbuilder-dist vivid armhf update
pbuilder and chroots are powerful tools in the world of packaging and beyond. There are scripting utilities, as well as pre- and post-build hooks which can customize your builds. There are ways to speed up clean builds using local caches or other “cheats”. You could use the throwaway terminal abilities to create and destroy tiny worlds as you please. All of this is very similar to the utility which comes from using docker and lxc, though the underlying “container” is quite a bit different. Using pbuilder seems to have a much lower threshold for setup, so I prefer it over docker for clean build environments, but I believe docker/lxc to be the better tool for managing the creation of consistent virtual environments.
KDE Neon developer Harald Sitter was able to package up the KDE calculator, kcalc, in a snap that weighs in at a mere 320KB! How did he do it?
Like most applications in KDE, kcalc depends on several KDE Frameworks (though not all), sets of libraries and services that provide the common functionality and shared UI/UX found in KDE and it’s suite of applications. This means that, while kcalc is itself a small application, it’s dependency chain is not. In the past, any KDE application snap had to include many megabytes of platforms dependencies, even for the smallest app.
Recently I introduced the new “content” interface that has been added to snapd. I used this interface to share plugin code with a text editor, but Harald has taken it even further and created a KDE Frameworks snap that can share the entire platform with applications that are built on it!
While still in the very early stages of development, this approach will allow the KDE project to deliver all of their applications as independent snaps, while still letting them all share the one common set of Frameworks that they depend on. The end result will be that you, the user, will get the very latest stable (or development!) version of the KDE platform and applications, direct from KDE themselves, even if you’re on a stable/LTS release of your distro.
If you are running a snap-capable distro, you can try these experimental packages yourself by downloading kde-frameworks-5_5.26_amd64.snap and kcalc_0_amd64.snap from Neon’s build servers, and installing them with “snap install –devmode –force-dangerous <snap_file>”. To learn more about how he did this, and to help him build more KDE application snaps, you can find Harald as <sitter> on #kde-neon on Freenode IRC.
Working with a new technology often brings you to see things in a new light and re-think previous habits. Especially when it challenges the status quo and expectations of years of traditional use. Snaps are no exception in this regard. As one example twenty years ago we simply didn’t have today’s confinement technologies.
Luckily is using snapcraft a real joy: you write one declarative file, define your snap’s parts, make use of snapcraft‘s many plugins and if really necessary, you write a quick and simple plugin using Python to run your custom build.
Many of the first issues new snaps ran into were solved by improvements and new features in snapd and snapcraft. If you are still seeing a problem with your snap, we want you to get in touch. We are all interested in seeing more software as snaps, so let’s work together on them!
Enter the Sandpit
I mentioned it in my last announcement of the last Snappy Playpen event already, but as we saw many new snaps being added there in the last days, I wanted to mention it again. We started a new initiative called the Sandpit.
It’s a place where you can easily
list a snap you are working on and are looking for some help
find out at a glance if your favourite piece of software is already being snapped
It’s a very light-weight process: simply edit a wiki and get in touch with whoever’s working on the snap. The list grew quite quickly, so there’s loads of opportunities to find like-minded snap authors and get snaps online together.
You can find many of the people listed on the Sandpit wiki either in #snappy on Freenode or on Gitter. Just ask around and somebody will help.
I always felt that learning something new, especially new concepts and workflows usually works best if you see it first-hand and get to do things yourself. If you experience directly how your actions influence the system you're working with, the new connections in your brain form much more quickly. Didier and I talked a while about how to introduce the processes and ideas behind snapd and snapcraft to a new audience, particularly at a workshop or a meet-up and we found we were of the same opinion.
Didier put quite a bit of work into solving the infrastructure question. We re-used the work which was put into Codelabs already, so adding a new codelab merely became a question of creating a Google Doc and adding it using a management command. It works nicely, the UI is simple and easy to understand and lets you focus on the content at hand. It was a lot of fun to work on the content and refine the individual steps in a self-teaching workshop style. Thanks a lot everyone for the reviews!
It's now available for everyone
After some discussion it became clear that a very fitting way for the codelabs to go out would be to ship them as a snap themselves. It's beautifully simple to get started:
$ sudo snap install snap-codelabs
All you need to do afterwards is point your browser to http://localhost:8123/ - that's all. You will be greeted with something like this:
From thereon you can quickly start your snap adventure and get up and running in no time. It's a step-by-step workshop and you always know how much more time you need to complete it.
Expect more codelabs to be added soon. If you have feedback, please let us know here.
Have fun and when you're done with your first codelab, let us know in the comments!
OK, I’m really overdue on posting something about this as I’ve had _something_ running on the dragonboard 410c for a while. If you don’t know about dragonboard you can check out dragonboard from 96boards .
So dragonboard is targeted to be a supported reference board by our Snappy team and they’re in the process of pushing out beta images to play with in the 16 series. I had been concerned that we I was going to have to go and build the graphics drivers into our Ubuntu core snap. When I started I wasn’t even sure of the state of the freedreno drivers vs closed source vendor drivers. But as luck would have it, someone quite recently had turned on the gallium drivers to be built and package as part of the Ubuntu distro, which means I got the freedreno drivers with no effort! Lots of love to Rob Clark for all the work he’s done on freedreno (if your interested in learning more check out freedreno on github ).
So getting a devmode mir snap demo up and running was relatively painless. However, I do want to say I found a little difference in my runs amd64 VM vs the native arm64. This resulted in some tweaks to the mir interface in snapd (which had already landed and should be in the next snapd release). Also, never use setterm when developing with Mir, that create all sorts of chaos for me I had used setterm for convenience to prevent the screen from blanking, ended up causing failures when I was working on making sure the mir snaps could run confined.
If you follow the good ol’ mir snaps wiki, you can easily duplicate this – running the mir snaps fully confined on dragonboard core snap. Also, I wanted to point out again there are other Qt demos you can try besides the clock app – simply modify the helper file in the client example (client-start) to be something besides “clock”, for example “maroon” or “samegame”. You can do this with an HDMI monitor and mouse attached like in this video of various Qt apps as mir-client snaps running on dragonboard . Still need to investigate some mouse oddities that seem to only occur with apps other than clock.
And lastly, I got new toy over the weekend. I ordered a 7″ touch screen from adafruit. Here’s a quick video of the 7″ display attached. I need to tinker with it to see about getting the touch to work, but it was nice to just hook it together and the display come up.
A feature added to the lp:miral trunk yesterday is making life a lot easier for developers working on MirAL based servers, and the toolkit extensions that support Mir. The feature is:
Actually, the –window-management-trace switch will work with any MirAL base server (so miral-kiosk and egmde support it too).
What this does is cause the server to log every interaction with the window management policy – all the notifications it receives and all the calls it makes to the tools as a result.
This means that it is easy to find out that, for example, a “modal” gtk based dialog window is being created without specifying a parent. (Which is why the behaviour isn’t quite as expected – Mir servers will treat it as non-modal.)
To use this feature before the next MirAL release you do need build it yourself, but this only takes a minute (depending on your kit).It really is the easiest way to see exactly what is going on and why.
If you are a member of Launchpad’s beta testers team, you’ll now have a slightly different interface for selecting source packages in the Launchpad web interface, and we’d like to know if it goes wrong for you.
One of our longer-standing bugs has been #42298 (“package picker lists unpublished (invalid) packages”). When selecting a package – for example, when filing a bug against Ubuntu, or if you select “Also affects distribution/package” on a bug – and using the “Choose…” link to pop up a picker widget, the resulting package picker has historically offered all possible source package names (or sometimes all possible source and binary package names) that Launchpad knows about, without much regard for whether they make sense in context. For example, packages that were removed in Ubuntu 5.10, or packages that only exists in Debian, would be offered in search results, and to make matters worse search results were often ordered alphabetically by name rather than by relevance. There was some work on this problem back in 2011 or so, but it suffered from performance problems and was never widely enabled.
We’ve now resurrected that work from 2011, fixed the performance problems, and converted all relevant views to use it. You should now see something like this:
Exact matches on either source or binary package names always come first, and we try to order other matches in a reasonable way as well. The disclosure triangles alongside each package allow you to check for more details before you make a selection.
Please report any bugs you find with this new feature. If all goes well, we’ll enable this for all users soon.
Update: as of 2016-09-22, this feature is enabled for all Launchpad users.
For a few weeks we have been running the Snappy Playpen as a pet/research project already. Many great things have happened since then:
With the Playpen we now have a repository of great best-practice examples.
We brought together a lot of people who are excited about snaps, who worked together, collaborated, wrote plugins together and improved snapcraft and friends.
A number of cloud parts were put together by the team as well.
We landed quite a few high-quality snaps in the store.
We had lots of fun.
Opening the Sandpit
With our next Snappy Playpen event tomorrow, 20th September 2016, we want to extend the scheme. We are opening the Sandpit part of the Playpen!
One thing we realised in the last weeks is that we treated the Playpen more and more like a place where well-working, tested and well-understood snaps go to inspire people who are new to snapping software. What we saw as well was that lots of fellow snappers kept their half-done snaps on their hard-disk instead of sharing them and giving others the chance to finish them or get involved in fixing. Time to change that, time for the Sandpit!
In the Sandpit things can get messy, but you get to explore and play around. It’s fun. Naturally things need to be light-weight, which is why we organise the Sandpit on just a simple wiki page. The way it works is that if you have a half-finished snap, you simply push it to a repo, add your name and the link to the wiki, so others get a chance to take a look and work together with you on it.
Tomorrow, 20th September 2016, we are going to get together again and help each other snapping, clean up old bits, fix things, explain, hang out and have a good time. If you want to join, you’re welcome. We’re on Gitter and on IRC.
As an added bonus, we are going to invite Michael Vogt, one of the core developers of snapd to the Ubuntu Community Q&A tomorrow. Join us at 15:00 UTC tomorrow on http://ubuntuonair.com and ask all the questions you always had!
Es tan fácil, y sale tan bien, que este plato es casi como un comodín.
Es tapa de asado al horno.
La tapa de asado es un corte que está al lado del costillar (arriba, adelante) y es más barato que el asado, el vacío, la paleta, etc, y es 100% carne (no huesos).
No se usa mucho porque como no tiene grasa dentro de la carne, tiende a salir un poco dura. Por otro lado, tiene grasa afuera, que le da un gustito muy rico, pero que al mismo tiempo uno puede sacar totalmente si quiere una preparación bien magra.
La forma de preparar este corte, que recomiendo porque me funciona siempre, es muy sencilla.
Poné papel metalizado en la mesada, y apoyá la tapa de asado arriba. Condimentala de un lado, dala vuelta y condimentala del otro. Yo le pongo sal gruesa, pimienta, y chimichurri deshidratado (directamente, sin hidratarlo previamente), pero podés ponerle lo que quieras: ajo, ají molido, orégano, mostaza (¡queda muy bien!), y muchos etcéteras más; no recomiendo ponerle verduras, porque salen medio "hervidas".
Envolvé la carne condimentada con el papel de aluminio que quedó abajo de la carne, y con otro que le pongas por arriba. Le podés poner más, si te quedaron agujeros: la idea es que no pierda líquido.
Luego, meté la carne envuelta en el horno, en una asadera, y dejala dos o tres horas a fuego bajo. Sí, tarda en cocinarse, pero lo que te lleva a vos son 5 minutos. Es buena idea ponerle un poco de agua a la asadera, para que si se escapan jugos de la carne (¡porque no la envolviste bien!) no se quemen en la asadera y hagan humo o tiren olor.
A mí me gusta acompañar este plato con cebollas al horno. De nuevo, muy fácil, las pelás y cortás en trozos grandes (te podría decir que alrededor de 25-30 cm3, pero quizás es más útil si te digo que las dejes más o menos del tamaño de una nuez grande. Las tirás en una asadera, un poco de aceite por arriba, y listo. Yo las salpimento cuando ya están casi hechas, para que no larguen mucho líquido (de nuevo, condimentar a gusto). Me gusta sacarlas cuando ya se empezaron a quemar, :)
So first, if you didn’t catch it, series 16 Ubuntu Core beta images are available here http://cdimage.ubuntu.com/ubuntu-snappy/16.04/current/
I just verified the Mir snaps are functioning and made some small updates to match here https://developer.ubuntu.com/en/snappy/build-apps/mir-snaps/
One of which being a command line switch for installing the snaps locally… –dangerous, what a great flag name
Also, I made some updates to the snaps themselves to check for the architecture from $SNAP_ARCH and then set up all the correct paths. So this means the scripts will work properly on the various archs without having to tinker. I’ve also changed the mir-client snap specifically to pull the demos from the archive, this way you’ll also get the right binary for the right arch as well.
After being asked about what I like about Click hooks I thought it would be nice to write up a little bit of the why behind them in a blog post. The precursor to this story is that I told Colin Watson that he was wrong to build hooks like this; he kindly corrected me and helped me fix my code to match but I still wasn't convinced. Now today I see some of the wisdom in the Click hook design and I'm happy to share it.
The standard way to think about hooks is as a way to react to the changes to the system. If a new application is installed then the hook gets information about the application and responds to the new data. This is how most libraries work with providing signals about the data that they maintain, and we apply that same logic to thinking about filesystem hooks. But filesystem hooks are different because the coherent state is harder to query. In your library you might respond the signal for a few things, but in many code paths the chances are you'll just go through the list of original objects to do operations. With filesystem hooks that complete state is almost never used, only the caches are that are created by the hooks themselves.
Click hooks work by creating a directory of symbolic links that matches the current state of the system, and then asks you to ensure your cache matches that state of the system. This seems inefficient because you have to determine which parts of your cache need to change, which get removed and which get added. But it results in better software because your software, including your hooks, has errors in it. I'm sorry to be the first one to tell you, but there are bugs. If your software is 99% correct, there is still something it is doing wrong. When you have delta updates that update the cache that error compounds and never gets completely corrected with each update because the complete state is never examined. So slowly the quality of your cache gets worse, not awful, but worse. By transferring the current system state to the cache each time you get the error rate of your software in the cache, but you don't get the compounded error rate of each delta. This adds up.
The design of the hooks system in Click might feel wrong as you start to implement one, but I think that after you create a few hooks you'll find there is wisdom in it. And as you use other hook systems in other platforms think about checking the system state to ensure you're always creating the best cache possible, even if the hook system there didn't force you to do it.