Canonical Voices

Posts tagged with 'html5'

Dustin Kirkland

Transcoding video is a very resource intensive process.

It can take many minutes to process a small, 30-second clip, or even hours to process a full movie.  There are numerous, excellent, open source video transcoding and processing tools freely available in Ubuntu, including libav-toolsffmpegmencoder, and handbrake.  Surprisingly, however, none of those support parallel computing easily or out of the box.  And disappointingly, I couldn't find any MPI support readily available either.

I happened to have an Orange Box for a few days recently, so I decided to tackle the problem myself, and develop a scalable, parallel video transcoding solution myself.  I'm delighted to share the result with you today!

When it comes to commercial video production, it can take thousands of machines, hundreds of compute hours to render a full movie.  I had the distinct privilege some time ago to visit WETA Digital in Wellington, New Zealand and tour the render farm that processed The Lord of the Rings triology, Avatar, and The Hobbit, etc.  And just a few weeks ago, I visited another quite visionary, cloud savvy digital film processing firm in Hollywood, called Digital Film Tree.

Windows and Mac OS may be the first platforms that come to mind, when you think about front end video production, Linux is far more widely used for batch video processing, and with Ubuntu, in particular, being extensively at both WETA Digital and Digital Film Tree, among others.

While I could have worked with any of a number of tools, I settled on avconv (the successor(?) of ffmpeg), as it was the first one that I got working well on my laptop, before scaling it out to the cluster.

I designed an approach on my whiteboard, in fact quite similar to some work I did parallelizing and scaling the john-the-ripper password quality checker.

At a high level, the algorithm looks like this:
  1. Create a shared network filesystem, simultaneously readable and writable by all nodes
  2. Have the master node split the work into even sized chunks for each worker
  3. Have each worker process their segment of the video, and raise a flag when done
  4. Have the master node wait for each of the all-done flags, and then concatenate the result
And that's exactly what I implemented that in a new transcode charm and transcode-cluster bundle.  It provides linear scalability and performance improvements, as you add additional units to the cluster.  A transcode job that takes 24 minutes on a single node, is down to 3 minutes on 8 worker nodes in the Orange Box, using Juju and MAAS against physical hardware nodes.

For the curious, the real magic is in the config-changed hook, which has decent inline documentation.

The trick, for anyone who might make their way into this by way of various StackExchange questions and (incorrect) answers, is in the command that splits up the original video (around line 54):

avconv -ss $start_time -i $filename -t $length -s $size -vcodec libx264 -acodec aac -bsf:v h264_mp4toannexb -f mpegts -strict experimental -y ${filename}.part${current_node}.ts

And the one that puts it back together (around line 72):

avconv -i concat:"$concat" -c copy -bsf:a aac_adtstoasc -y ${filename}_${size}_x264_aac.${format}

I found this post and this documentation particularly helpful in understanding and solving the problem.

In any case, once deployed, my cluster bundle looks like this.  8 units of transcoders, all connected to a shared filesystem, and performance monitoring too.

I was able to leverage the shared-fs relation provided by the nfs charm, as well as the ganglia charm to monitor the utilization of the cluster.  You can see the spikes in the cpu, disk, and network in the graphs below, during the course of a transcode job.

For my testing, I downloaded the movie Code Rushfreely available under the CC-BY-NC-SA 3.0 license.  If you haven't seen it, it's an excellent documentary about the open source software around Netscape/Mozilla/Firefox and the dotcom bubble of the late 1990s.

Oddly enough, the stock, 746MB high quality MP4 video doesn't play in Firefox, since it's an mpeg4 stream, rather than H264.  Fail.  (Yes, of course I could have used mplayer, vlc, etc., that's not the point ;-)

Perhaps one of the most useful, intriguing features of HTML5 is it's support for embedding multimedia, video, and sound into webpages.  HTML5 even supports multiple video formats.  Sounds nice, right?  If it only were that simple...  As it turns out, different browsers have, and lack support for the different formats.  While there is no one format to rule them all, MP4 is supported by the majority of browsers, including the two that I use (Chromium and Firefox).  This matrix from illustrates the mess.

The file format, however, is only half of the story.  The audio and video contents within the file also have to be encoded and compressed with very specific codecs, in order to work properly within the browsers.  For MP4, the video has to be encoded with H264, and the audio with AAC.

Among the various brands of phones, webcams, digital cameras, etc., the output format and codecs are seriously all over the map.  If you've ever wondered what's happening, when you upload a video to YouTube or Facebook, and it's a while before it's ready to be viewed, it's being transcoded and scaled in the background. 

In any case, I find it quite useful to transcode my videos to MP4/H264/AAC format.  And for that, a scalable, parallel computing approach to video processing would be quite helpful.

During the course of the 3 minute run, I liked watching the avconv log files of all of the nodes, using Byobu and Tmux in a tiled split screen format, like this:

Also, the transcode charm installs an Apache2 webserver on each node, so you can expose the service and point a browser to any of the nodes, where you can find the input, output, and intermediary data files, as well as the logs and DONE flags.

Once the job completes, I can simply click on the output file, Code_Rush.mp4_1280x720_x264_aac.mp4, and see that it's now perfectly viewable in the browser!

In case you're curious, I have verified the same charm with a couple of other OGG, AVI, MPEG, and MOV input files, too.

Beyond transcoding the format and codecs, I have also added configuration support within the charm itself to scale the video frame size, too.  This is useful to take a larger video, and scale it down to a more appropriate size, perhaps for a phone or tablet.  Again, this resource intensive procedure perfectly benefits from additional compute units.

File format, audio/video codec, and frame size changes are hardly the extent of video transcoding workloads.  There are hundreds of options and thousands of combinations, as the manpages of avconv and mencoder attest.  All of my scripts and configurations are free software, open source.  Your contributions and extensions are certainly welcome!

In the mean time, I hope you'll take a look at this charm and consider using it, if you have the need to scale up your own video transcoding ;-)


Read more
Kyle Nitzsche

Noting that this week I delivered three hour long sessions of interest to those interested in writing HTML5 apps for Ubuntu:

Read more
Kyle Nitzsche

Cordova 3.3 adds Ubuntu

Upstream Cordova 3.3.0 is released just in time for the holidays with a gift we can all appreciate: built-in Ubuntu support!

Cordova: multi-platform HTML5 apps

Apache Cordova is a framework for HTML5 app development that simplifies building and distributing HTML5 apps across multiple platforms, like Android and iOS. With Cordova 3.3.0, Ubuntu is an official platform!

The cool idea Cordova starts with is a single www/ app source directory tree that is built to different platforms for distribution. Behind the scenes, the app is built as needed for each target platform. You can develop your HTML5 app once and build it for many mobile platforms, with a single command.

With Cordova 3.3.0, one simply adds the Ubuntu platform, builds the app, and runs the Ubuntu app. This is done for Ubuntu with the same Cordova commands as for other platforms. Yes, it is as simple as:

$ cordova create myapp REVERSEDOMAINNAME.myapp myapp
$ cd myapp
(Optionally modify www/*)
$ cordova build [ ubuntu ]
$ cordova run ubuntu


Cordova is a lot more than an HTML5 cross-platform web framework though.
It provides JavaScript APIs that enable HTML5 apps to use of platform specific back-end code to access a common set of devices and capabilities. For example, you can access device Events (battery status, physical button clicks, and etc.), Gelocation, and a lot more. This is the Cordova "plugin" feature.

You can add Cordova standard plugins to an app easily with commands like this:

$ cordova plugin add org.apache.cordova.battery-status
(Optionally modify www/* to listen to the batterystatus event )
$ cordova build [ ubuntu ]
$ cordova run ubuntu

Keep an eye out for news about how Ubuntu click package cross compilation capabilities will soon weave together with Cordova to enable deployment of plugins that are compiled to specified target architecture, like the armhf architecture used in Ubuntu touch images (for phones, tablets and etc.).


As a side note, I'm happy to note that my documentation of initial Ubuntu platform support has landed and has been published at Cordova 3.3.0 docs.

Read more
Kyle Nitzsche

Ubuntu HTML5 API docs

HTML5 API docs published

I'm pleased to note that the Ubuntu HTML5 API docs I wrote are now done and published on These cover the complete set of JavaScript objects that are involved in the UbuntuUI framework for HTML5 apps (at this time). For each object, the docs show how the corresponding HTML is declared and, of course, all public methods are documented.

A couple notes:
  • I wrote an html5APIexerciser app that implements every available public method in the framework. This was helpful to ensure that what I wrote matched reality ;) It may be useful to folks exploring development of  Ubuntu HTML5 apps. The app can be run directly in a browser by opening its index.html, but it is also an Ubuntu SDK project, so it can be opened and run from the Ubuntu SDK, locally and on an attached device.
  • The html5APIexerciser app does not demonstrate the full set of Ubuntu CSS styles available. For example, the styles provide gorgeous toggle buttons and progress spinnners, but since they have no JavaScript objects and methods they are not included in the API docs. So be sure to explore the Gallery by installing the ubuntu-html5-theme-examples package and then checking out /usr/share/ubuntu-html5-theme/0.1/examples/
  • I decided to use yuidoc as the framework for adding source code comments as the basis for auto generated web docs.  After you install yuidoc using npm you can build the docs from source as follows:
  1. Get the ubuntu-html5-theme branch: bzr branch lp:ubuntu-html5-theme
  2. Move to the JavaScript directory: cd ubuntu-html5-theme/0.1/ambiance/js/
  3. Build the docs: yuidoc -c yuidoc.json . This creates the ./build directory.
  4. Launch the docs by opening build/index.html in your browser. They should look something like this 
Thanks to +Adnane Belmadiaf for some theme work and his always helpful consultation, to +Daniel Beck for his initial writeup of the Ubuntu HTML5 framework, and of course to the team for their always awesome work!

Read more

After experimenting recently with the YUI 3.5.0 Application framework, I wanted to take a bit of time to see what other HTML5 app development setups were offering while answering the question: “How can I make HTML5 app development more fun on Ubuntu” – and perhaps bring some of this back to my YUI 3.5 setup.

I’m quite happy with the initial result – here’s a brief (3 minute) video overview highlighting:

  • Tests running not only in a browser but also automatically on file save without even a headless browser (with pretty Ubuntu notifications)
  • Modular code in separate files (including html templates, via requirejs and its plugins)



Things that I really like about this setup:

  • requirejs - YUI-like module definitions and dependency specification makes for very clear code. It’s an implementation of the Asynchronous Module Definition “standard” which allows me to require dependencies on my own terms, like this:
    require(["underscore", "backbone"], function(_, Backbone) {
        // In here the underscore and backbone modules are loaded and
        // assigned to _ and Backbone respectively.

    There’s some indication that YUI may also implement AMD in its loader also. RequireJS also has a built in optimiser to combine and minify all your required JS during your build step. With two plugins for RequireJS I can also use CoffeeScript instead of Javascript, and load my separate HTML templates as resources into my modules (no more stuffing them all into your index.html.

  • mocha tests running on nodejs or in the browser – as shown in the above screencast. Once configured, this made it pretty trivial to add a `make watch` command to my project which runs tests automatically (using nodejs’ V8 engine) when files change, displaying the results using Ubuntu’s built-in notification system. (Mocha already has built in growl support for Mac users, it’d be great to get similar OSD notifications built in too).

The setup wasn’t without its difficulties [1], but the effort was worth it as now I have a fun environment to start building my dream app (we’ve all got one right?) and continue learning. I think it should also be possible for me to go back and re-create this nodejs dev environment using YUI also – which I’m keen to try if someone hasn’t already done something similar – or even possibly without needing nodejs? I think the challenge for YUI will be if and when most other modules can be loaded via AMD why, as an app developer, would I want to commit to one monolithic framework release when I can cleanly pick-n-chose the specific versions of small tightly-focused modules that I need (assuming my tests pass). Or perhaps YUI will join in and begin versioning modules (and module dependencies) rather than the one complete framework so that they were available via any AMD loader – that would rock!

Thanks to James Burke (author of RequireJS) and the brunch team for their help!

[1] For those interested, things that were difficult getting this setup were:

  • Many JS libraries are not yet AMD ready (or yet giving support), which means adding shims to load them correctly (or using the use plugin in some cases). And sometimes this gets complicated (as it did for me with expect.js). I don’t know if AMD will get widespread adoption, who knows? A result of this is that many JS libraries are designed to work within the browser or node only (ie. they assume that either window or module/exports will be available globally).
  • Using the coffeescript plugin is great for looking at the code, but the errors displayed when running tests that result from  coffeescript parse errors are often hard to decipher (although I could probably use an editor plugin to check the code on save and highlight errors).
  • A recent nodejs version isn’t yet available in the ubuntu archives for Precise. It wasn’t difficult, but I had to install Node 0.6.12 to my home directory and put its bin directory on my path before I could get started.

If you want to try it out or look at the code, just make sure NodeJS 0.6.12 is available on your path and do:

tmp$ bzr branch lp:~michael.nelson/open-goal-tracker/backbone-test/
Branched 42 revisions. 
tmp$ cd backbone-test/
backbone-test$ make
backbone-test$ make test
? 12 tests complete (47ms)

Filed under: javascript, open-goal-tracker, ubuntu

Read more