Canonical Voices

Posts tagged with 'article'

David Planella

Another month and another fresh ranking for you. Into the new year and time for recap of the top 10 list of apps downloaded from the Ubuntu Software Centre last month!

Top 10 commercial apps

1 > TRAUMA

TRAUMA is a game that tells a story of a young woman who survives a car accident. Recovering at the hospital, she has dreams that shed light on different aspects of her identity – such as the way she deals with the loss of her parents. TRAUMA lets you experience those dreams in an interactive way, reminiscent of Point-and-Click Adventure Games.

2 > Fluendo DVD Player

Fluendo DVD Player is a software application specially designed to reproduce DVD on Linux/Unix platforms, which provides end users with high quality standards.

3 > Braid

Braid is a platform game in painterly style where you manipulate the flow of time to solve puzzles. Every puzzle in Braid is unique; there is no filler. Braid treats your time and attention as precious, and it does everything it can to give you a mind-expanding experience.

4 > Oil Rush (pre-order, beta access)

Oil Rush is a real-time naval strategy game based on group control. It combines the strategic challenge of a classical RTS with the sheer fun of Tower Defence. Fight the naval war between furious armies across the boundless waters of the post-apocalyptic world.

5 > Steel Storm: Burning Retribution

Steel Storm: Burning Retribution marks the return of top-down shooters with new twists. The game has score oriented competitive gameplay, and is designed for people who like fast paced action, hordes of smart enemies, destructible worlds and ground shaking explosions.

6 > World of Goo

Drag and drop living, squirming, talking globs of goo to build structures, bridges, cannonballs, zeppelins, and giant tongues. The millions of innocent goo balls that live in the beautiful World of Goo are curious to explore. But they don’t know that they are in a game, or that they are extremely delicious. The most addicting and awe-inspiring puzzle game will set you on an adventure that you’ll never forget!

7 > Monster RPG 2

Monster RPG 2 is a fantasy quest that spans continents and worlds and lets you take a simple villager and develop her into a hero with the power to save her world. The next instalment in the classic Monster RPG series, Monster RPG 2 is a turn-based role-playing game with great variety of plot twists, secrets, and scenery.

8 > The Clockwork Man: The Hidden World

Explore a Victorian era filled with wondrous contraptions and fascinating machinery. Embark on a steam-powered, rollercoaster journey through land, air, and water in this unique Hidden Object Adventure! Miranda and Sprocket are back! Join them in their new adventures and discover the Hidden World, a land lost in time and glimpsed through legends.

9 > Uplink

You play an Uplink Agent who makes a living by performing jobs for major corporations. Your tasks involve hacking into rival computer systems, stealing research data, sabotaging other companies, laundering money, erasing evidence, or framing innocent people. You use the money you earn to upgrade your computer systems, and to buy new software and tools. As your experience level increases you find more dangerous and profitable missions become available.

10 > Memory Owl

Memory Owl is a unique, dynamic game that uses physics engine and adds multiple hurdles and helpers to spice up the classic game of finding pairs of pictures. It’s suitable for all ages – even children as young as 4 years will find it amusing (especially on easy diffuculty setting). Find all pairs before they drawn in the rising water or are pushed out of screen by fish.

Top 10 free apps

1 > Ryzom

Ryzom, one of the best role playing Massively Multiplayer Online Game of the moment (MMORPG), is set more than 2000 years in the future, on a living, evolving world: beautiful Atys!

2 > Crossover Games

Play Windows games like World of Warcraft on Ubuntu! CrossOver Games (Ubuntu Edition) makes it possible to play Windows games such as World of Warcraft and many others. CrossOver Games is built on the latest versions of Wine, based on contributions from both CodeWeavers and the open-source Wine community. CrossOver Games aims to bring you the latest, greatest, bleeding edge improvements in Wine technology.

3 > Vendetta Online

Vendetta Online is a 3D space combat MMORPG. This MMO permits thousands of players to interact as the pilots of spaceships in a vast universe. Users may build their characters in any direction they desire, becoming rich captains of industry, military heroes, or outlaws.

4 > CrossOver Pro (Trial)

CrossOver Linux allows you to install many popular Windows productivity applications, plugins and games in Linux. You can think of it as an emulator, but it’s different, because there’s no Windows OS license required. Your applications integrate seamlessly with your GNOME or KDE environment. It’s like running Windows on your Linux machine, but without Windows.

5 > Full Circle Magazine

Full Circle is a free, independent, monthly magazine dedicated to the Ubuntu family of Linux operating systems. Each month, it contains helpful how-to articles and reader submitted stories. Full Circle also features a companion podcast, the Full Circle Podcast, which covers the magazine along with other news of interest.

6 > CrossOver Standard (Trial)

CrossOver Linux allows you to install many popular Windows productivity applications, plugins and games in Linux. You can think of it as an emulator, but it’s different, because there’s no Windows OS license required. Your applications integrate seamlessly with your GNOME or KDE environment. It’s like running Windows on your Linux machine, but without Windows.

7 > Wunderlist

Free cloud-sync task manager, helps sharing your To-Do lists with friends and colleagues. Manage your to-dos and synchronize them with your free Wunderlist account. View and modify your tasks on Windows, Mac, iPad, iPhone/iPod Touch, Android and the Web. Hundreds of thousands of people worldwide use Wunderlist everyday. Wunderlist – your tasks anywhere, anytime.

8 > Tribal Trouble 2

Tribal Trouble 2 is a browser-based RTS game that takes place in the zany age of the Vikings. You are the Chief of a Viking tribe and are responsible for making a name for yourself by conquest and skill.

9 > Magic 8 Ball

Ask the Magic Eight Ball and get an answer to your question. If you’re looking for advice or fancy some fortune-telling, the ball will always listen to you and deliver, though it might not be what you expect!

10 > Clipardo (demo)

This program enables you to find clipart pictures by a hand-drawn sketch. It can be of service when looking for clipart for your presentations, mind maps, posters, charts, web-design, even programs.

Notes:

  • The lists of top 10 app downloads includes only those applications submitted through My Apps on the Ubuntu App Developer Site. For more information about of usage of other applications in the Ubuntu archive, check out the Ubuntu Popularity Contest statistics.
  • The top 10 free apps list contains gratis applications that are distributed under different types of licence, some of which might not be open source. For detailed licence information, please check each application’s description in the Ubuntu Software Centre.

Follow Ubuntu App Development on:

 

Social Media Icons by Paul Robert Lloyd

Read more
David Planella

Support for PayPal as an additional payment option has landed today in Ubuntu Pay, the service that handles electronic payment for commercial apps in the Ubuntu Software Centre.

This new feature will enable Ubuntu users to comfortably and safely buy commercial apps, ebooks and magazines from the Software Centre with their PayPal account. PayPal will be an additional payment option along with the already supported credit and debit card alternatives, “a frequently-requested option by users who don’t have credit/debit cards or who get charged conversion fees by their bank for non-local currency transactions“, adds Stuart Metcalfe, Canonical ISD manager.

All in all, a much welcomed feature that will contribute  reinforcing the virtuous circle of the Ubuntu apps ecosystem with an easier purchasing process, more profits for app authors and increased attractiveness of Ubuntu as a target platform for app developers.

Congratulations to the Canonical ISD team for this great addition to the Software Centre!

Follow Ubuntu App Development on:

 

Social Media Icons by Paul Robert Lloyd

Read more
David Planella

As you will probably have noticed every time you open the Ubuntu Software Centre and watch those beautiful banners, each month we pick some of the best apps available to Ubuntu users and highlight them for everyone to see whenever they search for apps. It is not only a nice way to celebrate and bring more visibility to cool new apps, but also to help app authors by promoting their work.

December brings 3 awesome featured applications. Read on to learn more about them and how to get them on your Ubuntu system.

Stellarium

Stellarium is a beautiful planetarium application, with which you can watch starry nights and identify celestial objects as if you were laying down on a nice field enjoying the clear sky. The first thing that catches the eye are the slick and captivating visuals, including the interface itself, which is very intuitive and blends in very well with the sky cruising experience. Quoting the Stellarium home page, it’s just a matter of setting your location and go.

You’ll enjoy watching and learning constellations, satellites, planets, nebulae… (you name it!) from the impressive catalogue of over 600 thousand stars, which can be extended to more than 210 million stars. You can even turn off the atmosphere and the Earth! Stellarium is open source software. Go and check it out for yourself downloading it from the Software Centre.

TRAUMA

TRAUMA is a game that develops around the story of a woman who survives a car accident and wakes up to confront her recovery. The player gets absorbed into a dream-like atmosphere and is challenged to solve puzzles or achieve a particular end goal, as if part of an interactive movie. This sensation of controlling the direction of the plot, along with the compelling visuals and gameplay make TRAUMA stand out as a fascinating game and story to dive into.

All in all, a unique and engaging experience, again, right at your fingertips from the Software Centre!

Openshot

Openshot is an easy-to-use yet very powerful video editor. From creating family videos to presentations, to professional video editing, Openshot will help you with all your video needs. Easily combine music and video using multiple tracks, cut, trim, resize, mix and match clips, add transitions… the list of features is pretty impressive.

Have you ever wanted to create a promotional video to showcase a project you are involved in? Or add captions and nice transitions to those holiday videos? Now is your chance to download Openshot from the Software Centre and supercharge your Ubuntu installation with some video editing power. Openshot is open source software.

We hope you enjoyed our picks for this month. Stay tuned for a new update on January’s featured apps, and remember to add comments if you want to suggest us an app to highlight. Looking forward to your own picks!

Follow Ubuntu App Development on:

 

Social Media Icons by Paul Robert Lloyd

Read more
David Planella

November is over and we’re moving fast into the Christmas season. Time for a quick recap of the top 10 list of apps downloaded from the Ubuntu Software Centre last month!

Top 10 commercial apps

Braid – Braid is a platform game in painterly style where you manipulate the flow of time to solve puzzles. Every puzzle in Braid is unique; there is no filler. Braid treats your time and attention as precious, and it does everything it can to give you a mind-expanding experience. Get Braid from the Software Centre ›

Oil Rush (pre-order, beta access) – Oil Rush is a real-time naval strategy game based on group control. It combines the strategic challenge of a classical RTS with the sheer fun of Tower Defence. Fight the naval war between furious armies across the boundless waters of the post-apocalyptic world. Get Oil Rush BETA from the Software Centre ›

TRAUMA – TRAUMA is a game that tells a story of a young woman who survives a car accident. Recovering at the hospital, she has dreams that shed light on different aspects of her identity – such as the way she deals with the loss of her parents. TRAUMA lets you experience those dreams in an interactive way, reminiscent of Point-and-Click Adventure Games. Get TRAUMA from the Software Centre ›

Fluendo DVD Player – Fluendo DVD Player is a software application specially designed to reproduce DVD on Linux/Unix platforms, which provides end users with high quality standards. Get Fluendo DVD Player from the Software Centre ›

Steel Storm: Burning Retribution – Steel Storm: Burning Retribution marks the return of top-down shooters with new twists. The game has score oriented competitive gameplay, and is designed for people who like fast paced action, hordes of smart enemies, destructible worlds and ground shaking explosions. Get Steel Storm from the Software Centre ›

Monster RPG 2 – Monster RPG 2 is a fantasy quest that spans continents and worlds and lets you take a simple villager and develop her into a hero with the power to save her world. The next instalment in the classic Monster RPG series, Monster RPG 2 is a turn-based role-playing game with great variety of plot twists, secrets, and scenery. Get Monster RPG 2 from the Software Centre ›

World of Goo – Drag and drop living, squirming, talking globs of goo to build structures, bridges, cannonballs, zeppelins, and giant tongues. The millions of innocent goo balls that live in the beautiful World of Goo are curious to explore. But they don’t know that they are in a game, or that they are extremely delicious. The most addicting and awe-inspiring puzzle game will set you on an adventure that you’ll never forget! Get World of Goo from the Software Centre ›

The Clockwork Man: The Hidden World – Explore a Victorian era filled with wondrous contraptions and fascinating machinery. Embark on a steam-powered, rollercoaster journey through land, air, and water in this unique Hidden Object Adventure! Miranda and Sprocket are back! Join them in their new adventures and discover the Hidden World, a land lost in time and glimpsed through legends. Get The Clockwork Man: The Hidden World from the Software Centre ›

Uplink – You play an Uplink Agent who makes a living by performing jobs for major corporations. Your tasks involve hacking into rival computer systems, stealing research data, sabotaging other companies, laundering money, erasing evidence, or framing innocent people. You use the money you earn to upgrade your computer systems, and to buy new software and tools. As your experience level increases you find more dangerous and profitable missions become available. Get Uplink from the Software Centre ›

Fluendo Complete Playback Pack – This product contains plug-ins which allow you to play certain proprietary audio and video formats. These plug-ins are not included in the default Ubuntu distribution because they are not free software. Ubuntu is driven by strong support for the principles of free and open source software, and these principles govern what we can and will include in Ubuntu. However, we recognise the common need for plug-ins such as these, and offer them here to provide a safe and legal way for our users to play back video and audio in the formats they wish. Get Fluendo Complete Playback Pack from the Software Centre ›

Top 10 free apps

Ryzom – Ryzom, one of the best role playing Massively Multiplayer Online Game of the moment (MMORPG), is set more than 2000 years in the future, on a living, evolving world: beautiful Atys! Get Ryzom from the Software Centre ›

Crossover Games – Play Windows games like World of Warcraft on Ubuntu! CrossOver Games (Ubuntu Edition) makes it possible to play Windows games such as World of Warcraft and many others. CrossOver Games is built on the latest versions of Wine, based on contributions from both CodeWeavers and the open-source Wine community. CrossOver Games aims to bring you the latest, greatest, bleeding edge improvements in Wine technology. Get Crossover Games from the Software Centre ›

Vendetta Online – Vendetta Online is a 3D space combat MMORPG. This MMO permits thousands of players to interact as the pilots of spaceships in a vast universe. Users may build their characters in any direction they desire, becoming rich captains of industry, military heroes, or outlaws. Get Vendetta Online from the Software Centre ›

CrossOver Pro (Trial) – CrossOver Linux allows you to install many popular Windows productivity applications, plugins and games in Linux. You can think of it as an emulator, but it’s different, because there’s no Windows OS license required. Your applications integrate seamlessly with your GNOME or KDE environment. It’s like running Windows on your Linux machine, but without Windows. Get Crossover Pro (Trial) from the Software Centre ›

Full Circle Magazine Issue 52 – Full Circle is a free, independent, monthly magazine dedicated to the Ubuntu family of Linux operating systems. Each month, it contains helpful how-to articles and reader submitted stories. Full Circle also features a companion podcast, the Full Circle Podcast, which covers the magazine along with other news of interest. Get Full Circle Magazine #52 from the Software Centre ›

CrossOver Standard (Trial) – CrossOver Linux allows you to install many popular Windows productivity applications, plugins and games in Linux. You can think of it as an emulator, but it’s different, because there’s no Windows OS license required. Your applications integrate seamlessly with your GNOME or KDE environment. It’s like running Windows on your Linux machine, but without Windows. Get Crossover Standard (Trial) from the Software Centre ›

Full Circle Magazine Issue 53 – Full Circle is a free, independent, monthly magazine dedicated to the Ubuntu family of Linux operating systems. Each month, it contains helpful how-to articles and reader submitted stories. Full Circle also features a companion podcast, the Full Circle Podcast, which covers the magazine along with other news of interest. Get Full Circle Magazine #53 from the Software Centre ›

Full Circle Magazine Issue 50 – Full Circle is a free, independent, monthly magazine dedicated to the Ubuntu family of Linux operating systems. Each month, it contains helpful how-to articles and reader submitted stories. Full Circle also features a companion podcast, the Full Circle Podcast, which covers the magazine along with other news of interest. Get Full Circle Magazine #50 from the Software Centre ›

Wunderlist – Free cloud-sync task manager, helps sharing your To-Do lists with friends and colleagues. Manage your to-dos and synchronize them with your free Wunderlist account. View and modify your tasks on Windows, Mac, iPad, iPhone/iPod Touch, Android and the Web. Hundreds of thousands of people worldwide use Wunderlist everyday. Wunderlist – your tasks anywhere, anytime. Get Wunderlist from the Software Centre ›

Full Circle Magazine Issue 51 – Full Circle is a free, independent, monthly magazine dedicated to the Ubuntu family of Linux operating systems. Each month, it contains helpful how-to articles and reader submitted stories. Full Circle also features a companion podcast, the Full Circle Podcast, which covers the magazine along with other news of interest. Get Full Circle Magazine #51 from the Software Centre ›

This month’s highlights

If you still haven’t seen them, open up the Ubuntu Software Centre and check out the beautiful banners for this month. Our carefully hand-picked app highlights this time round are:

Stellarium – Stellarium renders 3D photo-realistic skies in real-time. With Stellarium, you really see what you can see with your eyes, binoculars or a small telescope. Get Stellarium from the Software Centre ›

Openshot – OpenShot Video Editor is a free, open-source, non-linear video editor. It can create and edit videos and movies using many popular video, audio, and image formats. Create videos for YouTube, Flickr, Vimeo, Metacafe, iPod, Xbox, and many more common formats! Get Openshot from the Software Centre ›

TRAUMA – TRAUMA is a game that tells a story of a young woman who survives a car accident. Recovering at the hospital, she has dreams that shed light on different aspects of her identity – such as the way she deals with the loss of her parents. TRAUMA lets you experience those dreams in an interactive way, reminiscent of Point-and-Click Adventure Games. Get TRAUMA from the Software Centre ›

Notes:

  • The lists of top 10 app downloads includes only those applications submitted through My Apps on the Ubuntu App Developer Site. For more information about of usage of other applications in the Ubuntu archive, check out the Ubuntu Popularity Contest statistics.
  • The top 10 free apps list contains gratis applications that are distributed under different types of licence, some of which might not be open source. For detailed licence information, please check each application’s description in the Ubuntu Software Centre.

Follow Ubuntu App Development on:

 

Social Media Icons by Paul Robert Lloyd

Read more
David Planella

One of the main objectives for the Ubuntu 12.04 cycle is to build upon the foundations set by the Ubuntu App Developer site, My Apps and the Ubuntu Software Centre and start building an Ubuntu App Developer community to realize the vision of a rich ecosystem of apps around Ubuntu. This is the first of a series of posts that will discuss several aspects of this goal, how to get involved, and the benefits of Ubuntu as a target platform for both developers and users.

An important aspect of each community is to ensure that there are easily accessible resources that can act as a venue for communication for anyone wanting to get involved. For the Ubuntu App Developer community, but also in general, the degree of involvement will then vary according to what the individuals connecting to our app developer story are looking for. Some will be seeking help, some will be able to provide help, some will want to contribute to build the developer story, some will want to stay up to date with the news, some will write applications… The first step is to ensure that we cover the main venues, or connecting points to our story for them.

We already started out creating some of these resources ready for the launch of the Ubuntu App Developer last cycle, but we’ve been adding some more recently and I thought at this point it would be a good opportunity to provide an overview of the variety of ways to get involved and stay up to date with App Development in Ubuntu. So without further ado…

Stay up to date

This is a set of channels to follow and share the news and announcements related to Ubuntu App Development.

The Ubuntu App Developer Blog – the official source for news, updates, new tutorials and other application development content in Ubuntu. You can read it and subscribe to it

Ubuntu App Developers on G+ – the Google+ page to for anyone interested in app development in Ubuntu to read and share updates. You can add it to your circles or +1 it

Ubuntu App Developers on Facebook – the Facebook page, also for enthusiasts of app development in Ubuntu to follow and comment on the latest news. You can like it.

Ubuntu App Developers on LinkedIn – the LinkedIn group for professionals wanting to know more about publishing their apps in the Software Centre. You can join it.

Ubuntu App Developers on Twitter – you prefer 140 character updates? @ubuntuappdev is also tweeting away in the microblogs world, spreading the news on Ubuntu App Development. You can follow it.

Ubuntu App Developers on Identi.ca – if your microblogging choice is the open source alternative to Twitter, Ubuntu app developers are also on identi.ca. You can follow it.

Get (or give) support

This is a set of channels to either get help, give help, or actively contribute to discussions related to Ubuntu App Development.

Ubuntu App Development on Askubuntu – the central place to get and provide support for all your app development questions. You can ask questions, answer questions, read the FAQ and subscribe to the questions feed.

Ubuntu App Development Mailing list – the list is also the place for support, but also for discussion of new topics, coordination of work and announcements related to building the Ubuntu App Developer story. You can subscribe to it or send e-mail.

Ubuntu App Development on IRC – for those seeking real-time support on text or simply a friendly chat amongst app developers. You can enter the IRC channel.

Contribute

This is an overview of some of the ways in which to contribute to the Ubuntu App Developer story.

Create an app – the most obvious way to make an impact is to actually create an app to be distributed to millions in the Software Centre. You can learn how to get started, how to publish, and actually publish your application. Also check out the video tutorial in how to get started in app development on Ubuntu in a matter of minutes.

Submit a tutorial – knowledge sharing is a key contribution to app development in Ubuntu. If you know about an app development topic you’d like to see featured and shared in the Ubuntu App developer site, you can submit a tutorial.

Join the ARB – our vision is that both open source and commercial applications are the key to a successful app ecosystem in Ubuntu. The Application Review Board are a group of individuals committed to reviewing and helping open source apps thrive in this environment. If you have technical skills and want to contribute to this goal, they need your help.

All in all, this now gives no excuse not to know what’s going on in the app development world and to get involved. Now let’s get to work to have a stunning App Developer story!

Social Media Icons by Paul Robert Lloyd

Read more
Jonathan Lange

Automatic packaging progress

We’re doing some work to automatically package the things that application developers submit to the Ubuntu App Developer site, and we thought you might be interested in following along.

In a nutshell, we’re going to put a service behind MyApps that harnesses pkgme to automatically generate packaging for binary applications. We hope to extend it to automatically package other types of applications (e.g. ./configure; make; make install) some day soon.

If you want to know more, you can read the spec, look at the system diagram, read our implementation plan or watch the demo of pkgme doing a binary application.

At the moment, it’s James Westby and I working on this as our day jobs at Canonical. However, we’re both pretty keen to do everything in the open, and would gladly welcome contributions.

Oh, speaking of openness. We’re trying to gather up all of the work that Ubuntu is doing to make life for application developers easier and stick it on one wiki page so that everyone can figure out what everyone else is doing. That page is:

https://wiki.ubuntu.com/AppDevelopers

It’s so important I couldn’t possibly leave it as a mere hyperlink. It also has links to the various Launchpad projects that we’re using for the automated packaging service.

Read more
John Pugh

 

 

MumboJumbo, a leading developer and publisher of casual games,  has added four titlesto the Ubuntu Software Center. Chainz Galaxy, 7 Wonders: Magical Mystery Tour, Unlikely Suspects, and Midnight Mysteries3 Devil on the Mississippi are now available for Ubuntu 10.10 and Ubuntu 11.04. The same titles will be available for Ubuntu 11.10 very soon.

Chainz Galaxy

Chainz Galaxy is a puzzle game where the user twists links to create link-matches of three or more to clear the board. You earn bonuses by getting power-ups and gathering charms.

7 Wonders: Magical Mystery Tour

7 Wonders: Magical Mystery Tour is an all new release in the 7 Wonders series. In this game you’ll take a fantastic journey to some of the most enchanting sites in mythology and folklore.  With a team of master builders, you’ll match colorful runes to collect the building blocks needed for Camelot, El Dorado, Atlantis, Shangri-La, and many others.  Build all 7 Wonders and then unlock a magical 8th location, Nazca Valley!

Unlikely Suspects

Unlikely Suspects is a hidden object whodunit where you track 16 criminals across the globe! The Superintendent of Interpol needs your help to sort through evidence and decipher clues. With over 4,000 different outcomes, there’s always a new case to crack!

Midnight Mysteries 3 Devil on the Mississippi

In Midnight Mysteries 3 Devil on the Mississippi Mark Twain’s fleeing ghost begs you to free his soul from demons, both literal and emotional. Find objects and solve puzzles to unravel the mystery behind Shakspeare’s identity, squash the awakened evil spirit, and set history straight!

Check out these new titles in the Ubuntu Software Center today! If you have an application you wish to include in the Software Center submit it today in the MyApps portal on the Ubuntu App Developers site.

Read more
David Planella

UDS is here again. Tomorrow another week packed with content that will define the plans for a new Ubuntu LTS release will start, and this time around application development will be a prominent topic.

So for all of you interested in helping and being part of the effort of making Ubuntu a platform of choice for application developers, here’s a quick list with an overview of the sessions we’ve got in store this week.

Remember you can register your interest in sessions you want to attend or keep up to date with by using the Subscribe link on each session’s blueprint. The links in the list below will take you to the blueprints used to define the specifications for each feature or goal. You can also check out the full UDS schedule.

So, without further ado, here’s the list of app development sessions:

Oh, and don’t miss the Application development and the Qt keynotes on Tuesday

See you all there!

Read more
David Planella

Just a heads up that this week is Ubuntu Open Week!

Ubuntu Open Week is a series of online workshops where you can:

  • Learn about the Ubuntu landscape
  • Talk to some of the key developers from the Ubuntu project
  • Find out about the Community and its relationship with Canonical
  • Participate in an open Q&A with Mark Shuttleworth, the founder of Ubuntu. (*)

I’ll be running two sessions for everyone wanting to learn more about either translating Ubuntu or writing apps for Ubuntu (or both!):

  • WHERE: #ubuntu-classroom IRC channel on the Freenode network
  • WHAT, WHEN:
    • 14:00 UTC How to contribute translating Ubuntu – learn how to translate Ubuntu in your language
    • 16:00 UTC Writing your first Ubuntu app – a very gentle introduction to writing a simple “Ubuntu rocks!” app, and explaining its lifecycle from creation to publication, with tips along the way.

(*) Mark is on vacation this week, but we’ll schedule a separate IRC session with him, stay tuned!

Read more
David Planella

Announcing the Ubuntu App Developer site

I’m thrilled to announce the launch of a significant milestone in the ongoing effort of making Ubuntu a target for app developers: the new Ubuntu App Developer site.

Developer.ubuntu.com should now be the central point of reference for any topics related to application development. From creation to publication: porting, sharing, contributing, finding information… a site that should grow organically to provide the tools, share the knowledge and act as the springboard to foster app proliferation and developer community growth in this exciting area.

Embracing the concept that for sustainable business development and wider adoption Ubuntu should provide the ability to purchase apps as well as to install open source software for free, the site has been developed with that exact idea in mind. In that regard, developer.ubuntu.com has been built upon the foundations of existing tools such as the Ubuntu Software Centre and My Apps to offer a unified and consistent journey for both open-source, non-commercial and commercial app authors.

A quick walk through the App Developer site

All sections of the site have been carefully designed upon research and user testing to provide a simple yet intuitive journey to app developers, from software creation to publication and distribution in the Ubuntu Software Centre. Here’s a quick guide:

  • Get started – an introduction to the recommended tools and the workflow for Ubuntu app development, including a video to kickstart new developers in less than five minutes.
  • Resources – a growing directory of resources to support app development in Ubuntu: reference documentation, guides, tutorials and more.
  • Publish – a simple guide to app publication in the Ubuntu Software Centre, including an extensive FAQ on how to publish commercial applications.
  • Community – the place to get involved, share and actively participate within a network of like-minded developers, including the app developer blog.
  • My Apps – an online tool to enable app developers go straight from a finished app to the Software Centre, making publication a seamless process.

Big thanks

In this project I’ve had the privilege of working with the Web Design team, who essentially made the site happen. It’s been a fun ride, and they’ve produced a truly stunning result in a very tight schedule. To them, and to many Canonicalers and other members of the Ubuntu family go the thanks for making developer.ubuntu.com possible.

Next steps

This is just the beginning. For all its current awesomeness, we are aware that the site needs to pass the test of a wider audience, adapt to their needs, and grow. Expect more developer.ubuntu.com discussions at the next Ubuntu Developer Summit in Orlando, Florida, where any community member can participate and contribute to the discussion of the future plans for the site.

In the meantime, there are two easy ways in which you can already start participating:

This is a very exciting and new territory for Ubuntu, and I’m thrilled to see all the progress we’re making in this area. This is going to be awesome.

This post was originally published here

Read more
John Pugh

Things are really ramping up with submissions into the Ubuntu Software Center. With a app per day being submitted, the packaging team is getting busy keeping up with the cool applications arriving!

TRAUMA is likely the most interesting new submission. Very unique. You are in the mind of a traumatized young woman as she has just been in a car accident. You experience her dreams in a interactive way.

Check out the trailer:

Buy TRAUMA from the Ubuntu Software Center.

Books and Magazines

We also added some magazines to the mix. Several recent issues of Ubuntu User magazine by Linux New Media are available on the Software Center. You can find the most recent issues up through Ubuntu User issue #9 which has a section dedicated to Ubuntu 11.04 and Unity. Keep your eyes peeled for book titles about Linux and Ubuntu arriving soon.

Photobomb

Another really interesting title recently released is called Photobomb. It’s described as a “Easy and Social Image Editor”. It’s like a mashup tool for your images. Pretty slick and at $2.99 it’s a cinch to check out. Go buy it and provide some feedback today.

We have some very cool submissions pending the packaging process in the queue. Thanks to all of our interested developers out there we have officially backed up the packaging team! Don’t worry though…we’ll soon work through that backlog and have a lot of new and interesting titles showing up regularly in the software center.

Check them out, provide some feeback, and even submit more!
To submit a new application go to https://myapps.developer.ubuntu.com

And one last thing – keep your eye out for the updated developer.ubuntu.com website coming in early October!

Read more
David Planella

Another edition of the Ubuntu App Developer Week and another amazing knowledge sharing fest around everything related to application development in Ubuntu. Brought to you by a range of the best experts in the field, here’s just a sample of the topics they talked about: App Developer Strategy, Bazaar, Bazaar Explorer, Launchpad, Python, Internationalization, Launchpad Translations, Unity, Unity 2D, Gedit Developer Plugins, the MyApps Portal, the App Review Board, the UbuntuSoftware Centre, Unity Mail, Launchpad Daily Builds, Ubuntu One APIs, Rapid App Development, Quickly, GooCanvas, PyGame, Unity Launcher, Vala, the App Developer Site, Indicators, Python Desktop Integration, Libgrip, Multitouch, Unity Lenses, Ubuntu One Files Integration, The Business Side of Apps, Go, Qt Quick… and more. Oh my!

And a pick of what they had to say:

We believe that to get Ubuntu from 20 million to 200 million users, we need more and better apps on Ubuntu
Jonathan Lange on making Ubuntu a target for app developers

Bazaar is the world’s finest revision control system
Jonathan Riddell on Bazaar

So you’ve got your stuff, wherever you are, whichever device you’re on
Stuart Langridge on Ubuntu One

Oneiric’s EOG and Evince will be gesture-enabled out of the box
Jussi Pakkanen on multitouch in Ubuntu 11.10

I control the upper right corner of your screen ;-)
Ted Gould on Indicators

If you happened to miss any of the sessions, you’ll find the logs for all of them on the Ubuntu App Developer Week page, and the summaries for each day on the links below:

Ubuntu App Developer Week – Day 5 Summary

The last day came with a surprise: an extra session for all of those who wanted to know more about Qt Quick and QML. Here are the summaries:

Getting A Grip on Your Apps: Multitouch on GTK apps using Libgrip

By Jussi Pakkanen

In his session, Jussi talked about one of the most interesting technologies where Ubuntu is leading the way in the open source world: multitouch. Walking the audience through the Grip Tutorial, he described how to add gesture support to existing applications based on GTK+ 3. He chose to focus on the higher layer of the uTouch stack, where he explained the concepts on which libgrip, the gesture library, is built upon, such as device types and subscriptions. After having explored in detail the code examples, he then revealed that in Oneiric Eye Of GNOME and Evince, Ubuntu’s default image viewer and default PDF reader, will be gesture-enabled.

Check out the session log.

Creating a Google Docs Lens

By Neil Patel

Neil introduced his session explaining the background behind Lenses: a re-architecture effort of the now superseded Places concept to make them more powerful, provide more features and make it easier to add features through a re-engineered API. Lenses create its own instance, add categories, filters and leave the searching to Scopes. The Lenses/Scopes pairs are purely requests for data, independent of the type of UI, and being provided by the libunity library, they can be written in any of the programming languages supported by GObject Introspection (Python, Javascript, C/C++, Vala, etc.). To illustrate all of this concepts, Neil devoted the rest of the session to a real example of creating a Lens for Google Docs.

Check out the session log.

Practical Ubuntu One Files Integration

By Michael Terry

Another hands-on session from Michael, with a real world example on how to supercharge apps with cloud support. Using his experience in integrating the Ubuntu One Files API to Deja Dup, the default backup application in Ubuntu, he went in detail through the code of a simple program to talk to a user’s personal Ubuntu One file storage area. We liked Michael’s session so much that it will very soon be featured as a tutorial on developer.ubuntu.com!

Check out the session log and Michael’s awesome notes.

Publishing Your Apps in the Software Center: The Business Side

By John Pugh

Ubuntu directly benefits from Canonical becoming a sustainable business to support its development, and that’s exactly what John talked about. Being responsible for business development in the Ubuntu Software Centre, he’s got a privileged  insight on how to make it happen. He started off explaining that the main goal is to present Ubuntu users with a large catalog of apps available for purchase, and then continued concentrating on how to submit paid applications to be published in the Software Centre. A simple 5-step process, the behind-the-scenes work can be summarized in: Canonical helps packaging the app, it hosts the app and provides the payment via pay.ubuntu.com, in a 80%/20% split. Other highlights include the facts that only non-DRM, non-licensed apps cannot be submitted right now, but there is ongoing work to implement license key support, and that MyApps, the online app submission portal, can take any nearly any content: apps with adverts, “free” online game clients and HTML5 apps.

Check out the session log.

Writing an App with Go

By Gustavo Niemeyer

Gustavo’s enthusiasm for Go, the new programming language created by Google shows every time you start a conversation with him on that topic. And it showed as well on this session, in which he created yet another “Hello world” application in a new language -you guessed-: Go. Along the way, he had time to describe all of the features of this new addition of the extensive family of programming languages: statically compiled with good reflection capabilities, structural typing, interfaces and more.

Check out the session log.

Qt Quick At A Pace

By Donald Carr

Closing the week on the last -and surprise- session, we had the luxury of having Donald, from the Nokia Qt team, the makers of Qt itself, to talk about Qt Quick. Using a clear and concise definition, Qt Quick is an umbrella term used to refer to QML and its associated tooling; QML being a declarative markup language with tight bindings to Javascript. A technology equally suited to mobile or to the desktop, QML enables developers to rapidly create animation-rich, pixmap-oriented UIs. Through the qtmediahub and Qt tutorial examples, he explored QML’s capabilities and offered good practices for succesfully developing QML-based projects.

Check out the session log.

Wrapping Up

Finally, if you’ve got any feedback on UADW, on how to make it better, things you enjoyed or things you believe should be improved, your comments will be very appreciated and useful to tailor this event to your needs.

Thanks a lot for participating. I hope you enjoyed it  as much as I did, and see you again in 6 months time for another week full with app development goodness!

Read more
Gustavo Niemeyer

Back in 2009 I quickly talked about the obvious revolution in computing that was rolling in the form of mobile phone as computer, and mentioned as well the fact that touch-based interfaces were going to dominate the marketplace because of that.

Move forward a couple of years, and last week I got my first tablet, running Android (a Samsung Galaxy Tab 10.1, if you’re curious). I didn’t know exactly why I needed one, but being in the tech industry I always have that nice excuse for myself of buying things early on for learning about the experience of using them. Last night, I could clearly see this can be a real claim in some cases (in others it’s just an excuse for the wife).

After getting the tablet last week, I’ve started by experimenting with the usual stuff any person would (email, browser, etc), and then downloaded a few games to take on board a longish flight. Some of them were pretty good.. a vertical scrolling shooter, a puzzle-solver, and so on. On all of them, though, it took just a few minutes before the novelty of holding the screen in my hands for interacting with the game got old, and the interest went away with it.

This last night, though, I’ve decided to try another game from the top list, named Cut the Rope, and this time I was immediately hooked into it. That was certainly one of the most enjoyable gaming experiences I had in quite a while, and when going to bed I started to ponder about what was different there.

The game is obviously well executed, with cute drawings and sounds, and also smooth, but I think there was something else as well. In retrospect, the other games felt a lot like ports of a desktop/laptop experience. The side scrolling game, for instance, was quite well suited for a joystick, and at least one other game had an actual joystick emulated on the screen, which is an enabler, but far from nice to be honest.

This one game, though, felt very well suited for a hands-based interaction: quickly drawing lines for cutting ropes, tapping on balloons to push air out, moving levers around, etc. In some more advanced levels, it was clear that my dexterity (or lack thereof) was playing a much more important role in accomplishing the tasks than the traditional button/joystick version of it. This felt like an entirely novel gaming experience that just hadn’t happened yet.

It’s funny and ironic that I had this experience within a week from Microsoft reportedly saying (again!) that a tablet is just another PC. It’s not, and if they tried it out with some minimum attention they’d see why it’s so clearly not.

In that experience, the joystick felt familiar but at the same quite awkward to use, but using my hands naturally in an environment where that was suitable felt very pleasing. We can generalize that a bit and note a common way to relate to innovation: we first try to reuse the knowledge we have when facing a new concept, but when we understand the concept better quite often we’re able to come up with more effective and interesting ways to relate to it.

In the tablet vs. laptop/desktop thread, you probably won’t want to be typing long documents in a tablet, but would most likely prefer to shuffle items in an agenda with your fingers. Also, you likely wouldn’t want to do that detailed CAD work with a fat finger in a screen, but would certainly be happy to review code or a document sitting in your backyard with the birds (no whales).

So, let’s please put that hammer away for a second while creating a most enjoyable touch-based experience.

Read more
Ara

I have been asked to write a chapter for a book about the experiences of people involved in Open Source with the idea of “If I knew what I know today”. I asked if I could re-print my contribution here. I hope it is interesting for people concerned about Open Source testing.

Dogfooding Is Not Enough

I have been involved with Open Source since my early days at University, in Granada. There, with some friends, we founded the local Linux User Group and organized several activities to promote Free Software. But, since I left university, and until I started working at Canonical, my professional career had been in the proprietary software industry, first as a developer and after that as a tester.

When working in a proprietary software project, testing resources are very limited. A small testing team continues the work that developers started with unit testing, using their expertise to find as many bugs as possible, to release the product in good shape for end user consumption. In the free software world, however, everything changes.

When I was hired at Canonical, apart from fulfilling the dream of having a paid job in a free software project, I was amazed by the possibilities that testing a free software project brought. The development of the product happens in the open, and users can access to the software in the early stages, test it and file bugs as they encounter them. For a person passioned about testing, this is a new world with lots of new possibilities. I wanted to make the most of it.

As many people do, I thought that dogfooding, or using the software that you are aiming to release, was the most important testing activity that we could do in open source. But, if “given enough eyeballs all the bugs are shallow”, (one of the key lessons of Raymond’s “The Cathedral & The Bazaar”), and Ubuntu had millions of users, why very important bugs were still slipping into the release?

First thing that I found when I started working at Canonical was that the organized testing activities were very few or nonexistent. The only testing activities that were somehow organized were in the form of emails sent to a particular mailing list calling for testing a package in the development version of Ubuntu. I don’t believe that this can be considered a proper testing activity, but just another form of dogfooding. This kind of testing generates a lot of duplicated bugs, as a really easy to spot bug will be filed by hundreds of people. Unfortunately, the really hard to spot but potentially critical bug, if someone files it, is likely to remain unnoticed, due to the noise created by the other hundreds of bugs.

Looking better

Is this situation improving? Are we getting better at testing in FLOSS projects? Yes, I really believe so.

During the latest Ubuntu development cycles we have started several organized testing activities. The range of topics for these activities is wide, including areas like new desktop features, regression testing, X.org drivers testing or laptop hardware testing. The results of these activities are always tracked, and they prove to be really useful for developers, as they are able to know if the new features are working correctly, instead of guessing that they work correctly because of the absence of bugs.

Regarding tools that help testing, many improvements have been made:

  • Apport has contributed to increase the level of detail of the bugs reported against Ubuntu: crashers include all the debugging information and their duplicates are found and marked as such; people can report bugs based on symptoms, etc.
  • Launchpad, with its upstream connections, has allowed having a full view of the bugs, knowing that bugs happening in Ubuntu are usually bugs in the upstream projects, and allowing developers to know if the bugs are being solved there.
  • Firefox, with its Test Pilot extension and program, drives the testing without having to leave the browser. This is, I believe, a much better way to reach testers than a mailing list or an IRC channel.
  • The Ubuntu QA team is testing the desktop in an automated fashion and reporting results every week, allowing developers to have a very quick way to check that there have not been any major regressions during the development.

Although testing FLOSS projects is getting better, there is still a lot to be done.

Looking ahead

Testing is a skilled activity that requires lots of expertise, but in the FLOSS community is still seen as an activity that doesn’t require much effort. One of the reasons could be that the way we do testing is still very old fashioned and does not reflect the increase of complexity in the free software world in the last decade. How can it be possible that with the amount of innovation that we are generating in open source communities, testing is still done like it was in the 80s? Let’s face it, fixed testcases are boring and get easily outdated. How are we going to grow a testing community, who is supposed to find meaningful bugs if their main required activity is updating testcases?

But, how do we improve testing? Of course, we cannot completely get rid of testcases, but we need to change the way we create and maintain them. Our testers and users are intelligent, so, why creating step-by-step scripts? Those could easily get replaced by an automated testing tool. Instead of that, let’s just have a list of activities you perform with the application and some properties it should have, for example, “Shortcuts in the launcher can be rearranged” or “Starting up LibreOffice is fast”. Testers will figure out how to do it, and will create their testcases as they test.

But this is not enough, we need better tools to help testers know what to test, when and how.  What about having an API to allow developers to send messages to testers about updates or new features that need testing? What about an application that tell us what part of our system needs testing based on testing activity? In the case of Ubuntu we have the data in Launchpad (we would need testing data as well, but at least we have bug data). If I want to start a testing session against a particular component I would love to have the areas that haven’t been tested yet and a list of the 5 bugs with more duplicates for that particular version, so I avoid filing those again. I would love to have all this information without leaving the same desktop that I am testing. This is something that Firefox has started with Test Pilot, although they are currently mainly gathering browser activity. Google is also doing some research in this area.

Communication between downstream and upstream and vice-versa also needs to get better. During the development of a distribution, many of the upstream versions are also under development, and they already have a list of known bugs. If I am a tester of Firefox through Ubuntu, I would love to have a list of known bugs as soon as the new package gets to the archive. This could be done by having an acknowledged syntax for release notes, that could then get easily parsed and bugs automatically filed and connected to the upstream bugs. Again, all of this information should be easily available to the tester, without leaving the desktop.

Testing, if done this way, would allow the tester to concentrate on the things that really matter and that make testing a skilled activity; concentrate on the hidden bugs that haven’t been found yet, on the special configurations and environments, on generating new ways to break the software. On having fun while testing.

Wrapping Up

From what I have seen in the latest three years, testing has improved a lot in Ubuntu and the rest of FLOSS projects that I am somehow involved with, but this is not enough. If we really want to increase the quality of open source software we need to start investing in testing and innovating the ways we do it, the same way we invest in development. We cannot test 21st century software, with 20th century testing techniques. We need to react. Open Source is good because is open source is not enough anymore. Open Source will be good because it is open source and has the best quality that we can offer.

 

Read more
niemeyer

Circular buffers are based on an algorithm well known by any developer who’s got past the “Hello world!” days. They offer a number of key characteristics with wide applicability such as constant and efficient memory use, efficient FIFO semantics, etc.

One feature which is not always desired, though, it the fact that circular buffers traditionally will either overwrite the last element, or raise an overflow error, since they are generally implemented as a buffer of constant size. This is an unwanted property when one is attempting to consume items from the buffer and it is not an option to blindly drop items, for instance.

This post presents an efficient (and potentially novel) algorithm for implementing circular buffers which preserves most of the key aspects of the traditional version, while also supporting dynamic expansion when the buffer would otherwise have its oldest entry overwritten. It’s not clear if the described approach is novel or not (most of my novel ideas seem to have been written down 40 years ago), so I’ll publish it below and let you decide.

Traditional circular buffers

Before introducing the variant which can actually expand during use, let’s go through a quick review on traditional circular buffers, so that we can then reuse the nomenclature when extending the concept. All the snippets provided in this post are written in Python, as a better alternative to pseudo-code, but the concepts are naturally portable to any other language.

So, the most basic circular buffer needs the buffer itself, its total capacity, and a position where the next write should occur. The following snippet demonstrates the concept in practice:

buf = [None, None, None, None, None]
bufcap = len(buf)
pushi = 0   

for elem in range(7):
    buf[pushi] = elem
    pushi = (pushi + 1) % bufcap
    
print buf # => [5, 6, 2, 3, 4]

In the example above, the first two elements of the series (0 and 1) were overwritten once the pointer wrapped around. That’s the specific feature of circular buffers which the proposal in this post will offer an alternative for.

The snippet below provides a full implementation of the traditional approach, this time including both the pushing and popping logic, and raising an error when an overflow or underflow would occur. Please note that these snippets are not necessarily idiomatic Python. The intention is to highlight the algorithm itself.

class CircBuf(object):

    def __init__(self):
        self.buf = [None, None, None, None, None]
        self.buflen = self.pushi = self.popi = 0
        self.bufcap = len(self.buf)

    def push(self, x):
        assert self.buflen == 0 or self.pushi != self.popi, 
               "Buffer overflow!"
        self.buf[self.pushi] = x
        self.pushi = (self.pushi + 1) % self.bufcap
        self.buflen += 1

    def pop(self):
        assert self.buflen != 0, "Buffer underflow!"
        x = self.buf[self.popi]
        self.buf[self.popi] = None
        self.buflen -= 1
        self.popi = (self.popi + 1) % self.bufcap
        return x

With the basics covered, let’s look at how to extend this algorithm to support dynamic expansion in case of overflows.

Dynamically expanding a circular buffer

The approach consists in imagining that the same buffer can contain both a circular buffer area (referred to as the ring area from here on), and an overflow area, and that it is possible to transform a mixed buffer back into a pure circular buffer again. To clarify what this means, some examples are presented below. The full algorithm will be presented afterwards.

First, imagine that we have an empty buffer with a capacity of 5 elements as per the snippet above, and then the following operations take place:

for i in range(5):
    circbuf.push(i)

circbuf.pop() # => 0
circbuf.pop() # => 1

circbuf.push(5)
circbuf.push(6)

print circbuf.buf # => [5, 6, 2, 3, 4]

At this point we have a full buffer, and with the original implementation an additional push would raise an assertion error. To implement expansion, the algorithm will be changed so that those items will be appended at the end of the buffer. Following the example, pushing two additional elements would behave the following way:

circbuf.push(7)
circbuf.push(8)

print circbuf.buf # => [5, 6, 2, 3, 4, 7, 8]

In that example, elements 7 and 8 are part of the overflow area, and the ring area remains with the same capacity and length of the original buffer. Let’s perform a few additional operations to see how it would behave when items are popped and pushed while the buffer is split:

circbuf.pop() # => 2
circbuf.pop() # => 3
circbuf.push(9)

print circbuf.buf # => [5, 6, None, None, 4, 7, 8, 9]

In this case, even though there are two free slots available in the ring area, the last item pushed was still appended at the overflow area. That’s necessary to preserve the FIFO semantics of the circular buffer, and means that the buffer may expand more than strictly necessary given the space available. In most cases this should be a reasonable trade off, and should stop happening once the circular buffer size stabilizes to reflect the production vs. consumption pressure (if you have a producer which constantly operates faster than a consumer, though, please look at the literature for plenty of advice on the problem).

The remaining interesting step in that sequence of events is the moment when the ring area capacity is expanded to cover the full allocated buffer again, with the previous overflow area being integrated into the ring area. This will happen when the content of the previous partial ring area is fully consumed, as shown below:

circbuf.pop() # => 4
circbuf.pop() # => 5
circbuf.pop() # => 6
circbuf.push(10)

print circbuf.buf # => [10, None, None, None, None, 7, 8, 9]

At this point, the whole buffer contains just a ring area and the overflow area is again empty, which means it becomes a traditional circular buffer.

Sample algorithm

With some simple modifications in the traditional implementation presented previously, the above semantics may be easily supported. Note how the additional properties did not introduce significant overhead. Of course, this version will incur in additional memory allocation to support the buffer expansion, bu that’s inherent to the problem being solved.

class ExpandingCircBuf(object):

    def __init__(self):
        self.buf = [None, None, None, None, None]
        self.buflen = self.ringlen = self.pushi = self.popi = 0
        self.bufcap = self.ringcap = len(self.buf)

    def push(self, x):
        if self.ringlen == self.ringcap or 
           self.ringcap != self.bufcap:
            self.buf.append(x)
            self.buflen += 1
            self.bufcap += 1
            if self.pushi == 0: # Optimization.
                self.ringlen = self.buflen
                self.ringcap = self.bufcap
        else:
            self.buf[self.pushi] = x
            self.pushi = (self.pushi + 1) % self.ringcap
            self.buflen += 1
            self.ringlen += 1

    def pop(self):
        assert self.buflen != 0, "Buffer underflow!"
        x = self.buf[self.popi]
        self.buf[self.popi] = None
        self.buflen -= 1
        self.ringlen -= 1
        if self.ringlen == 0 and self.buflen != 0:
            self.popi = self.ringcap
            self.pushi = 0
            self.ringlen = self.buflen
            self.ringcap = self.bufcap
        else:
            self.popi = (self.popi + 1) % self.ringcap
        return x

Note that the above algorithm will allocate each element in the list individually, but in sensible situations it may be better to allocate additional space for the overflow area in advance, to avoid potentially frequent reallocation. In a situation when the rate of consumption of elements is about the same as the rate of production, for instance, there are advantages in doubling the amount of allocated memory per expansion. Given the way in which the algorithm works, the previous ring area will be exhausted before the mixed buffer becomes circular again, so with a constant rate of production and an equivalent consumption it will effectively have its size doubled on expansion.

UPDATE: Below is shown a version of the same algorithm which not only allows allocating more than one additional slot at a time during expansion, but also incorporates it in the overflow area immediately so that the allocated space is used optimally.

class ExpandingCircBuf2(object):

    def __init__(self):
        self.buf = []
        self.buflen = self.ringlen = self.pushi = self.popi = 0
        self.bufcap = self.ringcap = len(self.buf)

    def push(self, x):
        if self.ringcap != self.bufcap:
            expandbuf = (self.pushi == 0)
            expandring = False
        elif self.ringcap == self.ringlen:
            expandbuf = True
            expandring = (self.pushi == 0)
        else:
            expandbuf = False
            expandring = False

        if expandbuf:
            self.pushi = self.bufcap
            expansion = [None, None, None]
            self.buf.extend(expansion)
            self.bufcap += len(expansion)
            if expandring:
                self.ringcap = self.bufcap

        self.buf[self.pushi] = x
        self.buflen += 1
        if self.pushi < self.ringcap:
            self.ringlen += 1
        self.pushi = (self.pushi + 1) % self.bufcap

    def pop(self):
        assert self.buflen != 0, "Buffer underflow!"
        x = self.buf[self.popi]
        self.buf[self.popi] = None
        self.buflen -= 1
        self.ringlen -= 1
        if self.ringlen == 0 and self.buflen != 0:
            self.popi = self.ringcap
            self.ringlen = self.buflen
            self.ringcap = self.bufcap
        else:
            self.popi = (self.popi + 1) % self.ringcap
        return x

Conclusion

This blog post presented an algorithm which supports the expansion of circular buffers while preserving most of their key characteristics. When not faced with an overflowing buffer, the algorithm should offer very similar performance characteristics to a normal circular buffer, with a few additional instructions and constant space for registers only. When faced with an overflowing buffer, the algorithm maintains the FIFO property and enables using contiguous allocated memory to maintain both the original circular buffer and the additional elements, and follows up reusing the full area as part of a new circular buffer in an attempt to find the proper size for the given use case.

Read more
Gustavo Niemeyer

ZooKeeper is a clever generic coordination server for distributed systems, and is one of the core softwares which facilitate the development of Ensemble (project for automagic IaaS deployments which we push at Canonical), so it was a natural choice to experiment with.

Gozk is a complete binding for ZooKeeper which explores the native features of Go to facilitate the interaction with a ZooKeeper server. To avoid reimplementing the well tested bits of the protocol in an unstable way, Gozk is built on top of the standard C ZooKeeper library.

The experience of integrating ZooKeeper with Go was certainly valuable on itself, and worked as a nice way to learn the details of integrating the Go language with a C library. If you’re interested in learning a bit about Go, ZooKeeper, or other details related to the creation of bindings and asynchronous programming, please fasten the seatbelt now.

Basics of C wrapping in Go

Creating the binding on itself was a pretty interesting experiment already. I have worked on the creation of quite a few bindings and language bridges before, and must say I was pleasantly surprised with the experience of creating the Go binding. With Cgo, the name given to the “foreign function interface” mechanism for C integration, one basically declares a special import statement which causes a pre-processor to look at the comment preceding it. Something similar to this:

// #include <zookeeper.h>
import "C"

The comment doesn’t have to be restricted to a single line, or to #include statements even. The C code contained in the comment will be transparently inserted into a helper C file which is compiled and linked with the final object file, and the given snippet will also be parsed and inclusions processed. In the Go side, that “C” import is simulated as if it were a normal Go package so that the C functions, types, and values are all directly accessible.

As an example, a C function with this prototype:

int zoo_wexists(zhandle_t *zh, const char *path, watcher_fn watcher,
                void *context, struct Stat *stat);

In Go may be used as:

cstat := C.struct_Stat{}
rc, cerr := C.zoo_wexists(zk.handle, cpath, nil, nil, &cstat)

When the C function is used in a context where two result values are requested, as done above, Cgo will save the well known errno variable after the function has finished executing and will return it wrapped into an os.Errno value.

Also, note how the C struct is defined in a way that can be passed straight to the C function. Interestingly, the allocation of the memory backing the structure is going to be performed and tracked by the Go runtime, and will be garbage collected appropriately once no more references exist within the Go runtime. This fact has to be kept in mind since the application will crash if a value allocated normally within Go is saved with a foreign C function and maintained after all the Go references are gone. The alternative in these cases is to call the usual C functions to get hold of memory for the involved values. That memory won’t be touched by the garbage collector, and, of course, must be explicitly freed when no longer necessary. Here is a simple example showing explicit allocation:

cbuffer := (*C.char)(C.malloc(bufferSize))
defer C.free(unsafe.Pointer(cbuffer))

Note the use of the defer statement above. Even when dealing with foreign functionality, it comes in handy. The above call will ensure that the buffer is deallocated right before the current function returns, for instance, so it’s a nice way to ensure no leaks happen, even if in the future the function suddenly gets a new exit point which didn’t consider the allocation of resources.

In terms of typing, Go is more strict than C, and Cgo-based logic will also ensure that the types returned and passed into the foreign C functions are correctly typed, in the same way done for the native types. Note above, for instance, how the call to the free() function has to explicitly convert the value into an unsafe.Pointer, even though in C no casting would be necessary to pass a pointer into a void * parameter.

The unsafe.Pointer is in fact a very special type within Go. Using it, one can convert any pointer type into any other pointer type in an unsafe way (thus the package name), and also back and forth into a uintptr value with the address of the memory referenced by the pointer. For every other type conversion, Go will ensure at compilation time that doing the conversion at runtime is a safe operation.

With all of these resources, including the ability to use common Go syntax and functionality even when dealing with foreign types, values, and function calls, the integration task turns out to be quite a pleasant experience. That said, some of the things may still require some good thinking to get right, as we’ll see shortly.

Watch callbacks and channels

One of the most interesting (and slightly tricky) aspects of mapping the ZooKeeper concepts into Go was the “watch” functionality. ZooKeeper allows one to attach a “watch” to a node so that the server will report back when changes happen to the given node. In the C library, this functionality is exposed via a callback function which is executed once the monitored node aspect is modified.

It would certainly be possible to offer this functionality in Go using a similar mechanism, but Go channels provide a number of advantages for that kind of asynchronous notification: waiting for multiple events via the select statement, synchronous blocking until the event happens, testing if the event is already available, etc.

The tricky bit, though, isn’t the use of channels. That part is quite simple. The tricky detail is that the C callback function execution happens in a C thread started by the ZooKeeper library, and happens asynchronously, while the Go application is doing its business elsewhere. Right now, there’s no straightforward way to transfer the execution of this asynchronous C function back into the Go land. The solution for this problem was found with some help from the folks at the golang-nuts mailing list, and luckily it’s not that hard to support or understand. That said, this is a good opportunity to get some coffee or your preferred focus-enhancing drink.

The solution works like this: when the ZooKeeper C library gets a watch notification, it executes a C callback function which is inside a Gozk helper file. Rather than transferring control to Go right away, this C function simply appends data about the event onto a queue, and signals a pthread condition variable to notify that an event is available. Then, on the Go side, once the first ZooKeeper connection is initialized, a new goroutine is fired and loops waiting for events to be available. The interesting detail about this loop, is that it blocks within a foreign C function waiting for an event to be available, through the signaling of the shared pthread condition variable. In the Go side, that’s how the call looks like, just to give a more practical feeling:

// This will block until there's a watch available.
data := C.wait_for_watch()

Then, on the C side, here is the function definition:

watch_data *wait_for_watch() {
    watch_data *data = NULL;
    pthread_mutex_lock(&watch_mutex);
    if (first_watch == NULL)
        pthread_cond_wait(&watch_available, &watch_mutex);
    data = first_watch;
    first_watch = first_watch->next;
    pthread_mutex_unlock(&watch_mutex);
    return data;
}

As you can see, not really a big deal. When that kind of blocking occurs inside a foreign C function, the Go runtime will correctly continue the execution of other goroutines within other operating system threads.

The result of this mechanism is a nice to use interface based on channels, which may be explored in different ways depending on the application needs. Here is a simple example blocking on the event synchronously, for instance:

stat, watch, err := zk.ExistsW("/some/path")
if stat == nil && err == nil {
    event := <-watch
    // Use event ...
}

Concluding

Those were some of the interesting aspects of implementing the ZooKeeper binding. I would like to speak about some additional details, but this post is rather long already, so I'll keep that for a future opportunity. The code is available under the LGPL, so if you're curious about some other aspect, or would like to use ZooKeeper with Go, please move on and check it out!

Read more
Gustavo Niemeyer

A while ago Martin Pool made a very interesting post on the design of interfaces, inspired by a talk from Rusty Russel from 2003.

Besides the interesting scale of interface quality explained there, this is a very insightful comment, often overlooked:


Once the code gets too big for one person, it’s all about damage control. Interfaces make damage control possible… except when the interfaces themselves are the problem.

Designing a system as a group of people requires splitting tasks up among team members, and/or the community, and perhaps even separate teams. Interfaces are the touch points of that splitting, and is what represents the functionality offered within the module/library/file/command/service/whatever. Too often, people spend a long time working on the implementation details, thinking really deep about how to obtain the desired behavior, and forget to define clearly what is the interface to that behavior.

Having good interfaces is a key aspect of software development, and getting it correctly offers a number of important benefits:

Good encapsulation

Having good encapsulation is pretty much a synonym of having good interfaces. Too often, though, people focus on the encapsulation of the small pieces (the functions, the classes, etc), and forget about the encapsulation of the larger blocks (the libraries, modules, packages, commands, etc).

Also, in my experience trying to encourage good architectures, I have found that stating “We need good encapsulation!” gives developers no tangible line of action. It reminds me of a parent telling the child “You should be responsible!”. Sure, encapsulation and responsibility both sound great, but.. what does that really mean?

When inviting developers to think about the interfaces of the system parts they are responsible for, encapsulation becomes a natural outcome. It’s clear that there must be a line drawn between that part of the system and the rest, and the shape of this line must be considered while (or even better, before) the behavior is implemented.

Given well designed interfaces, the additional requirement of only using other parts of the system through their public interfaces seals the achievement of good encapsulation. Ideally, this barrier would be a natural property of the language used to develop the system (see the interface quality scale in Martin’s post). In other cases, this must be achieved through conventions, agreements, and good documentation.

Improved scope and communication

By inviting developers to think about the interfaces of the parts they are responsible for, one is basically encouraging the consideration of the interaction between those pieces and the rest of the system. This process gives an interesting perspective, both in terms of the external expectations (what do I need to offer other people?), as well as the internal goals (what do I need to implement for satisfying what other people need?).

Besides helping people to figure the scope and goal of the piece being developed, this will also give a nice structure to some of the communication which must inevitably happen to integrate correctly the separate parts of the system being developed.

Improved testing and experimentation

If an interface is well designed and defined, and encapsulates well part of the functionality of the system, it improves significantly the testing and experimentation related to that part of the system. Again, this has an effect internally and externally to the interface.

Internally in the sense that there’s a clear boundary between the part in development and the rest of the system, and thus it should be easier to verify that the bits which compose it are working according to plan without dragging the whole system together, and also to verify that the interface itself is behaving as intended (and hopefully as documented).

Externally in the sense that, given that there’s agreement regarding what is the public interface to the part being considered, one may easily provide a test double (a fake, or dummy, or mock) to simulate that part of the system. This is well known to be useful in a number of ways:

  • Dependent work may be run in parallel by different people
  • Real implementation backing the given interface may be postponed, until the idea is proven useful, and the interface feels suitable
  • External systems which would be hard to run locally may be simulated so that tests run fast and cheap, even without network connections
  • Faults may be injected in the system via the test doubles to verify behavior in hostile conditions

and so on.

Quality isolation

This point is also my understanding of what Rusty refers to as damage control in his talk. This property is very useful when designing a system, but even then it’s often missed when discussing interfaces and encapsulation.

If there’s a well defined interface to a piece of functionality in the system, and that interface was carefully considered to cover the needs of the system, the implementation of that interface may not start as the most beautiful, or most scalable, or even most reliable piece of software. As any developer responsible for a successful startup will happily point out, a half-baked implementation is often good enough to get things going, prove the concept, and extend the project runway.

Good interfaces play an important role in this kind of situation. They are, in this sense, a way to be better prepared for success (or, for failure, depending on the perspective). If the interface implementation suddenly becomes an issue for whatever reason, the implementation itself may be replaced by something which better suits the current reality, while preserving the interaction with the rest of the system.

Of course, it’s still very hard to predict future system behavior when facing a completely different reality. Changing the scale requirements for the system a few orders of magnitude, for instance, may easily break existing assumptions, and interfaces designed around these assumptions. Still, even if good interfaces won’t be enough to avoid modifications in the architecture and integration points in many cases, they will certainly help framing the conversations which will take place when this happens and new interfaces must be developed.

Conclusion

When developing non-trivial software products, there’s no other way but to split out the problem solving in several layers and components. Looking at the points where these layers and components touch each other is a very useful and natural way to organize conversations and structure work which must take place to push the product forward.

It’s quite revealing to look at the points above, and note that it’s not simply the existence of interfaces themselves which presents the advantages described, but the process which they encourage around them. Software architecture is essentially about people.

Read more
Gustavo Niemeyer

Recovering a bootable EBS image

Scott Moser has just announced this week that the new Ubuntu images which boot out of an EBS-based root filesystem in EC2, and thus will persist across reboots, are available for testing.

As usual with something that just left the oven and is explicitly labeled for testing purposes, there was a minor bug in the first iteration of images which was even mentioned in the announcement itself. The bug, if not worked around as specified in the announcement, will prevent the image from rebooting.

Having an bootable EBS image which can’t reboot is a quite interesting (and ironic) problem. You have an image which persists, but suddenly you have no way to see what is inside the image anymore because you can’t boot it. Naturally, even if the said bug didn’t exist in the first place, it’s fairly easy to get into such a situation accidentally if you’re fiddling with the image configuration.

So, in this post we’ll see how to recover from a situation where a bootable EBS image can’t boot.

Getting started

To start this up, we’ll boot one of the EBS images which Scott mentioned in his announcement: ami-8bec03e2. As we see in the output of ec2-describe-images, this is an EBS-based image for i386:

% ec2-describe-images ami-8bec03e2
IMAGE ami-8bec03e2 099720109477/ebs/ubuntu-images-testing/ubuntu-lucid-daily-i386-server-20100305 099720109477 available public i386 machine aki-3fdb3756 ebs
BLOCKDEVICEMAPPING /dev/sda1 snap-f1efd098 15

Let’s run this image. Remember to replace the value passed in the -k command line option with your own key pair name.

% ec2-run-instances -k gsg-keypair ami-8bec03e2
RESERVATION r-9e4615f6 626886203892 default
INSTANCE i-e3e33a88 ami-8bec03e2 pending gsg-keypair 0 m1.small 2010-03-09T20:04:12+0000 us-east-1c aki-3fdb3756 monitoring-disabled ebs

There we go. We got an instance allocated in the availability zone us-east-1c. It’s important to keep track of this information, since EBS volumes are zone-specific.

As part of the above command, we must have been allocated an EBS volume automatically, and it should be attached to the instance we just started. We can investigate it with the ec2-describe-volumes command:

% ec2-describe-volumes
VOLUME vol-edca1684 15 snap-f1efd098 us-east-1c in-use 2010-03-09T20:04:20+0000
ATTACHMENT vol-edca1684 i-e3e33a88 /dev/sda1 attached 2010-03-09T20:04:24+0000

Now, we’ll get into the running instance and do some arbitrary modifications, just as a way to demonstrate that the data we don’t want to lose actually survives the recovering operation. Note that the domain name is obtained with the ec2-describe-instances command.

% ssh -i ~/.ssh/id_dsa_gsg-keypair ubuntu@ec2-184-73-51-147.compute-1.amazonaws.com
(…)

ubuntu@domU-12-31-39-0E-A0-03:~$ echo “Important data” > important-data
ubuntu@domU-12-31-39-0E-A0-03:~$ ls -l important-data
-rw-r–r– 1 ubuntu ubuntu 15 Mar 9 20:15 important-data

ubuntu@domU-12-31-39-0E-A0-03:~$ sudo reboot
Broadcast message from ubuntu@domU-12-31-39-0E-A0-03
(/dev/pts/0) at 20:18 …
The system is going down for reboot NOW!

Note that we didn’t actually fix the problem reported by Scott, so our machine won’t really reboot. If we wait a while, we can even see that the problem is exactly what was reported in the announcement (note it really takes a bit for the output to be synced up):

% ec2-get-console-output i-e3e33a88 | tail -4
mount: special device ephemeral0 does not exist
mountall: mount /mnt [294] terminated with status 32
mountall: Filesystem could not be mounted: /mnt

Alright, now what? Machine is dead.. and can’t reboot. How do we get to our important data?

Fixing the problem

The first thing we do is to stop the instance. Do not terminate it, or you’ll lose the EBS volume! After stopping it, we’ll detach the EBS volume that was being used as the root filesystem, so that we can attach somewhere else.

% ec2-stop-instances i-e3e33a88
INSTANCE i-e3e33a88 running stopping

% ec2-detach-volume vol-edca1684
ATTACHMENT vol-edca1684 i-e3e33a88 /dev/sda1 detaching 2010-03-09T20:04:22+0000

Now, we need to attach this volume in an image which actually boots, so that we can fix it. For this experiment, we’ll pick one of the daily Lucid images, but we could use any other working image really. Just remind that the image must be running in the same availability zone as our previous instance, since the EBS volume won’t be accessible otherwise.

% ec2-run-instances -k gsg-keypair -z us-east-1c ami-b5f619dc
RESERVATION r-967427fe 626886203892 default
INSTANCE i-fd08d196 ami-b5f619dc pending gsg-keypair 0 m1.small 2010-03-09T21:10:11+0000 us-east-1c aki-3fdb3756 monitoring-disabled instance-store

% ec2-attach-volume vol-edca1684 -i i-fd08d196 -d /dev/sdh1
ATTACHMENT vol-edca1684 i-fd08d196 /dev/sdh1 attaching 2010-03-09T21:10:51+0000

With the instance running and the EBS root device attached with an alternative device name, we can then login to fix the original problem which prevented the image from booting correctly. In our case, we’ll simply do what Scott suggested in the announcement.

% ssh -i ~/.ssh/id_dsa_gsg-keypair ubuntu@ec2-204-236-194-196.compute-1.amazonaws.com
(…)
$ mkdir ebs-root
$ sudo mount /dev/sdh1 ebs-root
$ sudo sed -i ‘s/^ephemeral0/#ephemeral0/’ ebs-root/etc/fstab
$ sudo umount ebs-root
$ logout
Connection to ec2-204-236-194-196.compute-1.amazonaws.com closed.

Done! Our EBS volume is now correct, and it should boot alright. We’ll detach the volume from the temporary instance we created, and will reattach it back to the old bootable EBS instance which is stopped. Note that we won’t yet terminate the temporary instance, because we may need it in case something else is still wrong, and we are already paying to use it for the hour anyway. We just have to remind ourselves to terminate it once we’re fully done.

% ec2-detach-volume vol-edca1684
ATTACHMENT vol-edca1684 i-fd08d196 /dev/sdh1 detaching 2010-03-09T21:10:51+0000

% ec2-attach-volume vol-edca1684 -i i-e3e33a88 -d /dev/sda1
ATTACHMENT vol-edca1684 i-e3e33a88 /dev/sda1 attaching 2010-03-09T21:24:55+0000

% ec2-describe-volumes vol-edca1684
VOLUME vol-edca1684 15 snap-f1efd098 us-east-1c in-use 2010-03-09T20:04:20+0000
ATTACHMENT vol-edca1684 i-e3e33a88 /dev/sda1 attached 2010-03-09T21:24:55+0000

Okay! It should all be good now. It’s time to restart our instance, and see if it is working. Note that since you stopped and started the instance, the public domain name most probably has changed, and thus we need to find it out again with ec2-describe-instances once the instance is running.

% ec2-start-instances i-e3e33a88
INSTANCE i-e3e33a88 stopped pending

% ec2-describe-instances i-e3e33a88
RESERVATION r-9e4615f6 626886203892 default
INSTANCE i-e3e33a88 ami-8bec03e2 ec2-184-73-72-214.compute-1.amazonaws.com domU-12-31-39-03-B8-21.compute-1.internal running gsg-keypair 0 m1.small 2010-03-09T21:28:43+0000 us-east-1c aki-3fdb3756 monitoring-disabled 184.73.72.214 10.249.187.207 ebs
BLOCKDEVICE /dev/sda1 vol-edca1684 2010-03-09T21:24:55.000Z

% ssh -i ~/.ssh/id_dsa_gsg-keypair ubuntu@ec2-184-73-72-214.compute-1.amazonaws.com
(…)
$ cat important-data
Important data

$ logout
Connection to ec2-184-73-72-214.compute-1.amazonaws.com closed.

It worked, and our important data is still there!

Don’t forget to kill the temporary instance you’ve used to fix it after you’re comfortable with the result:

% ec2-terminate-instances i-fd08d196
INSTANCE i-fd08d196 running shutting-down

Conclusion

Concluding, in this post we have seen how to fix a bootable EBS machine which can’t actually boot. The technique consists of detaching the volume from the stopped instance, attaching it to a temporary instance, fixing the image, and then reattaching it back to the original image. This back and forth of EBS volumes is quite useful in many circumstances, so keep it in your tool belt.

Read more
jono

Those lovely people at ZDNet have let me write a guest post about community, and I have contributed an article called Failure as a springboard to success.

The article talks about how if we embrace our failures in our communities, it can help us become better community members, and provides a model for doing so. This is particularly important for leaders. In a nutshell (from the article):

Failure should [also] be embraced in your communities. We admire leaders who are humble, honest and frank, and we grumble about leaders who are defensive and abrasive. Be the former, and your community will love you for it.

If you are involved in a community, I recommend you give it a read. :-)

Read more