Canonical Voices

Posts tagged with 'projects'

Curtis Hovey

Launchpad’s bug and branch privacy features were replaced by information sharing that permits project maintainers to share kinds of confidential information with people at the project level. No one needs to manage bug and branch subscriptions to ensure trusted users have access to confidential information.

The Disclosure features

Disclosure is a super feature composed on many features that will allow commercial projects to work in private. Untrusted users cannot see the project’s data. Project maintainers can share their project with trusted users to reveal all or just some of the project’s data. The ultimate goal is to create private project in Launchpad, but that feature required several other features to be completed first. The Purple squad worked on Trusted Pickers, Privacy Transitions, Hardened Projects, Social Private Teams, and Sharing.

There was a lot of overlap between each feature the Purple squad worked on. Though we could start each feature independent of one another, we could only complete about 90% of each. When the Sharing UI changes entered beta, we were unblocked and fixes about most of the remaining issues, but fixing all the issues required all projects to switch to Sharing.   We did not consider Sharing, or any of the required features complete until we fixed all the bugs.

Disclosure facts

  • Planning started in June 2010 to replace the existing privacy mechanisms with something that would scale.
  • Early testing revealed that users did not trust Launchpad because the UI could not explain what was confidential, or what the consequences of a change would be — this needed to be fixed too.
  • 149 related bugs were identified in Launchpad.
  • Work started in June 2011 by the Purple squad.
  • Replacing the old privacy mechanisms and addressing the trust and information issues took 16 months.
  • About 45,000 lines code were added to support the features.
  • About 15% of the lines were for missing JavaScript test coverage.
  • More that 700 bugs were fixed in total.
  • About 5% of the fixed bugs were caused by the old non-scaling privacy mechanisms.
  • About 4% of the fixed bugs were caused by old JavaScript enhancements that broke features for non-JavaScript users.

Lessons learned

  • Misrepresentation of what is confidential, or what will be confidential or public is very important to users — more important than supporting private data.
  • Privacy/Sharing must be a first-class mechanism beneath all the mechanisms that work with confidential data.
    • Privacy was added on top of bugs, and it failed to scale to 100′s of bugs.
    • Privacy was added on top of branches, and it failed to scale to 1000′s of branches.
    • Filtering private items in code, or in database joins is not fast enough to work with 100,000′s of items.
  • Launchpad’s ReSTful object API is not suitable for working with large collections of objects like bugs or branches; a lighter, service-based approach was used to quickly work with large amounts of data.
  • Users need to work with confidential data via the API, using a text web browser from servers, using a browser with accessibility tools, as well as the common case of using a JavaScript enabled browser.
  • Lots of mock-ups and interactive tests will not predict all the interactions a user will have with real data; test with real code and data early to developer the final design.

Read more
Curtis Hovey

Launchpad’s bug and branch privacy features are being replaced by information sharing that permits project maintainers to share kinds of information with people at the project level. No one needs to manage bug and branch subscriptions to ensure trusted users have access to confidential information.

Maintainers can share and unshare their project with people

Project maintainers and drivers can see the “Sharing” link on their project’s front page. The page lists every user and team that the project shares with. During the transition period of the beta, you might see many users with “Some” access to “Private Security” or “Private” user information. They have this access because they are subscribed to bugs and branches. Maintainers can unshare with users who do not need access to any confidential information, or just unshare a bug or branch with a user. Maintainers can share share with a team to give them full access to one or more kinds of confidential information.

I have prepared a video that demonstrates the features (my apologies for the flickering)

Commercial projects can set bug and branch policies

Projects with commercial subscriptions can also change bug and branch sharing policies to set the default information type of a bug or branch, and control what types they may be changed to. Maintainers can set policies that ensure that bugs and branches are proprietary, and only proprietary, to ensure confidential information is never disclosed.

Sharing can be managed using API scripts

I maintain many project which have a lot of private bugs and branches. The sharing page lists a lot of people, too many to read quickly. I know most work for my organisation, but I don’t even know everyone in my organisation. So I wrote a Launchpad API script that can be run by any project maintainer to share the project with a team, then unshare with the team members. The members still have access to the bugs and branches and their subscriptions still work, but they will lose access to my project when they leave the team. This arrangement makes it very easy to manage who has access to my projects. share-projects-with-team.py is run with the name of the team and a list of projects to share with it.

./share-projects-with-team.py my-team project1 project2

Read more
Curtis Hovey

Nothing breaks my heart quite like a request to make a project private–make it invisible to everyone except to the people the project trusts. I am utterly crushed when someone who works for Canonical or on Launchpad asks for one. I have been planning this feature for more than two years, and the Purple squad has been working on it for 13 months. I blog about this, I send emails about this, I present reports on this, but the people who most need private projects don’t know what the Purple squad is doing. I think the problem here is that Launchpad squads no longer use Launchpad to plan and execute work. There is no place for any interested party to see what the goals of Disclosure is and gauge how we are progressing.

I present my first draft of a report that states the simple goals of that the Disclosure feature wants to achieve . The report provides some summaries of the work that allows anyone to see what the Purple squad is doing, recently done, and will do next. There is also some analysis that provides insight into the amount of work remaining. This report complements the Purple squad’s kanban board. While kanban is excellent for tracking branches of code and technical tasks, the level of detail is unsuitable for non-Launchpad developers. The kanban board is also only accessible to a small number of people. I want a report that anyone interested in private projects or managing the disclosure of private information can see and understand. Mostly, I want everyone to see that the Purple squad is delivering valuable features and know when we will be done.

I based the report on the intended reporting UI for Launchpad series and milestones. I really miss using series and milestones to plan releases. For every milestone, I wrote our goals in the summary, and targeted bugs to the milestone. Though we abandoned the analytics because of performance concerns, I could reliably judge  the contributors’ velocity, and see when I needed to retarget work to another milestone because the remaining effort exceeded the milestone’s work capacity. Though I didn’t provide a burn down chart of the work, I could sort the milestone to see the colour change. I could confidently see and predict 3 months of work.

This report replaces the canvas-based chart I planned for series and milestones with a YUI 3 chart. The listing of bugs are split into categories so that I can focus on scheduling or provide Diogo with a list of bugs that need exploratory testing. Though this report thinks it is talking to Launchpad, it is actually using JSON for the 500+ bugs that I pulled using a trivial Launchpad API script. Since the data is cheap to retrieve, I can load the chart multiple times, each looking a different set of bug tags so that I can see specific themes of work.

The report shows that there is more than 60 days of work to complete the features needed by private projects. The Orange squad will work on private projects while the Purple squad finishes the prerequisites.

 

Read more
Curtis Hovey

teams

We want the project maintainer to be the default party that the project shares private information with. The problem at the moment is that Launchpad does not know how to set a team as the project maintainer during setup. Improper project setup is the root cause of most cases where information is disclosed to the wrong people. We need to improve project registration and setup to ensure users can ensure private information is managed properly. This issue is complicated by a very old issue, it is not possible to register a team at the moment you discover you need one. Launchpad must let me register a new team that will maintain my project when I first setup my project.

The Purple Squad discussed what we can do to simplify team registration and perform the registration in any page that allows you to set a team. We discovered several areas where we can make improvements.

  • Do not ask for non-essential information like contact address.
  • We can simplify the team membership policy language.
  • We can fix the confusion about membership renewal.
  • Launchpad can pre-fill the form with sensible defaults when the team will be used in a role.

Ian put together a demonstration to prove we could extend the person picker to also permit you to register a team.

When you want to set the project maintainer to a new team, Launchpad will ask you to confirm its suggestions for the Launchpad Id, display name, and membership policy. You can change the values, but most of the time you will choose to continue, and Launchpad will register the team and place it in the role.

 

Read more
Curtis Hovey

Setting up a commercial project in Launchpad has gotten easier. You can now quickly register a proprietary project and enable private bugs. You can create private teams and private personal package archives, AKA private PPA or P3A without the assistance of a Launchpad admin.

When you select the Other/Proprietary license while registering a project, or changing the project’s details,

it is given a complimentary 30-day commercial subscription.

The delay between the moment when a commercial project was registered and when the commercial subscription was purchased and then applied to the project caused a lot of confusion. During this delay, proprietary data could be disclosed. We chose to award the project with a short term commercial subscription which enabled the project to be properly configured while the 12-month commercial subscription was being purchased and applied to the project.

Any project with a commercial subscription can enable

Default private bugs
Once enabled by configuring the project’s bug tracker, all new reported bugs are private. You can choose to make the report public.
Default private bugs
Default private branches
You can request a Launchpad admin to configure private branches for your teams. (You will be able to do this yourself in the near future when projects gain proprietary branches.)

As the maintainer of a project with a commercial subscription, you can register

Private teams
When you register a team, you can choose to set the team visibility to private. The team’s members and data is hidden from non-members.
Private mailing lists
When you create a mailing list for a private team, the archive is also private. Only team members may see the messages in the archive.
Private PPAs
When you create a PPA for your public team, you may choose to make it private; private teams can only have private PPAs. You can subscribe users to your archive so that they may install packages without revealing all your team’s members and data to the subscriber.

A secondary benefit of this change is that you can now try Launchpad’s commercial features before purchasing a 12-month commercial subscription. The features will be disabled at the end of 30-days. Your test data will remain private to ensure your data is not disclosed.

Any open source project may also have a commercial subscription to enable commercial features. You can purchase a commercial subscription at the Canonical store. Commercial subscriptions cost US$250/year/project + applicable V.A.T.

 

(Photo by Fred Dawson on flickr, creative commons license)

Read more
Curtis Hovey

Launchpad can now show you all the people that your project is sharing private bugs and branches with. This new sharing feature is a few weeks away from being in beta, but the UI is informative, so we’re enabling this feature for members of the Launchpad Beta Testers team now. If you’d like to join, click on the ‘join’ link on the team page.

What you’ll see

Project maintainers and drivers can see all the users that are subscribed to private bugs and branches. The listing might be surprising, maybe even daunting. You may see people who no longer contribute to the project, or people you do not know at all. The listing of users and teams illustrates why we are creating a new way of sharing project information without managing bug and branch subscriptions.

If you’re a member of (or once you’re a member of, if we want people to join) the Launchpad Beta Testers team, you can find the Sharing link on the front page of your project. I cannot see who your project is sharing with, nor can you see who my projects are sharing with, but I will use the Launchpad project as an example to explain what the Launchpad team is seeing.

The Launchpad project

The Launchpad project is sharing private bugs and branches with 250 users and teams. This is the first time Launchpad has ever provided this information. It was impossible to audit a project to ensure confidential information is not disclosed to untrusted parties. I still do not know how many private bugs and branches the Launchpad project has, nor do I even know how many of these are shared with me. Maybe Launchpad will provide this information in the future.

Former developers still have access

I see about 30 former Launchpad and Canonical developers still have access to private bugs and branches. I do not think we should be sharing this information with them. I’m pretty sure they do not want to notified about these bugs and branches either. I suspect Launchpad is wasting bandwidth by sending emails to dead addresses.

Unknown users

I see about 100 users that I do not know. I believe they reported bugs that were marked private. Some may have been subscribed by users who were already subscribed to the bug. I can investigate the users and see the specific bug and branches that are shared with them.

The majority

The majority of users and teams that the Launchpad project is sharing with are members of either the Launchpad team or the Canonical team. I am not interested in investigating these people. I do not want to be managing their individual bug and branch subscriptions to ensure they have access to the information that they need to do their jobs. Soon I won’t have to think about this issue, nor will I see them listed on this page.

Next steps — sharing ‘All information’

In a few weeks I will share the Launchpad project’s private information with both the Launchpad team and the Canonical team. It takes seconds to do, and about 130 rows of listed users will be replaced with just two rows stating that ‘All information’ is shared with the Launchpad and Canonical teams. I will then stop sharing private information with all the former Launchpad and Canonical employees.

Looking into access via bug and branch subscriptions

Then I will investigate the users who have exceptional access via bug and branch subscriptions. I may stop sharing information with half of them because either they do not need to know about it, or the information should be public.

Bugs and private bugs

I could start investigating which bugs are shared with users now, but I happen to know that there are 29 bugs that the Launchpad team cannot see because they are not subscribed to the private bug. There are hundreds of private bugs in Launchpad that cannot be fixed because the people who can fix them were never subscribed. This will be moot once all private information in the Launchpad project is shared with the Launchpad team.

Unsubscribing users from bugs

Launchpad does not currently let me unsubscribe users from bugs. When project maintainers discover confidential information is disclosed to untrusted users, they ask the Launchpad Admins to unsubscribe the user. There are not enough hours in the day to for the Admins to do this. Just as Launchpad will let me share all information with a team or user, I will also be able to stop sharing.

 

(Image by loop_oh on flickr, creative commons license)

Read more
Curtis Hovey

Social private teams

The title may sound like a contradiction, but I assure you that it is not. You can now use private teams in social contexts in Launchpad. Private teams can collaborate with public projects and other teams if they choose to reveal the existence of the private team.

  1. Private teams can me members of other teams.
  2. Other teams can be members of private teams.
  3. Private teams can subscribe to bugs, blueprints, branches, and merge proposals
  4. Private teams  can be in public project roles,  such as maintainers, drivers, and bug supervisors.
  5. You can change the branch owner to be a private team.
  6. Private team personal branches (+junk) are always private.

When a member places the private team in a social situation, a the member is asked to confirm that it is okay to reveal the team’s identifying information. This information is the team’s Launchpad Id used in URLs, the displayname, icon, and logo. A user can visit the private team’s Launchpad page and will only see that information. The rest of the page is not shared. Anonymous users cannot see a private team’s page because that user is not being social; logged in users can see the private team’s page

Private team page seen by a non-member

Launchpad did not permit these interactions previously because it was not clear who should know about the team. Someone has to know. If Launchpad permitted private teams to subscribe to bugs or be members of teams without anyone knowing of them, they would be unaccountable. Private teams could spy on organisation, or learn about security vulnerabilities to exploit. Launchpad will not ever permit such asocial behaviour. The resolution for social interactions was to permit other parties to know enough of the private team to make an informed decision. For example, when I choose to make a bug private, I want to know who was already seen the bug through a subscription. I may choose to unsubscribe the private team if I do not trust them.

Private teams may invite exclusive teams to be members. Exclusive teams (moderated or restricted teams) review all members so they are accountable. If a team admin trusts the admins of another team, and that team is accountable, Launchpad permits the other team to be a member. This is actually a rule that applied to all exclusive teams. private teams are always exclusive (restricted membership policy). The only nuance with private teams is when it is a member of another team; the super team may know the members of the private sub team because the super team has the right to audit all its members so that it too can be accountable.

Read more
Curtis Hovey

Launchpad will soon permit you to say your project is not affected by a bug shared with another project — you can delete the spurious bug task. This action can be done from the bug’s web page, and using Launchpad API.

You can remove a project from a bug if you are the project maintainer, bug supervisor, or are the person who added the project to the bug. This action can only be performed when the bug affects more than one project — you cannot delete an entire bug. This feature permits you to undo mistakes.

Launchpad beta testers will see the remove action next to the affected project name in the affects table.

Delete spurious bugs

The delete() method was added to the bug_task entry in the Launchpad API. There is a example API script, delete_bugtasks.py, that can remove a project from many bugs. There is also a split action to create a separate bug just for the specified project to track separate conversations in bug comments.

Usage: delete_bugtasks.py [options] bug_id:project_name [bug_id:project_name]

Options:
  -h, --help            show this help message and exit
  -v, --verbose
  -q, --quiet
  -f, --force           Delete or split bug tasks that belong to a public bug
  -d, --delete          Delete a spurious project bug task from a bug
  -s, --split           Split a project bug task from an existing bug so that
                        the issue can be tracked separately
  -w WEBSITE, --website=WEBSITE
                        The URI of Launchpad web site.
                        Default: https://api.launchpad.net;
                        Alternates: https://api.staging.launchpad.net,

https://api.qastating.launchpad.net

Previously, you could not remove spurious bug reports about your project. Many were cause by poor bug target management; you could not move a bug between projects and distributions. You can now move bugs between projects and distributions, but thousands of bugs still wrongly claim to affect a project or distribution. This causes clutter on bug pages and searches, and it causes Launchpad performance problems.

This change is a part of a super-feature called Disclosure. To ensure that confidential data is not accidentally disclosed, Launchpad will only permit private bugs to affect a single project. Soon, you may need to remove a project from a bug before marking the bug private.

Read more
Matthew Revell

Nikki and the Robots

Nikki and the RobotsNikki and the Robots is the first game from Berlin based games studio Joyride Labs. It’s a retro-styled platformer with beautiful colours and an open source licence, with bugs tracked in Launchpad.

I asked Iwan & Sönke from Joyride Labs about the game.

Matthew: Nikki and the Robots is LGPL and Creative Commons licensed. What made you choose open source licences?

Iwan & Sönke: This one is easy! Our love of open source software and free art made us do it! Also we expect that being open will get us additional attention and love.

Our work and how we license it is an experiment too, so actually we ‘hope’ rather than expect. :)

Matthew: How are you planning to distribute the game once it’s ready?

Iwan & Sönke: We are working on the first part of Nikki and the Robots and will start in-dev-sales (pre-sale) once it is ready.

Game, editor and user levels will be free as in freedom. A part-proprietary organic story mode with enhanced levels and additional in-game-art will be available for pay.

Matthew: Can you tell me a bit more about the technology behind the game and the decisions you made? I see, for example, you’re using Haskell.

Iwan & Sönke: Haskell is just my (Sönke’s) favourite language. I have the impression that I can do coding in Haskell much faster than in other languages and I produce less bugs. Besides Haskell we use Qt and OpenGL as the graphics backend and Chipmunk as the physics engine. By the way: we completely rely on free software for compilation on all platforms (so, no VisualStudio or XCode, just gcc, cmake, mingw32, etc.).

Matthew: How are you finding Launchpad as a bug tracker?

Iwan & Sönke: It’s quite usable. Forum and wiki might be useful features to consider for the platform. (We use a free wiki system and will probably set up an own forum soon).

Matthew: Thanks to Iwan and Sönke. You can download an alpha of Nikki and the Robots from the Joyride Labs website.

Read more
Matthew Revell

Beer as in beer

JolieBulle logoThere’s quite a bit of overlap between home beer brewing and hacking. Both usually involve experimentation, sharing and a love of what you’re doing.

It’s not surprising, then, that there’s more than one open source project aimed at helping home brewers to create the beer they want. A few home brewing projects are hosted on Launchpad, including JolieBulle (that’s French for “pretty bubble”).

Home brew ... cider in this casePierre Tavares started the project last year to support his own brewing. The result is an application that helps at every stage of the process. When you’re ready to get going, it helps formulate the recipe and allows for the sharing of recipes using the common BeerXML standard. It helps calculate what brewers call the beer’s profile (its bitterness, colour, how much alcohol it has), includes an ingredients database and has tools that help during the brewing process itself.

I emailed Pierre to ask about the development of JolieBulle. Here’s what he said:

From a technical point of view, JolieBulle is developped in pyQt and integrates well in both KDE and Gnome desktops.

I chose Launchpad mainly for the openness of the platform, and the great tools to manage code, bugs and blueprints. I’m pretty new to DVCSes but Bazaar seems fine, and I have no problem using it. I don’t use the translation tool, as I prefer Qt Linguist.

JolieBulle isn’t yet packaged for any distros and Pierre hopes to attract contributors who can help with that.

Photo by Sizbut. Licence CC BY 2.0.

Read more
Matthew Revell

Doing it for the Luz

Luz is a new project to Launchpad. It creates impressive visual effects that can react to music or be driven by a person using a MIDI controller or a gamepad.

It has been created by Ian McIntosh, part of a Portland, Oregon, artists collective who produce light and projection shows.

Here’s what the Luz page on the Light Troupe site has to say:

With one click, any movement or effect can dance to the beat, react to audio, or be driven directly by human input from any number of any device: Gamepads & Joysticks, MIDI knobs & sliders, MIDI Pianos & Drums, WiiMotes, Wacom Tablets, and any app that can send OpenSoundControl.

Ian has provided a handy series of YouTube tutorials, to get you started. If you want to try it out, here’s the first of those tutorials:

Read more
Matthew Revell

Gufw in Launchpad

GUFW's logoIf you’re an Ubuntu user, it’s likely that you’ve used UFW — the Uncomplicated Firewall — and maybe also its graphical front-end, Gufw.

I spoke to Marcos Costales from Gufw to ask about the project and their use of Launchpad.

Matthew: What prompted you to start the project?

Marcos: In 2008 I was translating and testing, but I wanted to contribute more. When Canonical released ufw, I read some reviews that a firewall using the shell was not an uncomplicated firewall; reviews did not see that ufw was perfect for servers and a base with great ideas for something else, I saw a clear opportunity for the development of an application like Gufw.

Matthew: How many of you are working on it?

Marcos: After the first week following release, Vadim joined the project and we were able to generate a lot of expectation. It was amazing to see how Gufw grew exponentially thanks to the contribution of the community. An important point was Cedrick’s mockup for setting a good interface.

From here I would like to thank Vadim, Emilio, Raúl, Planella, Rubén, Rogério, Cedrick and Devid, Gufw is what it is thanks to them.

We are currently working on an application that really simplifies the task of setting up a firewall: very minimalistic, easy to use and understand.

Matthew: How much connection is there between the Gufw and UFW communities?

Marcos: From Gufw we pay attention to the next ufw versions and we request changes we need. We want to thank Jamie for his effort and interest in all of our requests.

There is also an awesome work of documentation taking place in the Ubuntu Wiki.

Matthew: Does Launchpad help the two projects work together?

Marcos CostalesMarcos: Yes, we can follow the blueprints of the next ufw version and review which changes could affect Gufw. We can easily reassign ufw and gufw bugs…

Matthew: Why did you choose Launchpad?

Marcos: Gufw started in Google Code. After one month we moved to Launchpad, and it certainly was one of our best decisions because it offers us all necessary services and an amazing integration between them.

Matthew: What’s your favourite Launchpad feature?

Marcos: Translations. I am a translator in external projects to Launchpad, and I must say that the usability and simplicity of Launchpad is unique.

Matthew: What would you most like to see added to Launchpad?

Marcos: Maybe a small space for a custom website. We must use an external hosting for the main project website.

Matthew: What do you think is Launchpad’s biggest weakness?

Marcos: At first Launchpad seems overly complex and big (compared for example with Google Code). One idea would be to have two views according to the needs/experience of the project: the current one and an easier to use one.

Matthew: Are you looking for people to contribute?

Marcos: Of course we are. There are always pending tasks for anyone who wants to help and everyone is welcome.

Matthew: Thanks Marcos!

Read more
Matthew Revell

Tomdroid: Tomboy notes for Android

I like to use Tomboy for just about any time I need to make a note. Shopping lists, a GTD collection bucket, notes from a phone call: Tomboy’s ideal.

Actually, up until recently I didn’t use Tomboy for shopping lists all that often. Making the list was fine but getting it to the supermarket usually meant printing it out, or something else that wasn’t quite as convenient as I’d like.

Baskets only

Then I installed Tomdroid.

Tomdroid does one thing and it does it well: it synchronises your Tomboy notes from elsewhere and lets you read them on your Android phone. You can import your notes from an SD card or, more usefully, synchronise with a TomboyWeb provider such as Ubuntu One.

Now I can tap out my shopping list on a real keyboard and carry it with me without a second thought. And, of course, all those other notes that make my life run smoothly are there with me wherever I am.

Tomdroid uses Launchpad for bug tracking, code hosting, blueprints and questions/answers.

You can download the latest version from the Android Market.

Photo by Tristram Biggs. Licence: CC BY-ND 2.0.

Read more
Matthew Revell

Nautilus Terminal

Nautilus Terminal in action

If you’re a Gnome user and have watched with envy as your KDE4-using friends effortlessly open a terminal directly in their file-browser, you may be interested in Nautilus Terminal.

Fabien Loison is behind Nautilus Terminal. I asked him a little about the project.

Matthew: What were you doing when you realised that life would be easier if you had a terminal in Nautilus?

Fabien: I was programming and I had a lot terminals open on different folders. I realized I was losing a lot of time to find the one I wanted, and then I remembered that Midnight Commander has an interesting feature: it permits you to enter commands in the current folder. I searched on the web and saw that (KDE 4’s file manager) Dolphin offers this kind of functionality, but nothing about Nautilus… So I decided to do it myself: I started programming Nautilus Terminal.

Matthew: And are you happy with the result?

Fabien: Although Nautilus Terminal is not as well integrated with Nautilus I would like (due to limitations of its extension system), I think I have solved my problem. :)

Matthew: What sort of reaction have you had?

Fabien: Most reactions were positive: since the day of the first release I have received many emails and also some blogs have written about Nautilus Terminal (WebUpd8, OMG Ubuntu,…). It seems that many people wanted this feature in Nautilus.

Matthew: So what made you choose Launchpad?

Fabien: I had already been using Launchpad for other projects for several months, and I like it (especially its integration with Bazaar), so I used it one more time. :)

Matthew: What has been the most useful part of Launchpad?

Fabien: The most useful part of Launchpad for this project has been the bug tracker, because there were a lot of problems in the first versions.

Matthew: And, similarly, where would you like to see Launchpad improve?

Fabien: That is a difficult question… Maybe having a small wiki for every projects (for the documentation).

Matthew: Finally, are you looking for contributions from other people?

Fabien: Yes, especially for the translations, because I can’t do it myself for all languages (I can translate in French only). So thanks to all translators (and all the people who have helped with code, bug reports,…). :)

Matthew: Thanks Fabien!

Visit Nautilus Terminal in Launchpad.

Read more
Matthew Revell

The Economist and Launchpad

Economist logoThe online team at The Economist recently set up a Launchpad project, using a commercial subscription. I asked Mark Theunissen, from The Economist Group, about their plans.

Mark: We’re migrating the existing Economist.com stack from Coldfusion/Oracle to a LAMP stack running Drupal. At present, we’re about half way through — if you visit a blogs page, channel page, or comments page they will be served from Drupal, but the home page and actual articles are still served from Coldfusion. There’s a migration and syncronisation process happening in the background between Oracle and MySQL.

Matthew: Is much of your web infrastructure based on open source software? If so, what?

Mark: Our new stack sure is! :) We run almost all open source, in fact I can’t think of anything that isn’t.

  • Redhat Linux servers throughout (not Ubuntu, unfortunately).
  • MySQL enterprise database.
  • PHP 5.
  • Varnish HTTP accelerator.
  • Drupal content management system. Actually, a distribution called Pressflow.
  • Memcached for caching.
  • BCFG2 for configuration management.
  • The Grinder for load testing.

Matthew: Do you customise much of that?

Mark: We do, yes. We’ve sponsored or contributed patches that have mostly been for Drupal but also made their way into Varnish & BCFG2. We use Pressflow, and our changes go there first and often get back ported into core Drupal. Our policy is to open source as much as humanly possible!

Matthew: And, of course, I’d love to know what made The Economist choose Launchpad.

Mark: We chose Launchpad for its usability, mostly the workflow around reviewing code (merge proposals). It provides excellent tools for managing distributed teams, and we are a very large distributed team, with three locations where development is occurring on either side of the Atlantic.

The integration with Bazaar is great, and we are going to consider moving our bug tracker to Launchpad too at some time in the future.

Matthew: Thanks Mark!

Read more
Matthew Revell

Mopedix

Thomas Pietrowski is an 18 year old high school student in Solingen, Germany, whose mopedix project aims to create a control system for mopeds and other vehicles.

Matthew: Tell us more about mopedix

Thomas: My software system is split into two different parts.

First of all there is the control system, which is using an Arduino to control relays and H-bridges that also can dim light. This project is still under construction and soon I’ll be adding functions such as locking via a RFID reader and activating an alarm using an accelerometer.

But the important part of the project is the interaction between the control system and the computer. The client can be used for setting your values, e.g. when you want the moped to turn on its lights according to the intensity of the ambient light. In general it sends commands via serial connection and this gives the project a wide range of ways to communicate with the Arduino used to control the moped’s lights.

At the moment I am working with the Arduino Duemilanove, which has a built-in USB-to-serial adaptor. But there is also the possibility of communicating using a cable, e.g. RS232 or simply two wires, one for incoming data and one for outgoing data, or wireless via Bluetooth [50-100m], radio frequency [2-10m], Xbee[>90m-1.6km (1 mile)] and many more. Using Bluetooth you can also change your settings using a Nokia mobile phone running PyS60 or other devices that can send commands via the serial connection to the control system.

But to be honest, the control system has not been tested on the vehicle, as I still need to transport it from my grandparents in Poland to Germany. However, I’ve tested that it works in theory.

Matthew: What prompted you to start writing software that interacts with your moped?

Thomas: I decided to start the project in the end of 2009 when I looked on the web to see if there was something similar already available. I found some discussions in forums and elsewhere but didn’t come across anything that really did what I wanted.

The most important thing that interested me was how much such a control system would cost, because prices on the market are in the most cases not very realistic. Imagine a radio frequency controlled locking system kit for your car. Such a kit can cost around 100 Euros. Now think about making such a locking system on your own using Bluetooth and your mobile phone.

Here is a short calculation what you would need:

  • a bluetooth-serial module (14 Euros from China)
  • an Arduino board (20 Euros)
  • a 6v relay that can handle voltages of 12v (2 Euros)
  • and some other typical things like a few transistors, resistors, cables, and a solder clip (5-7 Euros).

That makes a total of 41 to 43 Euros and some hours programming your Arduino and a little Python script for your Nokia S60 mobile phone.

So, can you see it can be cost effective to offer such a system.

Another thing is that I had an Arduino Duemilanove and had been developing some applications in Python for my personal use, so I had the most important things that were needed to start my project: the resources and the knowledge.

Because of the great community, lots of Arduino users, a detailed instructions for programming the Arduino and my own Python experience, I decided to give it a go.

Matthew: What sort of interface does a moped have that allows you to hook it into a computer?

Thomas: You can use mopedix in general on any vehicle you want. I aimed it at mopeds because, as an 18 year old student, I only have a moped and access to its schematics.

By reading its schematics I noticed that the heart of my moped was actually the ignition lock. The control system that I am working on is nothing more than a digitally controlled ignition lock”, which will replace the old one and provide an interface for the client application.

The system will be powered on my moped with 6 volts and newer mopeds that have 12 volts will need at least a 9V rectifier for use with the Arduino Duemilanove or a 5V rectifier for the Arduino Mini. An Arduino Mini will be great for the control system because I believe that there will be also vehicles that will have room available than in my moped.

It should be possible to use the same software and control system on a car. So, for the future I should think about a project name that isn’t specific to a category of vehicle.

Matthew: Is there any other software, proprietary or open, that does a similar job?

Thomas: Not that I know. I found, via Google, a patent that describes a control system for bicycles, but I am sure that there is no other software, neither proprietary nor open, that can be compared with mine.

In addition to that I would really like to learn from people who are familiar in working on free software and earning money for their work to show me how to earn money on my own for buying additional modules, like a bluetooth-serial-module, to improve my software and provide the end-user a wider range using the control system.

Matthew: Why did you choose Launchpad?

Thomas: I chose Launchpad because I worked on packages that are hosted in my PPAs for the Canola project (that page is a little outdated) and I have been helping to test unstable Ubuntu packages since Lucid Alpha 3.

So I was already familiar with Launchpad but the main reason for choosing Launchpad was that it gives the possibility to other people translating my client application into their languages from English.

Moreover it gives the possibility for the end-user to get in contact with the project using the “Answers” tab on the project page or report errors by using the “Bugs” tab. As well as that the user is able to follow the future of the project by the “Blueprints” tab. But I haven’t used this feature yet because I didn’t think that there were people who would be interested in my project.

The only reason I don’t use Bazaar is that I’ve only previously worked with Subversion. I hope to take some time to learn Bazaar, in the future, switch over from Subversion.

Matthew: Thanks for telling us about mopedix!

Read more
Matthew Revell

Sikuli — scripting your use of GUIs

Sikuli logo
The Sikuli project recently switched to using Launchpad. I asked Tsung-Hsiang Chang to tell me more about the project.

Matthew: Your website says, “Sikuli is a visual technology to search and automate graphical user interfaces (GUI) using images (screenshots).” That sounds like it’s a general purpose screen-scraping solution. Is that right?

Tsung-Hsiang: Not exactly. Sikuli is not about extracting data from screen.

The current release of Sikuli is called Sikuli Script, which focuses on only automation using screenshots of GUI widgets.

We have another project called Sikuli Search, which queries a search engine using screenshots instead of keywords.

Although Sikuli Script is supposed to be able to “search” buttons or text on the screen, it isn’t good at scraping or analyzing information from screenshots yet.

Matthew: In your YouTube demo video, you show how you can write a Sikuli script that will open an app by name. Then, you can take a screen shot of one of the icons in that app and have Sikuli click it as part of the script. You even show how you just have to take a screen shot of the option you want from a drop-down list and Sikuli will select that option. That’s really cool but you say that Sikuli will even tolerate small changes in the icons you ask it to click. How does that work?

Tsung-Hsiang: Matching between a target image and the screen image is done by computing the normalized cross-correlation between the two images.

This is a standard technique in computer vision for finding patterns when variations are known to be small. This technique works incredibly well for matching desktop GUI patterns.

Matthew: Where do you think Sikuli will be most useful?

Tsung-Hsiang: Whenever the internal API of an application is not exposed. Lots of people have created their scripts to play the applications that were used to be very difficult to be automated, such as facebook (flash) games and testing android systems.

Matthew: Particularly on Mac OS X and Linux-based systems, where does Sikuli become a better option than standard shell scripting?

Tsung-Hsiang: If a command line interface is available, Sikuli may be not a good choice for a shell scripting guru. But sometimes you just can’t find command line tools or don’t want to learn complicated commands and parameters. In fact, the core of Sikuli is just a Jython library, so it can be mixed with other Python scripts or command line tools easily. Therefore, Sikuli can be an additional handy tool for command line gurus.

Besides, the primary goal of Sikuli is to help ordinary users who know nothing about command line tools and shell scripting to automate their tasks. We hope everyone can enjoy using computer efficiently.

Matthew: What’s next for Sikuli?

Tsung-Hsiang:Better, faster, more accurate.

We have a long list of planned improvements for Sikuli. Among the top of list are:

  1. Social programming: the ability to share scripts and search scripts by visual patterns. For example, when a user takes a screen shot of a recycle bin, the user can search a database for all the other scripts written by other users that involve the image of a recycle bin.
  2. Event-driven programming: the ability to register event handlers to handle visual events. For example, a user can define a function to pop up a warning message to handle the visual event that involves the appearance of the “low battery image”.
  3. Face detection: the ability to find faces on the screen.
  4. Recorder: the ability to record a sequence of clicking and typing operations and generate a visual script automatically.
  5. Tutorial converter: the ability to convert an existing step-by-step instruction with screenshots into executable scripts.

Matthew: Why did you choose Launchpad?

Tsung-Hsiang:We were using Trac before moving to Launchpad. Trac is more developer-oriented, just like Github.

But what we really want is a user-oriented project hosting site. We want a place to report and discuss bugs, ask and answer questions, and also download and track the development of Sikuli.

We compared Github and Launchpad, and at last chose Launchpad over Github because Launchpad has almost everything we need, except a wiki for writing documents. But we already had a wiki in our Trac system, so this was not a big problem.

Matthew: Do you have any requests for the Launchpad community?

Tsung-Hsiang: It would be great if we can write documents on Launchpad. A wiki or something like EtherPad would be fantastic.

Matthew: Thanks very much and good luck with Sikuli!

Read more