Canonical Voices

Posts tagged with 'ubuntu'

admin

Hello MAASters!

I’m happy to announce that MAAS 2.3.0 (final) is now available!
This new MAAS release introduces a set of exciting features and improvements to the overall user experience. It now becomes the focus of maintenance, as it fully replaces MAAS 2.2
In order to provide with sufficient notice, please be aware that 2.3.0 will replace MAAS 2.2 in the Ubuntu Archive in the coming weeks. In the meantime, MAAS 2.3 is available in PPA and as a Snap.
PPA’s Availability
MAAS 2.3.0 is currently available in ppa:maas/next for the coming week.
sudo add-apt-repository ppa:maas/next
sudo apt-get update
sudo apt-get install maas
Please be aware that MAAS 2.3 will replace MAAS 2.2 in ppa:maas/stable within a week.
Snap Availability
For those wanting to use the snap, you can obtain it from the stable channel:
sudo snap install maas –devmode –stable

MAAS 2.3.0 (final)

Important announcements

Machine network configuration now deferred to cloud-init.

Starting from MAAS 2.3, machine network configuration is now handled by cloud-init. In previous MAAS (and curtin) releases, the network configuration was performed by curtin during the installation process. In an effort to improve robustness, network configuration has now been consolidated in cloud-init. MAAS will continue to pass network configuration to curtin, which in turn, will delegate the configuration to cloud-init.

Ephemeral images over HTTP

As part of the effort to reduce dependencies and improve reliability, MAAS ephemeral (network boot) images are no longer loaded using iSCSI (tgt). By default, the ephemeral images are now obtained using HTTP requests to the rack controller.

After upgrading to MAAS 2.3, please ensure you have the latest available images. For more information please refer to the section below (New features & improvements).

Advanced network configuration for CentOS & Windows

MAAS 2.3 now supports the ability to perform network configuration for CentOS and Windows. The network configuration is performed via cloud-init. MAAS CentOS images now use the latest available version of cloud-init that includes these features.

New features & improvements

CentOS network configuration

MAAS can now perform machine network configuration for CentOS 6 and 7, providing networking feature parity with Ubuntu for those operating systems. The following can now be configured for MAAS deployed CentOS images:

  • Bonds, VLAN and bridge interfaces.
  • Static network configuration.

Our thanks to the cloud-init team for improving the network configuration support for CentOS.

Windows network configuration

MAAS can now configure NIC teaming (bonding) and VLAN interfaces for Windows deployments. This uses the native NetLBFO in Windows 2008+. Contact us for more information (https://maas.io/contact-us).

Improved Hardware Testing

MAAS 2.3 introduces a new and improved hardware testing framework that significantly improves the granularity and provision of hardware testing feedback. These improvements include:

  • An improved testing framework that allows MAAS to run each component individually. This allows MAAS to run tests against storage devices for example, and capture results individually.
  • The ability to describe custom hardware tests with a YAML definition:
    • This provides MAAS with information about the tests themselves, such as script name, description, required packages, and other metadata about what information the script will gather. All of which will be used by MAAS to render in the UI.
    • Determines whether the test supports a parameter, such as storage, allowing the test to be run against individual storage devices.
    • Provides the ability to run tests in parallel by setting this in the YAML definition.
  • Capture performance metrics for tests that can provide it.
    • CPU performance tests now offer a new ‘7z’ test, providing metrics.
    • Storage performance tests now include a new ‘fio’ test providing metrics.
    • Storage test ‘badblocks’ has been improved to provide the number of badblocks found as a metric.
  • The ability to override a machine that has been marked ‘Failed testing’. This allows administrators to acknowledge that a machine is usable despite it having failed testing.

Hardware testing improvements include the following UI changes:

  • Machine Listing page
    • Displays whether a test is pending, running or failed for the machine components (CPU, Memory or Storage.)
    • Displays whether a test not related to CPU, Memory or Storage has failed.
    • Displays a warning when the machine has been overridden and has failed tests, but is in a ‘Ready’ or ‘Deployed’ state.
  • Machine Details page
    • Summary tab – Provides hardware testing information about the different components (CPU, Memory, Storage).
    • Hardware Tests /Commission tab – Provides an improved view of the latest test run, its runtime as well as an improved view of previous results. It also adds more detailed information about specific tests, such as status, exit code, tags, runtime and logs/output (such as stdout and stderr).
    • Storage tab – Displays the status of specific disks, including whether a test is OK or failed after running hardware tests.

For more information please refer to https://docs.ubuntu.com/maas/2.3/en/nodes-hw-testing.

Network discovery & beaconing

In order to confirm network connectivity and aide with the discovery of VLANs, fabrics and subnets, MAAS 2.3 introduces network beaconing.

MAAS now sends out encrypted beacons, facilitating network discovery and monitoring. Beacons are sent using IPv4 and IPv6 multicast (and unicast) to UDP port 5240. When registering a new controller, MAAS uses the information gathered from the beaconing protocol to ensure that newly registered interfaces on each controller are associated with existing known networks in MAAS. This aids MAAS by providing better information on determining the network topology.

Using network beaconing, MAAS can better correlate which networks are connected to its controllers, even if interfaces on those controller are not configured with IP addresses. Future uses for beaconing could include validation of networks from commissioning nodes, MTU verification, and a better user experience for registering new controllers.

Upstream Proxy

MAAS 2.3 now enables an upstream HTTP proxy to be used while allowing MAAS deployed machines to continue to use the caching proxy for the repositories. Doing so provides greater flexibility for closed environments, including:

  • Enabling MAAS itself to use a corporate proxy while allowing machines to continue to use the MAAS proxy.
  • Allowing machines that don’t have access to a corporate proxy to gain network access using the MAAS proxy.

Adding upstream proxy support also includes an improved configuration on the settings page. Please refer to Settings > Proxy for more details.

Ephemeral Images over HTTP

Historically, MAAS has used ‘tgt’ to provide images over iSCSI for the ephemeral environments (e.g commissioning, deployment environment, rescue mode, etc). MAAS 2.3 changes the default behaviour by now providing images over HTTP.

These images are now downloaded directly by the initrd. The change means that the initrd loaded on PXE will contact the rack controller to download the image to load in the ephemeral environment. Support for using ‘tgt’ is being phased out in MAAS 2.3, and will no longer be supported from MAAS 2.4 onwards.

For users who would like to continue to use & load their ephemeral images via ‘tgt’, they can disable http boot with the following command.

  maas <user> maas set-config name=http_boot value=False

UI Improvements

Machines, Devices, Controllers

MAAS 2.3 introduces an improved design for the machines, devices and controllers detail pages that include the following changes.

  • “Summary” tab now only provides information about the specific node (machine, device or controller), organised across cards.
  • “Configuration” has been introduced, which includes all editable settings for the specific node (machine, device or controllers).
  • “Logs” consolidates the commissioning output and the installation log output.

Other UI improvements

Other UI improvements that have been made for MAAS 2.3 include:

  • Added DHCP status column on the ‘Subnet’s tab.
  • Added architecture filters
  • Updated VLAN and Space details page to no longer allow inline editing.
  • Updated VLAN page to include the IP ranges tables.
  • Zones page converted to AngularJS (away from YUI).
  • Added warnings when changing a Subnet’s mode (Unmanaged or Managed).
  • Renamed “Device Discovery” to “Network Discovery”.
  • Discovered devices where MAAS cannot determine the hostname now show the hostname as “unknown” and greyed out instead of using the MAC address manufacturer as the hostname.

Rack Controller Deployment

MAAS 2.3 can now automatically deploy rack controllers when deploying a machine. This is done by providing cloud-init user data, and once a machine is deployed, cloud-init will install and configure the rack controller. Upon rack controller registration, MAAS will automatically detect the machine is now a rack controller and it will be transitioned automatically. To deploy a rack controller, users can do so via the API (or CLI), e.g:

maas <user> machine deploy <system_id> install_rackd=True

Please note that this features makes use of the MAAS snap to configure the rack controller on the deployed machine. Since snap store mirrors are not yet available, this will require the machine to have access to the internet to be able to install the MAAS snap.

Controller Versions & Notifications

MAAS now surfaces the version of each running controller and notifies the users of any version mismatch between the region and rack controllers. This helps administrators identify mismatches when upgrading their MAAS on a multi-node MAAS cluster, such as within a HA setup.

Improved DNS Reloading

This new release introduces various improvements to the DNS reload mechanism. This allows MAAS to be smarter about when to reload DNS after changes have been automatically detected or made.

API Improvements

The machines API endpoint now provides more information on the configured storage and provides additional output that includes volume_groups, raids, cache_sets, and bcaches fields.

Django 1.11 support

MAAS 2.3 now supports the latest Django LTS version, Django 1.11. This allows MAAS to work with the newer Django version in Ubuntu Artful, which serves as a preparation for the next Ubuntu LTS release.

  • Users running MAAS in Ubuntu Artful will use Django 1.11.
  • Users running MAAS in Ubuntu Xenial will continue to use Django 1.9.

Read more
admin

Hello MAASters!

I’m happy to announce that MAAS 2.3.0 RC1 has now been released and it is currently available in PPA and as a snap.
PPA Availability
For those running Ubuntu Xenial and would like to use RC1, please use the following PPA:
ppa:maas/next
Snap Availability
For those running from the snap, or would like to test the snap, please use the Beta channel on the default track:
sudo snap install maas –devmode —beta
 

MAAS 2.3.0 (RC1)

Issues fixed in this release

For more information, visit: https://launchpad.net/maas/+milestone/2.3.0rc1

  • LP: #1727576    [2.3, HWTv2] When specific tests timesout there’s no log/output
  • LP: #1728300    [2.3, HWTv2] smartctl interval time checking is too short
  • LP: #1721887    [2.3, HWTv2] No way to override a machine that Failed Testing
  • LP: #1728302    [2.3, HWTv2, UI] Overall health status is redundant
  • LP: #1721827    [2.3, HWTv2] Logging when and why a machine failed testing (due to missing heartbeats/locked/hanged) not available in maas.log
  • LP: #1722665    [2.3, HWTv2] MAAS stores a limited amount of test results
  • LP: #1718779    [2.3] 00-maas-06-get-fruid-api-data fails to run on controller
  • LP: #1729857    [2.3, UI] Whitespace after checkbox on node listing page
  • LP: #1696122    [2.2] Failed to get virsh pod storage: cryptic message if no pools are defined
  • LP: #1716328    [2.2] VM creation with pod accepts the same hostname and push out the original VM
  • LP: #1718044    [2.2] Failed to process node status messages – twisted.internet.defer.QueueOverflow
  • LP: #1723944    [2.x, UI] Node auto-assigned address is not always shown while in rescue mode
  • LP: #1718776    [UI] Tooltips missing from the machines listing page
  • LP: #1724402    no output for failing test
  • LP: #1724627    00-maas-06-get-fruid-api-data fails relentlessly, causes commissioning to fail
  • LP: #1727962    Intermittent failure: TestDeviceHandler.test_list_num_queries_is_the_expected_number
  • LP: #1727360    Make partition size field optional in the API (CLI)
  • LP: #1418044    Avoid picking the wrong IP for MAAS_URL and DEFAULT_MAAS_URL
  • LP: #1729902    When commissioning don’t show message that user has overridden testing

Read more
admin

Hello MAASters!

I’m happy to announce that MAAS 2.3.0 Beta 3 has now been released and it is currently available in PPA and as a snap.
PPA Availability
For those running Ubuntu Xenial and would like to use beta 3, please use the following PPA:
ppa:maas/next
Snap Availability
For those running from the snap, or would like to test the snap, please use the Beta channel on the default track:
sudo snap install maas –devmode —beta
 

MAAS 2.3.0 (beta3)

Issues fixed in this release

For more information, visit: https://launchpad.net/maas/+milestone/2.3.0beta3
  • LP: #1727551    [2.3] Commissioning shows results from script that no longer exists

  • LP: #1696485    [2.2, HA] MAAS dhcp does not offer up multiple domains to search

  • LP: #1696661    [2.2, HA] MAAS should offer multiple DNS servers in HA case

  • LP: #1724235    [2.3, HWTv2] Aborted test should not show as failure

  • LP: #1721824    [2.3, HWTv2] Overall health status is missing

  • LP: #1727547    [2.3, HWTv2] Aborting testing goes back into the incorrect state

  • LP: #1722848    [2.3, HWTv2] Memtester test is not robust

  • LP: #1727568    [2.3, HWTv2, regression] Hardware Tests tab does not show what tests are running

  • LP: #1721268    [2.3, UI, HWTv2] Metrics table (e.g. from fio test) is not padded to MAAS’ standard

  • LP: #1721823    [2.3, UI, HWTv2] No way to surface a failed test that’s non CPU, Mem, Storage in machine listing page

  • LP: #1721886    [2.3, UI, HWTv2] Hardware Test tab doesn’t auto-update

  • LP: #1559353    [2.0a3] “Add Hardware > Chassis” cannot find off-subnet chassis BMCs

  • LP: #1705594    [2.2] rackd errors after fresh install

  • LP: #1718517    [2.3] Exceptions while processing commissioning output cause timeouts rather than being appropriately surfaced

  • LP: #1722406    [2.3] API allows “deploying” a machine that’s already deployed

  • LP: #1724677    [2.x] [critical] TFTP back-end failed right after node repeatedly requests same file via tftp

  • LP: #1726474    [2.x] psycopg2.IntegrityError: update or delete on table “maasserver_node” violates foreign key constraint

  • LP: #1727073    [2.3] rackd — 12% connected to region controllers.

  • LP: #1722671    [2.3, pod] Unable to delete a machine or a pod if the pod no longer exists

  • LP: #1680819    [2.x, UI] Tooltips go off screen

  • LP: #1725908    [2.x] deleting user with static ip mappings throws 500

  • LP: #1726865    [snap,2.3beta3] maas init uses the default gateway in the default region URL

  • LP: #1724181    maas-cli missing dependencies: netifaces, tempita

  • LP: #1724904    Changing PXE lease in DHCP snippets global sections does not work

Read more
admin

Hello MAASters!

It’s been two weeks since my last update when the MAAS 2.3 beta 2 release was announced! Since then, the MAAS team has been split, both by participating in internal events (and recovering from travel) as well as continue to focus on the stabilization of MAAS 2.3. As such, I’m happy to provide the updates of the past couple of weeks.

MAAS 2.3
In the past couple of weeks the team has been focused on stabilizing MAAS 2.3 and has fixed the following issues:

  • LP: #1705594    [2.2, HA] rackd errors after fresh install
  • LP: #1722848    [2.3, HWTv2] Memtester test is not robust
  • LP: #1724677    [2.x] [critical] TFTP back-end failed right after node repeatedly requests same file via tftp
  • LP: #1727073    [2.3, HA] rackd — 12% connected to region controllers.
  • LP: #1696485    [2.2, HA] MAAS dhcp does not offer up multiple domains to search
  • LP: #1696661    [2.2, HA] MAAS should offer multiple DNS servers in HA case
  • LP: #1721268    [2.3, UI, HWTv2] Metrics table (e.g. from fio test) is not padded to MAAS’ standard
  • LP: #1721886    [2.3, UI, HWTv2] Hardware Test tab doesn’t auto-update
  • LP: #1722671    [2.3, pod] Unable to delete a machine or a pod if the pod no longer exists
  • LP: #1724181    maas-cli missing dependencies: netifaces, tempita
  • LP: #1724235    [2.3, HWTv2] Aborted test should not show as failure
  • LP: #1726865    [snap,2.3beta3] maas init uses the default gateway in the default region URL
  • LP: #1724904    Changing PXE lease in DHCP snippets global sections does not work
  • LP: #1680819    [2.x, UI] Tooltips go off screen

 

MAAS 2.4
I’m happy to announce that the roadmap for  MAAS 2.4 has now been defined, and it is targeted for April 2018. However, I’ll create a bit of suspense as we will announce the upcoming features once MAAS 2.3 final has been released! Stay tuned!

Read more
James Henstridge

One of the options available when configuring the my ThinkPad was an Infrared camera. The main selling point being “Windows Hello” facial recognition based login. While I wasn’t planning on keeping Windows on the system, I was curious to see what I could do with it under Linux. Hopefully this is of use to anyone else trying to get it to work.

The camera is manufactured by Chicony Electronics (probably a CKFGE03 or similar), and shows up as two USB devices:

04f2:b5ce Integrated Camera
04f2:b5cf Integrated IR Camera

Both devices are bound by the uvcvideo driver, showing up as separate video4linux devices. Interestingly, the IR camera seems to be assigned /dev/video0, so generally gets picked by apps in preference to the colour camera. Unfortunately, the image it produces comes up garbled:

So it wasn’t going to be quite so easy to get things working. Looking at the advertised capture modes, the camera supports Motion-JPEG and YUYV raw mode. So I tried capturing a few JPEG frames with the following GStreamer pipeline:

gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=10 ! image/jpeg ! multifilesink location="frame-%02d.jpg"

Unlike in raw mode, the red illumination LEDs started flashing when in JPEG mode, which resulted in frames having alternating exposures. Here’s one of the better exposures:

What is interesting is that the JPEG frames have a different aspect ratio to the raw version: a more normal 640×480 rather than 400×480. So to start, I captured a few raw frames:

gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=10 ! "video/x-raw,format=(string)YUY2" ! multifilesink location="frame-%02d.raw"

The illumination LEDs stayed on constantly while recording in raw mode. The contents of the raw frames show something strange:

00000000  11 48 30 c1 04 13 44 20  81 04 13 4c 20 41 04 13  |.H0...D ...L A..|
00000010  40 10 41 04 11 40 10 81  04 11 44 00 81 04 12 40  |@.A..@....D....@|
00000020  00 c1 04 11 50 10 81 04  12 4c 10 81 03 11 44 00  |....P....L....D.|
00000030  41 04 10 48 30 01 04 11  40 10 01 04 11 40 10 81  |A..H0...@....@..|
...

The advertised YUYV format encodes two pixels in four bytes, so you would expect any repeating patterns to occur at a period of four bytes. But the data in these frames seems to repeat at a period of five bytes.

Looking closer it is actually repeating at a period of 10 bits, or four packed values for every five bytes. Furthermore, the 800 byte rows work out to 640 pixels when interpreted as packed 10 bit values (rather than the advertised 400 pixels), which matches the dimensions of the JPEG mode.

The following Python code can unpack the 10-bit pixel values:

def unpack(data):
    result = []
    for i in range(0, len(data), 5):
        block = (data[i] |
                 data[i+1] << 8 |
                 data[i+2] << 16 |
                 data[i+3] << 24 |
                 data[i+4] << 32)
        result.append((block >> 0) & 0x3ff)
        result.append((block >> 10) & 0x3ff)
        result.append((block >> 20) & 0x3ff)
        result.append((block >> 30) & 0x3ff)
    return result

After adjusting the brightness while converting to 8-bit greyscale, I get a usable image. Compare a fake YUYV frame with the decoded version:

I suppose this logic could be wrapped up in a GStreamer element to get usable infrared video capture.

I’m still not clear why the camera would lie about the pixel format it produces. My best guess is that they wanted to use the standard USB Video Class driver on Windows, and this let them get at the raw data to process in user space.

Read more
K. Tsakalozos

When was the last time you had one of those “I wish I knew about that <time period> ago” moments? One of mine was when I saw MAAS (Metal as a Service). Back at the University, as a member of the MADgIK lab, I had to administer a couple of racks. These were the days of Xen paravirtualization, Eucaliptus and the GRID. Interesting times, but I now realise that if I had MAAS and Juju back then, I would have finished with that adventure a couple of years sooner!

Was told blogs with figures are more attractive. There you go!

From a high level MAAS will render your physical cluster into an IaaS cloud that provisions physical instead of virtual machines. As you might expect a PXE server will serve the images you ask via a good looking web UI. MAAS will discover what is on your network and make it dead simple to set VLANs and network spaces. Juju can use your MaaS deployment as if it were a cloud. This enables you to deploy OpenStack, Kubernetes and BigData solutions directly onto your hardware. It is a really powerful combination that can save you time (in the range of years in my case).

Lets do it!

Do not waste any more time on this blog, go and try MAAS on QEMU instances on your own machine. This play list will get you started. At the end you will have a (virtual) cluster on QEMU. One of the machines will act as the MAAS controller, the rest will be the cluster nodes.

Your virtual cluster.
MAAS nodes

Next step is to use MAAS from Juju.

Just to show off :)

Conclusion

I am off to the shop to get more RAM. No wonder this is such a short blog.

Resources

Read more
admin

Hello MAASters!

I’m happy to announce that MAAS 2.3.0 Beta 2 has now been released and it is currently available in PPA and as a snap.
PPA Availability
For those running Ubuntu Xenial and would like to use beta 2, please use the following PPA:
ppa:maas/next
Snap Availability
For those running from the snap, or would like to test the snap, please use the Beta channel on the default track:
sudo snap install maas –devmode —beta
 

MAAS 2.3.0 (beta 2)

Issues fixed in this release

https://launchpad.net/maas/+milestone/2.3.0beta2

  • LP: #1711760    [2.3] resolv.conf is not set (during commissioning or testing)

  • LP: #1721108    [2.3, UI, HWTv2] Machine details cards – Don’t show “see results” when no tests have been run on a machine

  • LP: #1721111    [2.3, UI, HWTv2] Machine details cards – Storage card doesn’t match CPU/Memory one

  • LP: #1721548    [2.3] Failure on controller refresh seem to be causing version to not get updated

  • LP: #1710092    [2.3, HWTv2] Hardware Tests have a short timeout

  • LP: #1721113    [2.3, UI, HWTv2] Machine details cards – Storage – If multiple disks, condense the card instead of showing all disks

  • LP: #1721524    [2.3, UI, HWTv2] When upgrading from older MAAS, Storage HW tests are not mapped to the disks

  • LP: #1721587    [2.3, UI, HWTv2] Commissioning logs (and those of v2 HW Tests) are not being shown

  • LP: #1719015    $TTL in zone definition is not updated

  • LP: #1721276    [2.3, UI, HWTv2] Hardware Test tab – Table alignment for the results doesn’t align with titles

  • LP: #1721525    [2.3, UI, HWTv2] Storage card on machine details page missing red bar on top if there are failed tests

  • LP: #1722589    syslog full of “topology hint” logs

  • LP: #1719353    [2.3a3, Machine listing] Improve the information presentation of the exact tasks MAAS is running when running hardware testing

  • LP: #1719361    [2.3 alpha 3, HWTv2] On machine listing page, remove success icons for components that passed the tests

  • LP: #1721105    [2.3, UI, HWTv2] Remove green success icon from Machine listing page

  • LP: #1721273    [2.3, UI, HWTv2] Storage section on Hardware Test tab does not describe each disk to match the design

Read more
admin

MAAS 2.3.0 (beta1)

New Features & Improvements

Hardware Testing

MAAS 2.3 beta overhauls and improves the visibility of hardware test results and information. This includes various changes across MAAS:

  • Machine Listing page
    • Surface progress and failures of hardware tests, actively showing when a test is pending, running, successful or failed.
  • Machine Details page
    • Summary tab – Provide hardware testing information about the different components (CPU, Memory, Storage)
    • Hardware Tests tab – Completely re-design of the Hardware Test tab. It now shows a list of test results per component. Adds the ability to view more details about the test itself.
    • Storage tab – Ability to view tests results per storage component.

UI Improvements

Machines, Devices, Controllers

MAAS 2.3 beta 1 introduces a new design for the node summary pages:

  • “Summary tab” now only shows information of the machine, in a complete new design.
  • “Settings tab” has been introduced. It now includes the ability to edit such node.
  • “Logs tab” now consolidates the commissioning output and the installation log output.

Other UI improvements

Other UI improvements that have been made for MAAS 2.3 beta 1 includes:

  • Add DHCP status column on the ‘Subnet’s tab.
  • Add architecture filters
  • Update VLAN and Space details page to no longer allow inline editing.
  • Update VLAN page to include the IP ranges tables.
  • Convert the Zones page into AngularJS (away from YUI).
  • Add warnings when changing a Subnet’s mode (Unmanaged or Managed).

Rack Controller Deployment

MAAS beta 1 now adds the ability to deploy any machine with the rack controller, which is only available via the API.

API Improvements

MAAS 2.3 beta 1 introduces API output for volume_groups, raids, cache_sets, and bcaches field to the machines endpoint.

Known issues:

The following are a list of known UI issues affecting hardware testing:

Issues fixed in this release

https://launchpad.net/maas/+milestone/2.3.0beta1

  • #1711320    [2.3, UI] Can’t ‘Save changes’ and ‘Cancel’ on machine/device details page
  • #1696270    [2.3] Toggling Subnet from Managed to Unmanaged doesn’t warn the user that behavior changes
  • #1717287    maas-enlist doesn’t work when provided with serverurl with IPv6 address
  • #1718209    PXE configuration for dhcpv6 is wrong
  • #1718270    [2.3] MAAS improperly determines the version of some installs
  • #1718686    [2.3, master] Machine lists shows green checks on components even when no tests have been run
  • #1507712    cli: maas logout causes KeyError for other profiles
  • #1684085    [2.x, Accessibility] Inconsistent save states for fabric/subnet/vlan/space editing
  • #1718294    [packaging] dpkg-reconfigure for region controller refers to an incorrect network topology assumption

Read more
admin

Hello MAASters!

This past week, the MAAS team met face to face in NYC! The week was concentrated on finalizing the improvements that users will be able to enjoy in MAAS 2.3 and preparing the for the first beta release. While MAAS 2.3.0 beta 1 will be announced separately, we wanted to bring you an update of the work the team has been doing over the past couple weeks.

MAAS 2.3 (current development release)

  • Hardware Testing Phase 2
    • Backend work to support the new UX changes for hardware testing. This includes websockets handlers, directives and triggers.
    • UI – Add ability to upload custom hardware tests scripts.
    • UI – Update the machine listing page to show hardware status. This shows status of hardware testing while pending, running, failed, degraded, timed out, etc.
    • UI – Implement new designs for Hardware Testing:
      • Add cards (new design) on node details pages that include metrics (if tests have been run) and hardware test information.
      • Add a new Hardware Test tab that better surfaces status of hardware tests per component
      • Add a more detailed log view of hardware test results.
      • Surface hardware test results per storage device on each of the block devices (on the machines details page).
      • Add ability to view all test results performed on each of the components overtime.
  • Switch Support
    • Add actions to switch listing page (still under a feature flag)
    • Fetch Wedge 100 switch metadata using the FRUID API endpoint on the BMC.
    • UI – Add websockets and triggers to support the UI changes for switches.
    • UI – Update the UI to display the vendor and model on the switch listing page (behind feature flag)

  • UI improvements
    • Add DHCP status column on the ‘Subnet’s tab.
    • Add architecture filters
    • Implement a new design for node details page:
      • Consolidate all of machine, devices, controllers, switches Summary tab into cards.
      • Add a new Settings tab, combined with the Power tab to allow editing different components of machines, devices, controllers, etc.
      • Consolidate commissioning output and installation logs in a “Log” tab.
    • Update VLAN and Space details page to no longer allow inline editing.
    • Update VLAN page to include the IP ranges tables.
    • Convert the Zones page into AngularJS (away from YUI).
    • Add warnings when changing a Subnet’s mode (Unmanaged or Managed).

  • Rack controller deployment
    • Add ability to deploy any machine as a rack controller via the API.

  • API changes:
    • Add volume_groups, raids, cache_sets, and bcaches field to the Machine API output.

  • Issues fixed:
    • #1711320    [2.3, UI] Can’t ‘Save changes’ and ‘Cancel’ on machine/device details page
    • #1696270    [2.3] Toggling Subnet from Managed to Unmanaged doesn’t warn the user that behavior changes
    • #1717287    maas-enlist doesn’t work when provided with serverurl with IPv6 address
    • #1718209    PXE configuration for dhcpv6 is wrong
    • #1718270    [2.3] MAAS improperly determines the version of some installs
    • #1718686    [2.3, master] Machine lists shows green checks on components even when no tests have been run
    • #1507712    cli: maas logout causes KeyError for other profiles
    • #1684085    [2.x, Accessibility] Inconsistent save states for fabric/subnet/vlan/space editing
    • #1718294    [packaging] dpkg-reconfigure for region controller refers to an incorrect network topology assumption

Libmaas

We have improved the library to allow the managing of block devices and partitions.

  • Add ability to list machine’s block devices.
  • Add ability to update, create and delete block devices.
  • Add ability to list machine’s partitions.
  • Add ability to update, create and delete partitions.
  • Add ability to format/unformat partitions and block devices.
  • Add ability to mount/unmount partitions and block devices.

The release of a new version of libmaas will be announced separately.

CLI

MAAS has been working on a new CLI that’s based (and uses) MAAS’ python client library. The work that has been done includes:

  • Add ability to log in/log out via user and password.
  • Add ability to switch between profiles.
  • Add support for interactive login.
  • Add help command.
  • Ability to list nodes, machines, devices, controllers.
  • Ability to list all components in the networking model (subnets, vlans, spaces, fabrics).
  • Ability to obtain details on machines, devices and controllers.
  • Ability to obtain details on subnets, vlans, spaces, fabrics.
  • Ability to perform actions on machines (with the exception of testing and rescue mode).
  • Add ability to perform actions for multiple nodes
  • Add a ‘maas ssh’ command.
  • When listing, add support for automatic paging.
  • Add ability to view output in different formats (pretty, plain, json, yaml, csv).
  • Show progress indication on actions that are synchronous or blocking.

The release of the new CLI will be announced separately.

Read more
jdstrand

This may be totally obvious to many but I thought I would point out a problem I had with slow boot and how I solved it to highlight a neat feature of systemd to investigate the problem.

I noticed that I was having terrible boot times, so I used the `systemd-analyze critical-chain` command to see what the problem was. This showed me that `network-online.target` was taking 2 minutes. Looking at the logs (and confirming with `systemd-analyze blame`), I found it was timing out because `systemd-networkd` failed to bring interfaces either online or to fail (I was in an area where I had not connected to the existing wifi before and the wireless interface was scanning instead of failing). I looked around and found that I had no configuration for systemd-networkd (/etc/systemd/network was empty) or for netplan (/etc/netplan was empty), so I simply ran `sudo systemctl disable systemd-networkd` (leaving NetworkManager enabled) and I then had a very fast boot.

I need to file a bug on the cause of the problem, but I found the `systemd-analyze` command so helpful, I wanted to share. :)

UPDATE: this bug was reported as https://launchpad.net/bugs/1714301 and fixed in systemd 234-2ubuntu11.


Filed under: ubuntu

Read more
jdstrand

Finding your VMs and containers via DNS resolution so you can ssh into them can be tricky. I was talking with Stéphane Graber today about this and he reminded me of his excellent article: Easily ssh to your containers and VMs on Ubuntu 12.04.

These days, libvirt has the `virsh dominfo` command and LXD has a slightly different way of finding the IP address.

Here is an updated `~/.ssh/config` that I’m now using (thank you Stéphane for the update for LXD):

Host *.lxd
    #User ubuntu
    #StrictHostKeyChecking no
    #UserKnownHostsFile /dev/null
    ProxyCommand nc $(lxc list -c s4 $(echo %h | sed "s/\.lxd//g") %h | grep RUNNING | cut -d' ' -f4) %p
 
Host *.vm
    #StrictHostKeyChecking no
    #UserKnownHostsFile /dev/null
    ProxyCommand nc $(virsh domifaddr $(echo %h | sed "s/\.vm//g") | awk -F'[ /]+' '{if (NR>2 && $5) print $5}') %p

You may want to uncomment `StrictHostKeyChecking` and `UserKnownHostsFile` depending on your environment (see `man ssh_config`) for details.

With the above, I can ssh in with:

$ ssh foo.vm uptime
16:37:26 up 50 min, 0 users, load average: 0.00, 0.00, 0.00
$ ssh bar.lxd uptime
21:37:35 up 12:39, 2 users, load average: 0.55, 0.73, 0.66

Enjoy!


Filed under: canonical, ubuntu, ubuntu-server

Read more
admin

MAAS 2.3.0 (alpha3)

New Features & Improvements

Hardware Testing (backend only)

MAAS has now introduced an improved hardware testing framework. This new framework allows MAAS to test individual components of a single machine, as well as providing better feedback to the user for each of those tests. This feature has introduced:

  • Ability to define a custom testing script with a YAML definition – Each custom test can be defined with YAML that will provide information about the test. This information includes the script name, description, required packages, and other metadata about what information the script will gather. This information can then be displayed in the UI.

  • Ability to pass parameters – Adds the ability to pass specific parameters to the scripts. For example, in upcoming beta releases, users would be able to select which disks they want to test if they don’t want to test all disks.

  • Running test individually – Improves the way how hardware tests are run per component. This allows MAAS to run tests against any individual component (such a single disk).

  • Adding additional performance tests

    • Added a CPU performance test with 7z.

    • Added a storage performance test with fio.

Please note that individual results for each of the components is currently only available over the API. Upcoming beta release will include various UI improvements that will allow the user to better surface and interface with these new features.

Rack Controller Deployment in Whitebox Switches

MAAS has now the ability to install and configure a MAAS rack controller once a machine has been deployed. As of today, this feature is only available when MAAS detects the machine is a whitebox switch. As such, all MAAS certified whitebox switches will be deployed with a MAAS rack controller. Currently certified switches include the Wedge 100 and the Wedge 40.

Please note that this features makes use of the MAAS snap to configure the rack controller on the deployed machine. Since snap store mirrors are not yet available, this will require the machine to have access to the internet to be able to install the MAAS snap.

Improved DNS Reloading

This new release introduces various improvements to the DNS reload mechanism. This allows MAAS to be smarter about when to reload DNS after changes have been automatically detected or made.

UI – Controller Versions & Notifications

MAAS now surfaces the version of each running controller, and notifies the users of any version mismatch between the region and rack controllers. This helps administrators identify mismatches when upgrading their MAAS on a multi-node MAAS cluster, such as a HA setup.

Issues fixed in this release

  • #1702703    Cannot run maas-regiond without /bin/maas-rack
  • #1711414    [2.3, snap] Cannot delete a rack controller running from the snap
  • #1712450    [2.3] 500 error when uploading a new commissioning script
  • #1714273    [2.3, snap] Rack Controller from the snap fails to power manage on IPMI
  • #1715634    ‘tags machines’ takes 30+ seconds to respond with list of 9 nodes
  • #1676992    [2.2] Zesty ISO install fails on region controller due to postgresql not running
  • #1703035    MAAS should warn on version skew between controllers
  • #1708512    [2.3, UI] DNS and Description Labels misaligned on subnet details page
  • #1711700    [2.x] MAAS should avoid updating DNS if nothing changed
  • #1712422    [2.3] MAAS does not report form errors on script upload
  • #1712423    [2.3] 500 error when clicking the ‘Upload’ button with no script selected.
  • #1684094    [2.2.0rc2, UI, Subnets] Make the contextual menu language consistent across MAAS
  • #1688066    [2.2] VNC/SPICE graphical console for debugging purpose on libvirt pod created VMs
  • #1707850    [2.2] MAAS doesn’t report cloud-init failures post-deployment
  • #1711714    [2.3] cloud-init reporting not configured for deployed ubuntu core systems
  • #1681801    [2.2, UI] Device discovery – Tooltip misspelled
  • #1686246    [CLI help] set-storage-layout says Allocated when it should say Ready
  • #1621175    BMC acc setup during auto-enlistment fails on Huawei model RH1288 V3

For full details please visit:

https://launchpad.net/maas/+milestone/2.3.0alpha3

Read more
admin

Hello MAASters! This is the development summary for the past couple of weeks:

MAAS 2.3 (current development release)

  • Hardware Testing Phase 2
    • Added parameters form for script parameters validation.
    • Accept and validate results from nodes.
    • Added hardware testing 7zip CPU benchmarking builtin script.
    • WIP – ability to send parameters to test scripts and process results of individual components. (e.g. will provide the ability for users to select which disk they want to test, and capture results accordingly)
    • WIP – disk benchmark test via Fio.
  • Network beaconing & better network discovery
    • MAAS controllers now send out beacon advertisements every 30 seconds, regardless of whether or not any solicitations were received.
  • Switch Support
    • Backend changes to automatically detect switches (during commissioning) and make use of the new switch model.
    • Introduce base infrastructure for NOS drivers, similar to the power management one.
    • Install the Rack Controller when deploying a supported Switch (Wedge 40, Wedge 100)
    • UI – Add a switch listing tab behind a feature flag.
  • Minor UI improvements
    • The version of MAAS installed on each controller is now reported on the controller details page.
  • python-libmaas
    • Added ability to power on, power off, and query the power state of a machine.
    • Added PowerState enum to make it easy to check the current power state of a machine.
    • Added ability to reference the children and parent interfaces of an interface.
    • Added ability to reference the owner of node.
    • Added base level `Node` object that `Machine`, `Device`, `RackController`, and `RegionController` extend from.
    • Added `as_machine`, `as_device`, `as_rack_controller`, and `as_region_controller` to the Node object. Allowing the ability to convert a `Node` into the type you need to perform an action on.
  • Bug fixes:
    • LP: #1676992 – force Postgresql restart on maas-region-controller installation.
    • LP: #1708512 – Fix DNS & Description misalignment
    • LP: #1711714 – Add cloud-init reporting for deployed Ubuntu Core systems
    • LP: #1684094 – Make context menu language consistent for IP ranges.
    • LP: #1686246 – Fix docstring for set-storage-layout operation
    • LP: #1681801 – Device discovery – Tooltip misspelled
    • LP: #1688066 – Add Spice graphical console to pod created VM’s
    • LP: #1711700 – Improve DNS reloading so its happens only when required.
    • LP: #1712423, #1712450, #1712422 – Properly handle a ScriptForm being sent an empty file.
    • LP: #1621175 – Generate password for BMC’s with non-spec compliant password policy
    • LP: #1711414 – Fix deleting a rack when it is installed via the snap
    • LP: #1702703 – Can’t run region controller without a rack controller installed.

Read more
admin

Hello MAASters!

I’m happy to annouce that MAAS 2.3.0 Alpha 2 has now been released and it is currently available in Ubuntu Artful, PPA for Xenial and as a snap.
PPA Availability
For those running Ubuntu Xenial and would like to use Alpha 1, please use the following PPA:
Snap Availability
For those running from the snap, or would like to test the snap, please use the Beta channel on the default track:
sudo snap install maas –devmode –beta

MAAS 2.3.0 (alpha2)

Important announcements

Advanced Network for CentOS & Windows

The MAAS team is happy to announce that MAAS 2.3 now supports the ability to perform network configuration for CentOS and Windows. The network configuration is performed via cloud-init. MAAS CentOS images now use the latest available version of cloud-init that includes these features.

New Features & Improvements

CentOS Networking support

MAAS can now perform machine network configuration for CentOS, giving CentOS networking feature parity with Ubuntu. The following can now be configured for MAAS deployed CentOS images:

  • Static network configuration.

  • Bonds, VLAN and bridge interfaces.

Thanks for the cloud-init team for improving the network configuration support for CentOS.

Windows Networking support

MAAS can now configure NIC teaming (bonding) and VLAN interfaces for Windows deployments. This uses the native NetLBFO in Windows 2008+. Contact us for more information (https://maas.io/contact-us).

Network Discovery & Beaconing

MAAS now sends out encrypted beacons to facilitate network discovery and monitoring. Beacons are sent using IPv4 and IPv6 multicast (and unicast) to UDP port 5240. When registering a new controller, MAAS uses the information gathered from the beaconing protocol to ensure that newly registered interfaces on each controller are associated with existing known networks in MAAS.

UI improvements

Minor UI improvements have been made

  • Renamed “Device Discovery” to “Network Discovery”.

  • Discovered devices where MAAS cannot determine the hostname now show the hostname as “unknown” and greyed out instead of using the MAC address manufacturer as the hostname.

Issues fixed in this release

Issues fixed in this release are detailed at:

https://launchpad.net/maas/+milestone/2.3.0alpha1

Read more
Dustin Kirkland

Earlier this month, I spoke at ContainerDays, part of the excellent DevOpsDays series of conferences -- this one in lovely Portland, Oregon.

I gave a live demo of Kubernetes running directly on bare metal.  I was running it on an 11-node Ubuntu Orange Box -- but I used the exact same tools Canonical's world class consulting team uses to deploy Kubernetes onto racks of physical machines.
You see, the ability to run Kubernetes on bare metal, behind your firewall is essential to the yin-yang duality of Cloud Native computing.  Sometimes, what you need is actually a Native Cloud.
Deploying Kubernetes into virtual machines in the cloud is rather easy, straightforward, with dozens of tools now that can handle that.

But there's only one tool today, that can deploy the exact same Kubernetes to AWS, Azure, GCE, as well as VMware, OpenStack, and bare metal machines.  That tools is conjure-up, which acts as a command line front end to several essential Ubuntu tools: MAAS, LXD, and Juju.

I don't know if the presentation was recorded, but I'm happy to share with you my slides for download, and embedded here below.  There are a few screenshots within that help convey the demo.




Cheers,
Dustin

Read more
admin

Hello MAASters! This is the development summary for the past couple of weeks:

MAAS 2.3 (current development release)

The team is preparing and testing the next official release, MAAS 2.3 alpha2. It is currently undergoing a heavy round of testing and will be announced separately the beginning of the upcoming week. In the past three weeks, the team has:

  • Support for CentOS Network configuration
    We have completed the work to support CentOS Advanced Networking, which provides the ability for users to configure VLAN, bond and bridge interfaces, bringing it feature parity with Ubuntu. This will be available in MAAS 2.3 alpha 2.
  • Support for Windows Network configuration
    MAAS can now configure NIC teaming (bonding) and VLAN interfaces for Windows deployments. This uses the native NetLBFO in Windows 2008+. Contact us for more information [1].
  • Hardware Testing Phase 2

    • Testing scripts now define a type field that informs MAAS for which component will be tested and where the resulting metrics will apply. This may be node, cpu, memory, or storage, defaults to node.
    • Completed work to support the definition and parsing of a YAML based description for custom test scripts. This allows the user to defined the test’s title, description, and the metrics the test will output, which allows MAAS to parse and eventually display over the UI/API.
  • Network beaconing & better network discovery

    • Beaconing is now fully functional for controller registration and interface updates!
    • When registering or updating a new controller (either the first standalone controller, or a secondary/HA controller), new interfaces that have been determined to be on an existing VLAN will not cause a new fabric to be created in MAAS.
  • Switch modeling
    • The basic database model for the new switching model has been implemented.
    • On-going progress of presenting switches in the node listing is under way.
    • Work is in-progress to allow MAAS to deploy a rack controller which will be utilized when deploying a new switch with MAAS.
  • Minor UI improvements
    • Renamed “Device Discovery” to “Network Discovery”.
    • Discovered devices where MAAS cannot determine the hostname now just show the hostname as “unknown” and grayed out instead of using the MAC address manufacturer as the hostname.
  • Bug fixes:
    • LP: #1704444 – MAAS API returns 500 internal server error instead of raising actual error.
    • LP: #1705501 – django warning on install
    • LP: #1707971 – MAAS becomes unstable after rack controller restarts
    • LP: #1708052 – Quick erase doesn’t remove md superblock
    • LP: #1710681 – Cannot delete an Ubuntu image, “Update Selection” is disabled

MAAS 2.2.2 Released in the Ubuntu Archive!

MAAS 2.2.2 has now also been released in the Ubuntu Archive. For more details on MAAS 2.2.2, please see [2].

 

[1]: https://maas.io/contact-us

[2]: https://lists.ubuntu.com/archives/maas-devel/2017-August/002663.html

Read more
admin

Hello MAASters! The MAAS development summaries are back!

The past three weeks the team has been made good progress on three main areas, the development of 2.3, maintenance for 2.2, and out new and improved python library (libmaas).

MAAS 2.3 (current development release)

The first official MAAS 2.3 release has been prepared. It is currently undergoing a heavy round of testing and will be announced separately once completed. In the past three weeks, the team has:

  • Completed Upstream Proxy UI
    • Improve the UI to better configure the different proxy modes.
    • Added the ability to configure an upstream proxy.
  • Network beaconing & better network discovery
  • Started Hardware Testing Phase 2
      • UX team has completed the initial wireframes and gathered feedback.
      • Started changes to collect and gather better test results.
  • Started Switch modeling
      • Started changes to support switch and switch port modeling.
  • Bug fixes
    • LP: #1703403 – regiond workers can use too many postgres connections
    • LP: #1651165 – Unable to change disk name using maas gui
    • LP: #1702690 – [2.2] Commissioning a machine prefers minimum kernel over commissioning global
    • LP: #1700802 – [2.x] maas cli allocate interfaces=<label>:ip=<ADDRESS> errors with Unknown interfaces constraint Edit
    • LP: #1703713 – [2.3] Devices don’t have a link from the DNS page
    • LP: #1702976 – Cavium ThunderX lacks power settings after enlistment apparently due to missing kernel
    • LP: #1664822 – Enable IPMI over LAN if disabled
    • LP: #1703713 – Fix missing link on domain details page
    • LP: #1702669 – Add index on family(ip) for each StaticIPAddress to improve execution time of the maasserver_routable_pairs view.
    • LP: #1703845 – Set the re-check interval for rack to region RPC connections to the lowest value when a RPC connection is closed or lost.

MAAS 2.2 (current stable release)

  • Last week, MAAS 2.2 was SRU’d into the Ubuntu Archives and to our latest LTS release, Ubuntu Xenial, replacing the MAAS 2.1 series.
  • This week, a new MAAS 2.2 point release has also been prepared. It is currently undergoing heavy testing. Once testing is completed, it will be released in a separate announcement.

Libmaas

Last week, the team has worked on increasing the level

  • Added ability to create machines.
  • Added ability to commission machines.
  • Added ability to manage MAAS networking definitions. Including Subnet, Fabrics, Spaces, vlans, IP Ranges, Static Routes and DHCP.

Read more
Dustin Kirkland

Back in March, we asked the HackerNews community, “What do you want to see in Ubuntu 17.10?”: https://ubu.one/AskHN

A passionate discussion ensued, the results of which are distilled into this post: http://ubu.one/thankHN

In fact, you can check that link, http://ubu.one/thankHN and see our progress so far this cycle.  We already have a beta code in 17.10 available for your testing for several of those:

And several others have excellent work in progress, and will be complete by 17.10:

In summary -- your feedback matters!  There are hundreds of engineers and designers working for *you* to continue making Ubuntu amazing!

Along with the switch from Unity to GNOME, we’re also reviewing some of the desktop applications we package and ship in Ubuntu.  We’re looking to crowdsource input on your favorite Linux applications across a broad set of classic desktop functionality.

We invite you to contribute by listing the applications you find most useful in Linux in order of preference. To help us parse your input, please copy and paste the following bullets with your preferred apps in Linux desktop environments.  You’re welcome to suggest multiple apps, please just order them prioritized (e.g. Web Browser: Firefox, Chrome, Chromium).  If some of your functionality has moved entirely to the web, please note that too (e.g. Email Client: Gmail web, Office Suite: Office360 web).  If the software isn’t free/open source, please note that (e.g. Music Player: Spotify client non-free).  If I’ve missed a category, please add it in the same format.  If your favorites aren’t packaged for Ubuntu yet, please let us know, as we’re creating hundreds of new snap packages for Ubuntu desktop applications, and we’re keen to learn what key snaps we’re missing.

  • Web Browser: ???
  • Email Client: ???
  • Terminal: ???
  • IDE: ???
  • File manager: ???
  • Basic Text Editor: ???
  • IRC/Messaging Client: ???
  • PDF Reader: ???
  • Office Suite: ???
  • Calendar: ???
  • Video Player: ???
  • Music Player: ???
  • Photo Viewer: ???
  • Screen recording: ???

In the interest of opening this survey as widely as possible, we’ve cross-posted this thread to HackerNews, Reddit, and Slashdot.  We very much look forward to another friendly, energetic, collaborative discussion.

Or, you can fill out the survey here: https://ubu.one/apps1804

Thank you!
On behalf of @Canonical and @Ubuntu

Read more
Dustin Kirkland


I met up with the excellent hosts of the The Changelog podcast at OSCON in Austin a few weeks back, and joined them for a short segment.

That podcast recording is now live!  Enjoy!


The Changelog 256: Ubuntu Snaps and Bash on Windows Server with Dustin Kirkland
Listen on Changelog.com



Cheers,
Dustin

Read more
Leo Arias

I'm a Quality Assurance Engineer. A big part of my job is to find problems, then make sure that they are fixed and automated so they don't regress. If I do my job well, then our process will identify new and potential problems early without manual intervention from anybody in the team. It's like trying to automate myself, everyday, until I'm no longer needed and have to jump to another project.

However, as we work in the project, it's unavoidable that many small manual tasks accumulate on my hands. This happens because I set up the continuous integration infrastructure, so I'm the one who knows more about it and have easier access, or because I'm the one who requested access to the build farm so I'm the one with the password, or because I configured the staging environment and I'm the only one who knows the details. This is a great way to achieve job security, but it doesn't lead us to higher quality. It's a job half done, and it's terribly boring to be a bottleneck and a silo of information about testing and the release process. All of these tasks should be shared by the whole team, as with all the other tasks in the project.

There are two problems. First, most of these tasks involve delicate credentials that shouldn't be freely shared with everybody. Second, even if the task itself is simple and quick to execute, it's not very simple to document how to set up the environment to be able to execute them, nor how to make sure that the right task is executed in the right moment.

Chatops is how I like to solve all of this. The idea is that every task that requires manual intervention is implemented in a script that can be executed by a bot. This bot joins the communication channel where the entire team is present, and it will execute the tasks and report about their results as a response to external events that happen somewhere in the project infrastructure, or as a response to the direct request of a team member in the channel. The credentials are kept safe, they only have to be shared with the bot and the permissions can be handled with access control lists or membership to the channel. And the operative knowledge is shared with all the team, because they are all listening in the same channel with the bot. This means that anybody can execute the tasks, and the bot assists them to make it simple.

In snapcraft we started writing our bot not so long ago. It's called snappy-m-o (Microbe Obliterator), and it's written in python with errbot. We, of course, packaged it as a snap so we have automated delivery every time we change it's source code, and the bot is also autoupdated in the server, so in the chat we are always interacting with the latest and greatest.

Let me show you how we started it, in case you want to get your own. But let's call this one Baymax, and let's make a virtual environment with errbot, to experiment.

drawing of the Baymax bot

$ mkdir -p ~/workspace/baymax
$ cd ~/workspace/baymax
$ sudo apt install python3-venv
$ python3 -m venv .venv
$ source .venv/bin/activate
$ pip install errbot
$ errbot --init

The last command will initialize this bot with a super simple plugin, and will configure it to work in text mode. This means that the bot won't be listening on any channel, you can just interact with it through the command line (the ops, without the chat). Let's try it:

$ errbot
[...]
>>> !help
All commands
[...]
!tryme - Execute to check if Errbot responds to command.
[...]
>>> !tryme
It works !
>>> !shutdown --confirm

tryme is the command provided by the example plugin that errbot --init created. Take a look at the file plugins/err-example/example.py, errbot is just lovely. In order to define your own plugin you will just need a class that inherits from errbot.BotPlugin, and the commands are methods decorated with @errbot.botcmd. I won't dig into how to write plugins, because they have an amazing documentation about Plugin development. You can also read the plugins we have in our snappy-m-o, one for triggering autopkgtests on GitHub pull requests, and the other for subscribing to the results of the pull requests tests.

Let's change the config of Baymax to put it in an IRC chat:

$ pip install irc

And in the config.py file, set the following values:

BACKEND = 'IRC'
BOT_IDENTITY = {
    'nickname' : 'baymax-elopio',  # Nicknames need to be unique, so append your own.
                                   # Remember to replace 'elopio' with your nick everywhere
                                   # from now on.
    'server' : 'irc.freenode.net',
}
CHATROOM_PRESENCE = ('#snappy',)

Run it again with the errbot command, but this time join the #snappy channel in irc.freenode.net, and write in there !tryme. It works ! :)

screenshot of errbot on IRC

So, this is very simple, but let's package it now to start with the good practice of continuous delivery before it gets more complicated. As usual, it just requires a snapcraft.yaml file with all the packaging info and metadata:

name: baymax-elopio
version: '0.1-dev'
summary: A test bot with errbot.
description: Chat ops bot for my team.
grade: stable
confinement: strict

apps:
  baymax-elopio:
    command: env LC_ALL=C.UTF-8 errbot -c $SNAP/config.py
    plugs: [home, network, network-bind]

parts:
  errbot:
    plugin: python
    python-packages: [errbot, irc]
  baymax:
    source: .
    plugin: dump
    stage:
      - config.py
      - plugins
    after: [errbot]

And we need to change a few more values in config.py to make sure that the bot is relocatable, that we can run it in the isolated snap environment, and that we can add plugins after it has been installed:

import os

BOT_DATA_DIR = os.environ.get('SNAP_USER_DATA')
BOT_EXTRA_PLUGIN_DIR = os.path.join(os.environ.get('SNAP'), 'plugins')
BOT_LOG_FILE = BOT_DATA_DIR + '/err.log'

One final try, this time from the snap:

$ sudo apt install snapcraft
$ snapcraft
$ sudo snap install baymax*.snap --dangerous
$ baymax-elopio

And go back to IRC to check.

Last thing would be to push the source code we have just written to a GitHub repo, and enable the continuous delivery in build.snapcraft.io. Go to your server and install the bot with sudo snap install baymax-elopio --edge. Now everytime somebody from your team makes a change in the master repo in GitHub, the bot in your server will be automatically updated to get those changes within a few hours without any work from your side.

If you are into chatops, make sure that every time you do a manual task, you also plan for some time to turn that task into a script that can be executed by your bot. And get ready to enjoy tons and tons of free time, or just keep going through those 400 open bugs, whichever you prefer :)

Read more