Canonical Voices

Posts tagged with 'powernap'

Dustin Kirkland


Servers in Concert!

Ubuntu Orchestra is one of the most exciting features of the Ubuntu 11.10 Server release, and we're already improving upon it for the big 12.04 LTS!

I've previously given an architectural introduction to the design of Orchestra.  Now, let's take a practical look at it in this how-to guide.

Prerequisites

To follow this particular guide, you'll need at least two physical systems and administrative access rights on your local DHCP server (perhaps on your network's router).  With a little ingenuity, you can probably use two virtual machines and work around the router configuration.  I'll follow this guide with another one using entirely virtual machines.

To build this demonstration, I'm using two older ASUS (P1AH2) desktop systems.  They're both dual-core 2.4GHz AMD processors and 2GB of RAM each.  I'm also using a Linksys WRT310n router flashed with DD-WRT.  Most importantly, at least one of the systems must be able to boot over the network using PXE.

Orchestra Installation

You will need to manually install Ubuntu 11.10 Server on one of the systems, using an ISO or a USB flash disk.  I used the 64-bit Ubuntu 11.10 Server ISO, and my no-questions-asked uquick installation method.  This took me a little less than 10 minutes.

After this system reboots, update and upgrade all packages on the system, and then install the ubuntu-orchestra-server package.

sudo apt-get update
sudo apt-get dist-upgrade -y
sudo apt-get install -y ubuntu-orchestra-server

You'll be prompted to enter a couple of configuration parameters, such as setting the cobbler user's password.  It's important to read and understand each question.  The default values are probably acceptable, except for one, which you'll want to be very careful about...the one that asks about DHCP/DNS management.

In this post, I selected "No", as I want my DD-WRT router to continue handling DHCP/DNS.  However, in a production environment (and if you want to use Orchestra with Juju), you might need to select "Yes" here.


And a about five minutes later, you should have an Ubuntu Orchestra Server up and running!

Target System Setup

Once your Orchestra Server is installed, you're ready to prepare your target system for installation.  You will need to enter your target system's BIOS settings, and ensure that the system is set to first boot from PXE (netboot), and then to local disk (hdd).  Orchestra uses Cobbler (a project maintained by our friends at Fedora) to prepare the network installation using PXE and TFTP, and thus your machine needs to boot from the network.  While you're in your BIOS configuration, you might also ensure that Wake on LAN (WoL) is also enabled.

Next, you'll need to obtain the MAC address of the network card in your target system.  One of many ways to obtain this is by booting that Ubuntu ISO, pressing ctrl-alt-F2, and running ip addr show.

Now, you should add the system to Cobbler.  Ubuntu 11.10 ships a feature called cobbler-enlist that automates this, however, for this guide, we'll use the Cobbler web interface.  Give the system a hostname (e.g., asus1), select its profile (e.g., oneiric-x86_64), IP address (e.g. 192.168.1.70), and MAC address (e.g., 00:1a:92:88:b7:d9).  In the case of this system, I needed to tweak the Kernel Options, since this machine has more than one attached hard drive, and I want to ensure that Ubuntu installs onto /dev/sdc, so I set the Kernel Options to partman-auto/disk=/dev/sdc.  You might have other tweaks on a system-by-system basis that you need or want to adjust here (like IPMI configuration).


Finally, I adjusted my DD-WRT router to add a static lease for my target system, and point dnsmasq to PXE boot against the Orchestra Server.  You'll need to do something similar-but-different here, depending on how your network handles DHCP.


NOTE: As of October 27, 2011, Bug #882726 must be manually worked around, though this should be fixed in oneiric-updates any day now.  To work around this bug, login to the Orchestra Server and run:

RELEASES=$(distro-info --supported)
ARCHES="x86_64 i386"
KSDIR="/var/lib/orchestra/kickstarts"
for r in $RELEASES; do
for a in $ARCHES; do
sudo cobbler profile edit --name="$r-$a" \
--kickstart="$KSDIR/orchestra.preseed"
done
done

Target Installation

All set!  Now, let's trigger the installation.  In the web interface, enable the machine for netbooting.


If you have WoL working for this system, you can even use the web interface to power the system on.  If not, you'll need to press the power button yourself.

Now, we can watch the installation remotely, from an SSH session into our Orchestra Server!  For extra bling, install these two packages:

sudo apt-get install -y tmux ccze

Now launch byobu-tmux (which handles splits much better than byobu-screen).  In the current window, run:

tail -f /var/log/syslog | ccze

Now, split the screen vertically with ctrl-F2.  In the new split, run:

sudo tail -f /var/log/squid/access.log | ccze

Move back and forth between splits with shift-F3 and shift-F4.  The ccze command colorizes log files.

syslog progress of your installation scrolling by.  In the right split, you'll see your squid logs, as your Orchestra server caches the binary deb files it downloads.  On your first installation, you'll see a lot of TCP_MISS messages.  But if you try this installation a second time, subsequent installs will roll along much faster and you should see lots of TCP_HIT messages.


It takes me about 5 minutes to install these machines with a warm squid cache (and maybe 10 mintues to do that first installation downloading all of those debs over the Internet).  More importantly, I have installed as many as 30 machines simultaneously in a little over 5 minutes with a warm cache!  I'd love to try more, but that's as much hardware as I've had concurrent access to, at this point.

Post Installation

Most of what you've seen above is the provisioning aspect of Orchestra -- how to get the Ubuntu Server installed to bare metal, over the network, and at scale.  Cobbler does much of the hard work there,  but remarkably, that's only the first pillar of Orchestra.

What you can do after the system is installed is even more exciting!  Each system installed by Orchestra automatically uses rsyslog to push logs back to the Orchestra server.  To keep the logs of multiple clients in sync, NTP is installed and running on every Orchestra managed system.  The Orchestra Server also includes the Nagios web front end, and each installed client runs a Nagios client.  We're working on improving the out-of-the-box Nagios experience for 12.04, but the fundamentals are already there.  Orchestra clients are running PowerNap in power-save mode, by default, so that Orchestra installed servers operate as energy efficiently as possible.

Perhaps most importantly, Orchestra can actually serve as a machine provider to Juju, which can then offer complete Service Orchestration to your physical servers.  I'll explain in another post soon how to point Juju to your Orchestra infrastructure, and deploy services directly to your bare metal servers.

Questions?  Comments?

I won't be able to offer support in the comments below, but if you have questions or comments, drop by the friendly #ubuntu-server IRC channel on irc.freenode.net, where we have at least a dozen Ubuntu Server developers with Orchestra expertise, hanging around and happy to help!

Cheers,
:-Dustin

Read more
Dustin Kirkland


I was honored to speak at LinuxCon North America in beautiful Vancouver yesterday, about one of my favorite topics -- energy efficiency opportunities using Ubuntu Servers in the data center (something I've blogged before).

I'm pleased to share those slides with you today!  The talk is entitled PowerNap Your Data Center, and focused on Ubuntu's innovative PowerNap suite, from the system administrator's or data center manager's perspective.

We discussed the original, Cloud motivations for PowerNap, its evolution from the basic process monitoring and suspend/hibernate methods of PowerNap1, to our complete rewrite of PowerNap2 (thanks, Andres!) which added nearly a dozen monitors and the ubiquitously useful PowerSave mode.  PowerNap is now more useful and configurable than ever!

Flip through the presentation below, or download the PDF here.



Get Adobe Flash player


Stay tuned for another PowerNap presentation I'm giving at Linux Plumbers next month in California.  That one should be a bit deeper dive into the technical implementation, and hopefully generate some plumbing layer discussion and improvement suggestions.

:-Dustin

Read more
roaksoax

After the success of the improvements of PowerNap in Ubuntu Natty 11.04, we will be having another session st UDS-O Thursday the 12th at 15:00. In this session we will discuss the following:

  • Second Stage action when running in PowerSave mode.
  • Support for port-ranges in Network Monitors
  • Changing the polling monitoring system to an event based system.
  • Client/Server approach to monitor/manage PowerNap “client machines” over the network for data center wide deployments
  • Server ARP network Monitoring for Automatic Wake-up of Clients.
  • API like approach for Integration with other projects.

Everyone who’s interested are more than welcome to join! For more information, the blueprint can be found HERE.

Read more
roaksoax

For all of those who don’t know, “PowerNap is a screen saver for servers except it doesn’t save your screen, it saves the environment and lowers your energy bill.” Dustin Kirkland :) . PowerNap was originally created by Dustin to be integrated with (UEC), but it has been extended for Home use. Originally, it put to sleep machines (suspend, hibernate, poweroff) when a list of Processes were not found in the process table for  a determined period of time. However, during the Natty cycle improvements were made. So, PowerNap now puts to sleep (suspend, poweroff, powersave) machines that are tagged as underutilized  by a set if Monitors.

Improvements Overview

  • PowerNap, has a set of Monitors to be able to detect activity within the server and determine if it is idled or not. If it is, PowerNap will execute an ACTION. Administrators can chose what monitors to enable/disable. These are:
    • ProcessMonitor: Looks for a process in the process table.
    • IOMonitor: Monitors IO activity by process name.
    • InputMonitor: Monitors Mouse/Keyboard input activity connected to USB.
    • LoadMonitor: Monitors a server load threshold.
    • TCPMonitor: Monitors active TCP connections (i.e. SSH).
    • UDPMonitor: Monitors activity received in any user defined UDP port.
    • WoLMonitor: Monitors WoL packets on ports 7 and/or 9.
    • ConsoleMonitor: Monitors console activity.
  • The process starts when PowerNap begins monitoring for an ABSENT_PERIOD (i.e. 300secs). If within that period no activity has been detected, then PowerNap executes an ACTION.
    • Before the ACTION is taken, PowerNap enters to the GRACE_PERIOD (I.e 30 seconds), notifying the user that the ACTION will be taken in GRACE_PERIOD amount of seconds. (i.e. On second 270 PowerNap will notifies its users and the period between 270 and 300 seconds is known as GRACE_PERIOD).
  • The possible ACTIONS are:
    • Best-effort – Automatically decide between a user defined action or any of the other methods listed below (these methods rely on pm-utils)
    • Suspend (Command: pm-suspend)
    • Hibernate (Command: pm-hibernate)
    • Poweroff (Command: poweroff)
    • Powersave – Newly added method that reduces the Power Consumption (Command: pm-powersave)
  • The PowerSave method executes a set of scripts both provided by pm-utils and PowerNap. These scripts have the objective to reduce the power consumption of the machine by turning off hardware capabilities or tuning the OS. It is possible to provide any custom script as well as chose which to enable or disable. Examples of these scripts are:
    • Turn off all the CPU cores except of one.
    • Reduce the cores frequency to the lowest possible.
    • Disable WoL from Network Cards.
    • Change the NIC speed from 1Gbps to 100Mbps.
    • Turn off USB ports.
    • Disable HAL polling.
  • Now, when the PowerSave ACTION is taken, the machine keeps running in a lower power state. PowerNap keeps Monitoring until activity is detected. Once any of the Monitors detects activity, the PowerSave action is reverted.

 

PowerWake

  • PowerWake is simply a tool that sends WoL packets to an specified IP/Broadcast address to be able to wakeup a server.

 

Additional Tools

  • powernap-now: Sends a signal to the PowerNap daemon to execute the ACTION regardless of the state of the monitors.
  • powerwake-now: Sends a signal to the PowerNap daemon to wakeup during the PowerSave mode.
  • Note that these commands have to be executed in the machine running PowerNap. If this needs to be done through the network, then the command will have to be sent remotely to be executed in the machine.


The Future

  • Second Stage Action: Second Stage Action when entered into PowerSave mode. (i.e. Suspend after 2 hours after running in PowerSave mode).
  • Client/Server Model: The main Idea is to create a powerwaked Server that tracks all the machines using PowerNap in the network and is able to schedule wakeups, upstates, etc, etc.

Read more