Servers in Concert!
I've previously given an architectural introduction
to the design of Orchestra
. Now, let's take a practical look at it in this how-to guide.
To follow this particular guide, you'll need at least two physical systems and administrative access rights on your local DHCP server (perhaps on your network's router). With a little ingenuity, you can probably use two virtual machines and work around the router configuration. I'll follow this guide with another one using entirely virtual machines.
To build this demonstration, I'm using two older ASUS (P1AH2
) desktop systems. They're both dual-core 2.4GHz AMD processors and 2GB of RAM each. I'm also using a Linksys WRT310n router flashed with DD-WRT
. Most importantly, at least one of the systems must be able to boot over the network using PXE
After this system reboots, update and upgrade all packages on the system, and then install the ubuntu-orchestra-server package
sudo apt-get update
sudo apt-get dist-upgrade -y
sudo apt-get install -y ubuntu-orchestra-server
You'll be prompted to enter a couple of configuration parameters, such as setting the cobbler user's password. It's important to read and understand each question. The default values are probably acceptable, except for one, which you'll want to be very careful about...the one that asks about DHCP/DNS management.
In this post, I selected "No", as I want my DD-WRT
router to continue handling DHCP/DNS
. However, in a production environment (and if you want to use Orchestra
), you might need to select "Yes" here.
And a about five minutes later, you should have an Ubuntu Orchestra Server up and running!
Target System Setup
Once your Orchestra
Server is installed, you're ready to prepare your target system for installation. You will need to enter your target system's BIOS
settings, and ensure that the system is set to first boot from PXE
(netboot), and then to local disk (hdd). Orchestra
(a project maintained by our friends at Fedora
) to prepare the network installation using PXE
, and thus your machine needs to boot from the network. While you're in your BIOS
configuration, you might also ensure that Wake on LAN
) is also enabled.
Next, you'll need to obtain the MAC address
of the network card in your target system. One of many ways to obtain this is by booting that Ubuntu ISO, pressing ctrl-alt-F2
, and running ip addr show.
Now, you should add the system to Cobbler
. Ubuntu 11.10 ships a feature called cobbler-enlist
that automates this, however, for this guide, we'll use the Cobbler
web interface. Give the system a hostname (e.g., asus1
), select its profile (e.g., oneiric-x86_64
), IP address (e.g. 192.168.1.70
), and MAC address (e.g., 00:1a:92:88:b7:d9
). In the case of this system, I needed to tweak the Kernel Options
, since this machine has more than one attached hard drive, and I want to ensure that Ubuntu installs onto /dev/sdc
, so I set the Kernel Options
. You might have other tweaks on a system-by-system basis that you need or want to adjust here (like IPMI
Finally, I adjusted my DD-WRT
router to add a static lease for my target system, and point dnsmasq
boot against the Orchestra
Server. You'll need to do something similar-but-different here, depending on how your network handles DHCP
NOTE: As of October 27, 2011, Bug #882726 must be manually worked around, though this should be fixed in oneiric-updates any day now. To work around this bug, login to the Orchestra Server and run:
for r in $RELEASES; do
for a in $ARCHES; do
sudo cobbler profile edit --name="$r-$a" \
All set! Now, let's trigger the installation. In the web interface, enable the machine for netbooting.
If you have WoL working for this system, you can even use the web interface to power the system on. If not, you'll need to press the power button yourself.
Now, we can watch the installation remotely, from an SSH
session into our Orchestra
Server! For extra bling, install these two packages:
sudo apt-get install -y tmux ccze
Now launch byobu-tmux
(which handles splits much better than byobu-screen
). In the current window, run:
tail -f /var/log/syslog | ccze
Now, split the screen vertically with ctrl-F2. In the new split, run:
sudo tail -f /var/log/squid/access.log | ccze
Move back and forth between splits with shift-F3
. The ccze
command colorizes log files.
progress of your installation scrolling by. In the right split, you'll see your squid
logs, as your Orchestra
server caches the binary deb
files it downloads. On your first installation, you'll see a lot of TCP_MISS
messages. But if you try this installation a second time, subsequent installs will roll along much
faster and you should see lots of TCP_HIT
It takes me about 5 minutes to install these machines with a warm squid cache (and maybe 10 mintues to do that first installation downloading all of those debs over the Internet). More importantly, I have installed as many as 30 machines simultaneously in a little over 5 minutes with a warm cache! I'd love to try more, but that's as much hardware as I've had concurrent access to, at this point.
Most of what you've seen above is the provisioning aspect of Orchestra -- how to get the Ubuntu Server installed to bare metal, over the network, and at scale. Cobbler does much of the hard work there, but remarkably, that's only the first pillar of Orchestra.
What you can do after
the system is installed is even more exciting! Each system installed by Orchestra
automatically uses rsyslog
to push logs back to the Orchestra
server. To keep the logs of multiple clients in sync, NTP
is installed and running on every Orchestra
managed system. The Orchestra
Server also includes the Nagios
web front end, and each installed client runs a Nagios
client. We're working on improving the out-of-the-box Nagios
experience for 12.04, but the fundamentals are already there. Orchestra
clients are running PowerNap
in power-save mode, by default, so that Orchestra
installed servers operate as energy efficiently as possible.
Perhaps most importantly, Orchestra
can actually serve as a machine provider to Juju
, which can then offer complete Service Orchestration
to your physical servers. I'll explain in another post soon how to point Juju
to your Orchestra
infrastructure, and deploy services directly to your bare metal servers.
I won't be able to offer support in the comments below, but if you have questions or comments, drop by the friendly #ubuntu-server
IRC channel on irc.freenode.net, where we have at least a dozen Ubuntu Server developers with Orchestra
expertise, hanging around and happy to help!