A few months ago, I received two test SIM cards for Orange Poland’s new IPv6 network.
The interesting thing about this network is that it’s running IPv6 in a fairly unusual configuration and it was interesting to see how to get that work on Ubuntu.
This network uses two separate APNs, one for IPv4 (internet) and one for IPv6 (internetipv6).
Using two separate APNs is certainly easier on the carrier’s infrastructure side as they can get IPv6 online without actually changing anything on the IPv4 equipement, however that means that any client wanting to use both protocols at once needs to use multiple PDP contexts.
I’m now going to detail how to manually configure ppp to connect to such a network:
noauth connect "/usr/sbin/chat -e -f /etc/ppp/peers/orange-connect" /dev/ttyACM0 460800 +ipv6
TIMEOUT 5 ABORT BUSY ABORT 'NO CARRIER' ABORT VOICE ABORT 'NO DIALTONE' ABORT 'NO ANSWER' ABORT DELAYED ABORT ERROR '' \nAT TIMEOUT 12 OK ATH OK ATE1 OK 'AT+CGDCONT=1,"IP","internet"' OK 'AT+CGDCONT=2,"IPV6","internetipv6"' OK ATD*99# CONNECT ""
Then all that’s needed is a good old:
And a few seconds later, I’m getting the following on ppp0:
ppp0 Link encap:Point-to-Point Protocol inet addr:22.214.171.124 P-t-P:10.6.6.6 Mask:255.255.255.255 inet6 addr: 2a00:f40:2100:ac9:8c1e:da60:93e2:c234/64 Scope:Global inet6 addr: 2a00:f40:2100:ac9::1/64 Scope:Global inet6 addr: fe80::1/10 Scope:Link UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1 RX packets:13 errors:0 dropped:0 overruns:0 frame:0 TX packets:23 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:3 RX bytes:354 (354.0 B) TX bytes:767 (767.0 B
This config should work for any mobile network using a similar setup (likely to become more and more popular as the various RIRs are running out of IPv4).
Sadly ModemManager/NetworkManager don’t support mutliple PDP contexts yet, though it’s being discussed upstream, so we can hope to see something land soon.
Apparently multiple PDP contexts support is also dependant on hardware. In my case, I’ve been using an “old” Nokia E51 over USB as I didn’t have any luck getting that to work with an Android phone. My Nokia N900 also worked but required a custom kernel to be installed first to properly handle IPv6.Read more
The Internet has been alive with doom saying since the IPv4 global address pool was parcelled out. Now I do not subscribe to the view that the Internet is going to end imminently, but I do feel that if the technical people out there do not start playing with IPv6 soon then what hope is there for the masses?
In the UK getting native IPv6 is not a trivial task, only one ISP I can find seems to offer it and of course it is not the one I am with. So what options do I have? Well there are a number of different types of IPv4 tunnelling techniques such as 6to4 but these seem to require the ability to handle the transition on your NAT router, not an option here. The other is a proper 6in4 tunnel to a tunnel broker but this needs an end-point.
As I have a local server that makes a sensible anchor for such a tunnel. Talking round with those in the know I settled on getting a tunnel from Hurricane Electric (HE), a company which gives out tunnels to individuals for free and seems to have local presence for their tunnel hosts. HE even supply you with tools to cope with your endpoint having a dynamic address, handy. So with an HE tunnel configuration in hand I set about making my backup server into my IPv6 gateway.
First I had to ensure that protocol 41 (the tunnelling protocol) was being forwarded to the appropriate host. This is a little tricky as this required me to talk to the configurator for my wireless router. With that passed on to my server I was able to start configuring the tunnel.
Following the instructions on my HE tunnel broker page, a simple cut-n-paste into /etc/network/interfaces added the new tunnel network device, a quick ifup and my server started using IPv6. Interestingly my apt-cacher-ng immediately switched backhaul of its incoming IPv4 requests to IPv6 no configuration needed.
Enabling IPv6 for the rest of the network was surprisingly easy. I had to install and configure radv with my assigned prefix. It also passed out information on the HE DNS servers, prioritising IPv6 in DNS lookup results. No changes were required for any of the client systems; well other than enabling firewalls. Win.
Overall IPv6 is still not simple as it is hard to obtain native IPv6 support, but if you can get it onto your network the client side is working very well indeed.
One of my focus for this cycle is to get Ubuntu’s support for complex networking working in a predictable way. The idea was to review exactly what’s happening at boot time, get a list of possible scenario that are used on servers in corporate environment and make sure these always work.
Bonding basically means aggregating multiple physical link into one virtual link for high availability and load balancing. There are different ways of setting up such a link though the industry standard is 802.3ad (LACP – Link Aggregation Control Protocol). In that mode your server will negotiate with your switch to establish an aggregate link, then send monitoring packets to detect failure. LACP also does load balancing (MAC, IP and protocol based depending on hardware support).
One problem we had since at least Ubuntu 10.04 LTS is that Ubuntu’s boot sequence is event based, including bringing up network interfaces. The good old “ifup -a” is only done at the end of the boot sequence to try and fix anything that wasn’t brought up through events.
Unfortunately that meant that if your server takes a long time to detect the hardware, your bond would be initialised before the network cards have been detected, giving you a bond0 without a MAC address, making DHCP queries fail in pretty weird ways and making bridging or tagging fail with “Operation not permitted”.
As that all depends on hardware detection timing, it was racy, giving you random results at boot time.
Thankfully that should now be all fixed in 12.04, the new ifenslave-2.6 I uploaded a few weeks ago now initialises the bond whenever the first slave appears. If no slave appeared by the time we get to the catch-all “ifup -a”, it’ll simply wait for up to an additional minute for a slave to appear before giving up and continuing the boot sequence.
To avoid another common race condition where a bridge is brought up with a bond as one of its members before the bond is ready, ifenslave will now detect a bond is part of a bridge and add it only once ready.
Another pretty common thing on corporate networks is the use of VLANs (802.1q), letting you create up to 4096 virtual networks on one link.
In the past, Ubuntu would rely on the catch all “ifup -a” to create any required vlan interface, once again, that’s a problem when an interface that depends on that vlan interface is initialised before the vlan interface is created.
To fix that, Ubuntu 12.04′s vlan package now ships with a udev rule that triggers the creation of the vlan interface whenever its parent interface is created.
Bridging on Linux can be seen as creating a virtual switch on your system (including STP support).
Bridges have been working pretty well for a while on Ubuntu as we’ve been shipping a udev rule similar to the one for vlans for a few releases already. Members are simply added to the bridge as they appear on the system. The changes to ifenslave and the vlan package make sure that even bond interfaces with VLANs get properly added to bridges.
My current test setup for networking on Ubuntu 12.04 is actually something I’ve been using on my network for years.
As you may know, I’m also working on LXC (Linux Containers), so my servers usually run somewhere between 15 and 80 containers, each of these container has a virtual ethernet interface that’s bridged.
I have one bridge per network zone, each of these network zone being a separate VLAN. These VLANs are created on top of a two gigabit link bond.
At boot time, the following happens (roughly):
My /etc/network/interfaces can be found here:
This contains the very strict minimum needed for LACP to work. One thing worth noting is that the two physical interfaces are listed before bond0, this is to ensure that even if the events don’t work and we have to rely on the fallback “ifup -a”, the interfaces will be initialised in the right order avoiding the 60s delay.
Please note that this example will only reliably work with Ubuntu Precise (to become 12.04 LTS). It’s still a correct configuration for previous releases but race conditions may give you a random result.
I’ll be trying to push these changes to Ubuntu 11.10 as they are pretty easy to backport there, however it’d be much harder and very likely dangerous to backport these to even older releases.
For these, the only recommendation I can give is to add some “pre-up sleep 5″ or similar to your bridges and vlan interfaces to make sure whatever interface they depend on exists and is ready by the time the “ifup -a” call is reached.
Another interesting topic for 12.04 is IPv6, as a release that’ll be supported for 5 years on both servers and desktops, it’s important to get IPv6 right.
Ubuntu 12.04 LTS will be the first Ubuntu release shipping with IPv6 private extensions turned on by default. Ubuntu 11.10 already brought most of what’s needed for IPv6 support on the desktop and in the installer, supporting SLAAC (stateless autoconfiguration), stateless DHCPv6 and stateful DHCPv6.
Once we get a new ifupdown in Ubuntu Precise, we’ll have full support for IPv6 also for people that aren’t using Network Manager (mostly servers) which should at this point give us support for any IPv6 setup you may find.
The userspace has been working pretty well with IPv6 for years. I recently made my whole network dual-stack and now have all my servers and services defaulting to IPv6 for a total of 40% of my network traffic (roughly 1.5TB a month of IPv6 traffic). The only user space related problem I noticed is the lack of IPv6 support in Nagios’ nrpe plugin, meaning I can’t start converting servers to single stack IPv6 as I’d loose monitoring …
I also wrote a python script using pypcap to give me the percentage of ipv6 and ipv4 traffic going through a given interface, the script can be found here: http://www.stgraber.org/download/v6stats.py (start with: python v6stats.py eth0)
At this point, I think Ubuntu Precise is pretty much ready as far as networking is concerned. The only remaining change is the new ifupdown and the small installer change that comes with it for DHCPv6 support.
If you have a spare machine or VM, now is the time to grab a recent build of Ubuntu Precise and make sure whatever network configuration you use in production works reliably for you and that any hack you needed in the past can now be dropped.
If your configuration doesn’t work at all or fails randomly, please file a bug so we can have a look at your configuration and fix any bug (or documentation).
One of my focus for the Oneiric development cycle is to make sure we get proper support of IPv6 both at install time and during regular use of the system.
To achieve this, I started working on the list of all possible scenarios I could think of with all possible combinations of IPv4 and IPv6. Then checked how well these were supported on Ubuntu.
Since Ubuntu 11.04, we now have a DHCPv6 aware DHCP client but that’s not working as well as it should because Network Manager didn’t do IPv6 by default back then and because the DHCP client configuration for IPv6 wasn’t too clear (dhclient wasn’t requesting any attribute).
Most of these issues are now fixed in Oneiric with Mathieu Trudel-Lapierre‘s great work on updating Network Manager in Oneiric to have IPv6 on by default and make sure people don’t have to wait for IPv6 to timeout to get their IPv4 connectivity.
The result is something like you can see below, on a network that has both DHCPv4 and stateless DHCPv6:
The use cases that are currently tested are:
For these interested, you can look at the following files to get some example DHCPv4, DHCPv6 and RADVD configuration:
It’s worth noting that you have to start a separate dhcpd server for IPv6 (with the -6 flag) as dhcpd can’t answer both dhcpv4 and dhcpv6 at the same time. You need two separate daemons with two separate configuration files.
As you can see from the files above, I have a pretty complete IPv6 test setup, running on libvirt. I’m now working on automating all of this so we can get some easy regression testing of IPv6 support on Ubuntu.
During our sprint last month in Dublin, Colin Watson also got netcfg to support IPv6 thereby making debian-installer working with IPv6. The missing piece now is ifupdown support of DHCPv6 (so you can configure DHCPv6 in /etc/network/interfaces) and we should then have Ubuntu install on IPv6 from the alternate/server disks.
IPv6 support is starting to look really good for Oneiric and should be awesome for the next LTS.
If you’re already running Oneiric on an IPv6 capable network, please test the new Network Manager and if you encounter any problem, please file bugs or poke me so I can add some more tests to my list!
© 2010 Canonical Ltd. Ubuntu and Canonical are registered trademarks of Canonical Ltd.