Bird / Quagga with MD5 Support for IPv4/6 on FreeBSD & Linux

Over in INEX we run a route server cluster which alleviates the burden of setting up bilateral peering sessions for the more than 80% of the members that use them. The current hardware is now about six years old and we have a forklift upgrade in the works.

BGP allows for MD5 authentication between clients (using the TCP MD5 signature option, see RFC 2385) and – while recently obsoleted in RFC 5925 – it is still widely used in shared LAN mediums such as IXPs; primarily to prevent packet spoofing and session hijacking via recycled IP addresses.

Our current route server implementation runs on FreeBSD which does not support TCP MD5 in its stock kernel (you are required to compile a custom kernel – see below for details). Additionally, specifying the session MD5 is not done in the BGP daemon configuration but separately in the IPsec configuration. Lastly, our current FreeBSD version has no support for TCP MD5  over IPv6. These have all led to unnecessarily complex configurations and a degree of confusion.

Because of this, we decided to test up to date Linux and FreeBSD versions for native IPv4 and IPv6 TCP MD5 support with Bird and Quagga (our route server daemons of choice).

In each case, BGP sessions were tested for:

  • no MD5 on each end (expected to work);
  • same MD5 on each end (expected to work);
  • different MD5 on each end (expected not to work); and
  • MD5 on one end with no MD5 on the other end (expected not to work).

For Linux, the platform chosen was Ubuntu 12.04 LTS with the stock 3.2.0-40-generic kernel.

  • Sessions were tested for Quagga to Quagga and Quagga to Bird;
  • Sessions were tested over both IPv4 and IPv6;
  • The presence of valid MD5 signatures were confirmed using tcpdump -M xxx;
  • Stock Quagga and Bird from the 12.04 apt repositories were used.

The results - everything worked and worked as expected:

  • BGP sessions only established when expected (no MD5 configured, same MD5 configured);
  • This held for both IPv4 and IPv6.

Summary: Linux will support TCP MD5 nativily for IPv4 and IPv6 when using Quagga or Bird.

For FreeBSD, we used the latest production release of 9.1. TCP MD5 support is not compiled in by default so a custom kernel must be built with the additional options of:

options   TCP_SIGNATURE
options   IPSEC
device    crypto
device    cryptodev

In addition to this, the MD5 shared secrets need to be added to the IPsec SA/SD database via the setkey utility or, preferably, via the /etc/ipsec.conf file which, for example, would contain entries for IPv4 and IPv6 addresses such as:

add 192.0.2.1 192.0.2.2 tcp 0x1000 -A tcp-md5 "supersecret1";
add 2001:db8::1 2001:db8::2 tcp 0x1000 -A tcp-md5 "supersecret2";

where the addresses ending in .1/:1 are local and .2/:2 are the BGP neighbor addresses. This file can be processed by setting ipsec_enable="YES" in /etc/rc.conf and executing /etc/rc.d/ipsec reload.

  • Sessions were tested for Quagga/Linux to Quagga/FreeBSD and  from Quagga/Linux to Bird/FreeBSD;
  • Sessions were tested over both IPv4 and IPv6;
  • The presence of valid MD5 signatures were confirmed using tcpdump -M xxx;
  • Stock Quagga from the 12.04 apt repositories and stock Quagga and Bird from FreeBSD ports were used.

The results – almost everything worked and worked as expected:

  • BGP sessions only established when expected (no MD5 configured, same MD5 configured);
  • This held for both IPv4 and IPv6;
  • one odd but expected behavior – you only need to set the MD5 via setkey / ipsec.conf – setting it (or not) in the Quagga and Bird config has no effect so long as it is set via setkey (but is useful for documentation purposes). However, trying to set it in Quagga without having rebuilt the kernel will result in an error.

Summary: FreeBSD will support TCP MD5 via a custom kernel and setkey / ipsec.conf for IPv4 and IPv6. Note that there is an additional complexity when changing or removing MD5 passwords as these need to be amended / deleted via setkey which can put an extra burden on automatic route server configuration generators.

MySQL 5.6 – Memcached / NoSQL Support and More

MySQL 5.6 has been released with some interesting new features and performance increases:

  • What’s New in MySQL 5.6
  • DBA and Developer Guide to MySQL 5.6
  • InnoDB Integration with memcached:MySQL 5.6 includes a NoSQL interface, using an integrated memcached daemon that can automatically store data and retrieve it from InnoDB tables, turning the MySQL server into a fast “key-value store” for single-row insert, update, or delete operations. You can still also access the same tables through SQL for convenience, complex queries, bulk operations, application compatibility, and other strengths of traditional database software.

    With this NoSQL interface, you use the familiar memcached API to speed up database operations, letting InnoDB handle memory caching using its buffer pool mechanism. Data modified through memcached operations such as ADD, SET, INCR are stored to disk, using the familiar InnoDB mechanisms such as change buffering, the doublewrite buffer, and crash recovery. The combination of memcached simplicity and InnoDB durability provides users with the best of both worlds.

  • Multi-threaded Slaves
  • Improved IPv6 Support – both in the bind to address option and the INET_ATON() function.
  • Replication improvements.

All in all, some nice new features. Especially the memcached integration.

That said, MariaDB seems to be making inroads on MySQL with some distributions considering a switch. Some interesting reading from that project includes:

So it’s finally happened…

RIPE put out a press release today:

RIPE NCC Begins to Allocate IPv4 Address Space From the Last /8

14 Sep 2012

On Friday 14 September, 2012, the RIPE NCC, the Regional Internet Registry (RIR) for Europe, the Middle East and parts of Central Asia, distributed the last blocks of IPv4 address space from the available pool.

This means that we are now distributing IPv4 address space to Local Internet Registries (LIRs) from the last /8 according tosection 5.6 of “IPv4 Address Allocation and Assignment Policies for the RIPE NCC Service Region“.

This section states that an LIR may receive one /22 allocation (1,024 IPv4 addresses), even if they can justify a larger allocation. This /22 allocation will only be made to LIRs if they have already received an IPv6 allocation from an upstream LIR or the RIPE NCC. No new IPv4 Provider Independent (PI) space will be assigned.

It is now imperative that all stakeholders deploy IPv6 on their networks to ensure the continuity of their online operations and the future growth of the Internet.

In other words, for all intents and purposes, Europe (and Central Asia and the Middle East) is out of IPv4 addresses. Funnily enough, I’m actually happy that this long predicted day has arrived and we can start the next phase of IPv6 deployment.

Follow Up – IPv6 Statistics at INEX

A couple of days ago, I was talking about World IPv6 day with some notes on the Irish context.

INEX is a neutral, industry-owned association, founded in 1996, that provides IP peering facilities for its members. INEX membership is open to all organisations that can benefit from peering their IP traffic and there are currently 57 members.

INEX can also be considered Ireland’s IP Peering Hub. INEX membership provides high-speed, reliable and resilient IP traffic exchange facilities for both Irish and International organisations, allowing them to route IP traffic efficiently thereby providing faster, more reliable and lower-latency internet access for their customers.

As a follow up to the previous post, here’s a like for like comparison of IPv4 and IPv6 traffic over peering LAN 1 of the exchange:

Notes:

  • As a layer 2 exchange, traffic over INEX is symmetrical – traffic originating from one member is destined for another.
  • INEX runs two peering LANs for resiliency. The IPv6 traffic on LAN 2 was negligible over the same period. See the public statistics and the weathermaps of each LAN showing the network topology.

 

World IPv6 Day with Irish Statistics

In case it passed you by, today was World IPv6 Day. In a nutshell: “Major Internet service providers (ISPs), home networking equipment manufacturers, and web companies around the world are coming together to permanently enable IPv6 for their products and services by 6 June 2012.” This includes top content providers such as Facebook (see under their hood), Google (read what they had to say), Yahoo! and Microsoft. In fact, you may not even have noticed but Google were advertising it front and centre on their search page:

Google Announcing World IPv6 Day on Their Search Page

Over at INEX, we were unable to pull out IPv6 traffic statistics on the exchange until recently and my colleague just got the first pass of that project complete this week in time for World IPv6 Day. Here’s how it looked over the hours leading up to and into World IPv6 Day:

Now, the peek of almost 40Mbps is, most assuredly, small compared to the overall peek of 24Gbps, but there is a very pronounced jump in IPv6 traffic which is certainly a good sign and a move in the right direction. The overall peering statistics at INEX are public and we’ll be breaking out IPv4 and IPv6 into separate graphs shortly also.

Why does IPv6 amount to < 0.2% of the traffic at the exchange? Well there are two main factors:

  • Until today, there has been very little mass or popular content available over IPv6. So, even if you were IPv6 enabled, there was very little for you access.
  • None of the large ISPs in Ireland are providing IPv6 connectivity to end users outside of certain closed test programs.

This is the classic chicken and egg problem: with no content available the ISPs were not motivated to provide IPv6 connectivity; and, conversely, with no IPv6 enabled eyeballs the content providers were not motivated to make their services available over IPv6.

While today was not necessarily a content provider only day, I’m unaware of any Irish ISPs that got involved. But, now that we have significant content available over IPv6, hopefully the ISPs will begin to ramp up their own programs. And – to be fair – it’s not all bad news with the ISPs in Ireland. Most have their core and edge networks IPv6 enabled, it’s the access layer that’s the issue (and it’s a really really big issue and a very difficult issue).

AMS-IX (the Amsterdam Internet Exchange) is in the top three IXPs in the world by traffic volume and they also make their IPv6 statistics public. As a second demonstration of traffic levels on World IPv6 Day, here is the week to date showing a huge differential for today:

If you’re not sure what all this is about, well then here are a few words from the creator of the Internet himself:

And if you’re keen to start experimenting with IPv6, first email and ask your ISP. They’ll say no, but do it anyway! Then head over to SixXS (and be sure to choose either HEAnet or Digiweb as your PoP as both are INEX members and as such you’ll have the lowest possible latency).

Benchmarking the Mikrotik Routerboards RB750 and RB750G

Continuing on from today’s earlier post, Benchmarking the Mikrotik Routerboard RB1100, I now present some results for the RB750 and RB750G using the same methodology and platform.

The RB750 and the RB750G are two identical looking routers intended for the SOHO environment:

The specifications for the RB750 (with differences for the RB750G in italics and parenthesis) are:

  • five FastEthernet 100Mbps (Gigabit 1Gbps) ports;
  • 32MB DDR SDRAM ;
  • 64MB on board NAND storage;
  • Atheros AR7240 400MHz (AR7161 680MHz) CPU;
  • powered by PoE or power jack;
  • up to 3W (6W) power consumption;
  • ports 2-5 share dedicated switch chip allowing full 100Mbps (1Gbps) throughput;
  • all ports can be individually configured.
  • €31.73 (€54.61) from Wireless Connect.

Both routers come with an L4 license of Mikritik’s RouterOS which is built on the Linux kernel so anyone familiar with Linux networking will get up to speed on these boxes in no time.

As a disclaimer in case it is not clear, all routing tests are done using just two ports – one for the traffic generator and one for the receiver – with the device under testing routing the packets between two networks. As such, on the RB750, the maximum throughput we could achieve would be 100Mbps.

I ran tests for plain routing and also, in evaluating it for certain uses, over a VPN tunnel.

All results are presented below. Given the wealth of features, I think these are super boxes at a super price. So far I’ve put them on the end of an Imagine DSL line providing IPv4 and v6 over PPPoE and the end of a 30Mb UPC line taking its UPC IP via DHCP. They provide firewall, NAT, port forwarding, OpenVPN tunnels, QoS, DHCP, DNS caching and VLANs for phone / VoIP and managment networks.

 

 

 

World IPv6 Day – Do Something – Anything!

June 8th, 2011 is World IPv6 Day and on that day, Google, Facebook, Yahoo!, Akamai and Limelight Networkswill be amongst some of the major organisations that will offer their content over IPv6 for a 24-hour “test flight”.

I’m trying to push June 8th as a ‘flag day’ for smaller companies to get something – anything – done with IPv6. Enabling AAAA on their websites (and leaving it on) would be super. Some other suggestions I have:

  1. Register on IPv6Ready.ie and add the badge to your site. Even if it’s Pending IPv6, the whole point of the project is to nudge the level of awareness up a notch and we need badges on sites for that.
  2. If you haven’t even used IPv6 before, get a SixXS tunnel and be sure to choose either HEAnet, Airwire or Digiweb as your tunnel broker. All are members of INEX with good IPv6 connectivity so you’ll see low latency with good connectivity on these.
  3. If you want to get IPv6 on your LAN and your ISP won’t provide it, then (a) bug them some more; and (b) as a intermediate measure, also get a subnet from SixXS for your LAN.
  4. Dual stack your mail server and add a AAAA record to your MX hosts. This is a really simple and painless first step as SMTP is such a resilient protocol, if the mail cannot be delivered over v6, it’ll fall back to v4. Postfix, Sendmail and others have been IPv6 capable for years.
  5. Dual stack your DNS server. Like Postfix / Sendmail, Bind has been IPv6 capable for years. Get it listening on v6 and then add AAAA records to at least one of them.
  6. Hurricane Electric have a very useful IPv6 Certification program (see it at http://ipv6.he.net/certification/) which certifies an individuals ability. It’s a free process and what’s great about it is that, even if not interested in the cert, working through the process gets you configuring IPv6 on your web server, email server and DNS.
  7. Always look for IPv6 when choosing an ISP, a hosting provider, equipment vendors, and SaaS. Even if not a deciding factor, ask for IPv6 support to keep nudging it up the list of priorities for service providers.
  8. Register and display a badge from www.ipv6ready.ie. Did I say that already?

 

We’re IPv6 Ready! Are you?

IPv6 ReadyOver in INEX, we just launched a new initiative to promote and increase awareness of IPv6 among content owners and businesses generating revenue from an online presence.

This project is called IPv6 Ready and it is essential a certification program for websites that are IPv6 ready to one of two standards:

Gold: The website has a AAAA (IPv6) DNS record; and

Platinum: At least one of the websites DNS name servers is additionally IPv6 enabled.

IPv6 PendingFor those websites that are not IPv6 enabled (and in many cases this is dependent on a third party hosting company), we also have a very cool IPv6 Pending badge which you can use to let your customers know that you are IPv6 aware.

The badges shown here are the large versions but we also have an extra large, medium and small so you’ll find an appropriate one for your site.

How do you get your badges? Easy, just head over to IPv6Ready.ie and register your site. Once you complete the simple process, you’ll be emailed all four personalised badges!

Help us make this a success! Please repost, blog, tweet and spread the word any way you can to help us raise awareness and push IPv6 forward – even just a little. If nothing else, please register and display a badge! You’ll also get a link such as this to your own certificate!

Querying for DNS Glue Records (using dig)

On a project I’m working on, I need to establish if a domain has IPv6 glue records or not. If I had to do it on a once off, a whois lookup would answer that nicely:

$ /usr/bin/whois opensolutions.ie
<snip>
nserver:     dns1.dns.opensolutions.ie 87.232.1.40 2a01:268:4::40
nserver:     dns2.dns.opensolutions.ie 87.232.1.41 2a01:268:4::41
nserver:     dns3.dns.opensolutions.ie 87.232.16.61 2a01:268:3002::61

However, in this case, I will need to do it many times on many domains and do not need to have to worry about whois servers limiting the queries or parsing the output from different whois servers.

After some digging, it looks like the nameservers of TLDs return glue records in the additional section. Let’s look by example on opensolutions.ie. First, find the TLD servers for .ie:

$ dig NS ie
<snip>
;; ANSWER SECTION:
ie.                     172800  IN      NS      gns1.domainregistry.ie.
ie.                     172800  IN      NS      uucp-gw-1.pa.dec.com.
ie.                     172800  IN      NS      uucp-gw-2.pa.dec.com.
ie.                     172800  IN      NS      ns3.ns.esat.net.
ie.                     172800  IN      NS      banba.domainregistry.ie.
ie.                     172800  IN      NS      ice.netsource.ie.
ie.                     172800  IN      NS      gns2.domainregistry.ie.
ie.                     172800  IN      NS      ns-ie.nic.fr.
ie.                     172800  IN      NS      b.iedr.ie.

Now query one of these for the nameservers for opensolutions.ie:

$ dig NS opensolutions.ie @banba.domainregistry.ie.
<snip>
;; AUTHORITY SECTION:
opensolutions.ie.       172800  IN      NS      dns3.dns.opensolutions.ie.
opensolutions.ie.       172800  IN      NS      dns2.dns.opensolutions.ie.
opensolutions.ie.       172800  IN      NS      dns1.dns.opensolutions.ie.

;; ADDITIONAL SECTION:
dns1.dns.opensolutions.ie. 172800 IN    A       87.232.1.40
dns1.dns.opensolutions.ie. 172800 IN    AAAA    2a01:268:4::40
dns2.dns.opensolutions.ie. 172800 IN    A       87.232.1.41
dns2.dns.opensolutions.ie. 172800 IN    AAAA    2a01:268:4::41
dns3.dns.opensolutions.ie. 172800 IN    A       87.232.16.61
dns3.dns.opensolutions.ie. 172800 IN    AAAA    2a01:268:3002::61

As you can see, the authority section contains the nameservers for opensolutions.ie which are all on the opensolutions.ie domain. We then find the glue records for these nameservers in the additional section.

IANA IPv4 Free Pool Exhausted

The IANA IPv4 free pool was exhausted today, 3 February 2011. Each of the Regional Internet Registries (RIRs) has now received one of the final five /8s.

The RIPE NCC updated yesterday that the IANA IPv4 free pool has been exhausted:

The IANA IPv4 free pool was exhausted today, 3 February 2011. Each of the Regional Internet Registries (RIRs) has now received one of the final five /8s. The RIPE NCC has been allocated 185/8.

The RIPE NCC is holding reserves totaling approximately four /8s (around 75 million individual IPv4 addresses), not including 185/8.

RIPE will most likely exhaust their reserves sometime in 2011:

As we unable to anticipate consumption rates, we cannot fully predict how long our reserves will last. However, we would like to reassure you that our supplies will not be exhausted within the coming months.