Tag: ixp manager

A Brief History of IXP Manager

For another INEX project, I was asked to put together a timeline for IXP Manager – an open source application for managing Internet eXchange Points. Reproduced here:

IXP Manager was originally a web portal written in PHP by Nick Hilliard in 2005. It was a basic database frontend that just did fairly simple CRUD (CReate, Update, Delete operations) and allowed our members to log in and view their traffic usage graphs.

Around this was a ton of Perl scripts that sucked that data out of the the database and created configuration files for route collectors, graphing, monitoring, etc.

The major achievement of Nick’s original system was the database design (the schema). The core of that schema is still the core of IXP Manager over 10 years later.

I started in INEX in 2007 and started to expand IXP Manager using what was becoming a more modern web development paradigm – Model/View/Controller with a framework called Zend Framework.

There wasn’t a grand plan here – it was just “as we needed” organic growth over the coming years.

In 2010 we decided what we had was actually pretty good and could be very useful for other IXPs. We got committee approval to open source the software and we released IXP Manager V2 in 2010 under the GPL2 license (GNU Public License v2).

This license essentially means anyone can use the software free of charge but also that they should contribute back improvements that they may make. The idea being that INEX would eventually benefit from other IXPs contributing to the project.

Open sourcing a project doesn’t mean it’ll be successful though! What we didn’t do in 2010 was put infrastructure around it such as: presentations at IXP conferences, mailing lists for user support, decent documentation, etc.

We corrected all that and re-released an updated version called IXP Manager v3 in 2012. This time it took off! We also started collaborating with ISOC (The Internet Society) around this time to help start-up IXPs (mainly eastern Europe and Africa) use IXP Manager.

Some established IXPs also contributed money towards development of missing features – most notably LONAP in the UK – and these new features fed back into INEX.

We’ve worked hard on v3 and it’s developed well since with many new features and improvements. Sometime in late 2016 – maybe even this month – we’ll release v4 which is a major leap forward again and should hopefully attract new users and developers.

INEX is very well regarded in the IXP community as an exchange that is well run and both operates and teaches best practice. All of what we’ve learnt running a good exchange has fed into IXP Manager and it helps those IXPs that use it to implement those same good practices. IXP Manager has helped raise INEX’s reputation even further.

Lately we’ve begun to realise that as a small team we can’t do it all ourselves – the more exchanges that use it, the more requests for help and features we receive and as a result, new developments take a back seat.

To try and improve this we launched a new website in 2016 – http://www.ixpmanager.org/ – and issued a call for sponsorship so we could hire a full time developer. The ‘we’ here by the way is my and Nick’s own company – Island Bridge Networks. We’re doing this on a purely cost recovery basis. I’m delighted to say we’ve just about reached our funding goal with three top line sponsors all contributing about €20k each – ISOC, Netflix and SwissIX. The hiring process has now begun!

I’m also delighted to say that there are 33 exchanges around the world using IXP Manager /that we know of/.

Personal Profile for INEX

I was asked to write a personal profile for INEX in <= 300 words. Reproduced here.

If you want to confuse Barry, ask him where he’s from: born in Cork, spent his formative years in Galway and married into Dublin. He got a honours degree in Maths, his first love, from NUI, Galway in 2001 and went on to do four years research in information theory in UCD’s Computer Science department.

In 2005 he took a job with imag!ne to help build their ADSL broadband service from scratch to a position where it supported tens of thousands of subscribers. Barry branched out on his own in 2007 when he formed his own consultancy business, Open Solutions, which continued working with imag!ne as well as building up a portfolio of network, VoIP and web application development customers.

It was 2008 when Nick Hilliard, INEX’s CTO, approached Barry to provide a couple days operational support to INEX. Little did Barry realise what a huge part of his life INEX would become – both here in Ireland supporting INEX’s infrastructure and membership but also as part of the larger European and international IXP community.

Barry is the lead developer of IXP Manager (http://www.ixpmanager.org/) – a full stack management system for IXPs which includes an administration and customer portal; provides end to end provisioning; and both teaches and implements best practice. INEX is very proud to say that this project is now in use at over 33 IXPs and has grown legs of its own with the wider community sponsoring a full-time developer.

INEX has always been happy to help other IXPs and, through our relationship with the Internet Society (ISOC), RIPE and Euro-IX, Barry has travelled to countries with a less developed internet infrastructure to advise on best practice, has delivered a number of IXP Manager workshops and contributes to policy development.

IXP Manager – Planning for v4

A lot has changed in the 3 to 5 years that the decision was made to use certain libraries / technologies / methods on IXP Manager.

In previous major version changes we made some serious architecture changes in one sweep. For example v2 -> v3 saw the complete migration from Doctrine ORM v1 to v2 (which was a change from the Active Record pattern to the Data Mapper pattern).

Today, IXP Manager is a very large project and to do such a sweeping migration in one go would stifle development, break something that isn’t actually broken and take a lot of time.

But, sticking with older technologies and libraries has negative effects also. It creates developer apathy (for which I can personally vouch for). It also provides a major stumbling block for bringing on new developers and contributors (who wants to learn Zend Framework 1 now which has been EOL’d for sometime?).

So, our plan for v4 is to bring in new technologies without throwing away or rewriting everything we have.

IXP Manager is a MVC application that currently uses Doctrine2 as the Model, Smarty as the View and ZF1 as the Controller. Doctrine2 is still current and won’t be changing.

Smarty will remain as the view engine for current / unmigrated functionality. But Smarty is… oh my God… soooooo bad. v4 will default to Twig which is more modern and far better structured from a programming point of view. Coupled with the new framework, it will also allow for a nicer means of skinning. For the interested, Twig has some very nice features including layouts, macros and also some nice security features.

ZF1 has served us well but it’s been EOL’d and is now quite outdated. The new hotness in PHP is Laravel, which I’ve been using to great effect for a while now. Laravel show cases some of the new and best functionality of PHP and using very modern techniques (such as IoC).

But more importantly, Laravel will let us do things in a much different and much more flexible manner for the IXPs using IXP Manager. Some of these include:

  •  Job queues: built-in and simple (to use) support for job queues via Beanstalkd and others. Queuing jobs will provide functionality that we at INEX have been looking for (and it’s also an FAQ from other IXPs) -> reconfiguring services on demand (or, at least quicker than a twice daily cronjob).

Put this together with:

  • Events: Laravel allows us to trigger events and subscribe to them.

A key example of queue and event functionality would be that a change to a VLAN interface (such as checking the route server client box) would trigger a vlan interface changed event. One subscriber to this event would be the route server configuration manager. Based on the VLAN change, this event handler can then queue events. The route servers themselves would monitor these queues and rebuild / reconfigure the route servers appropriately on demand.

Similar handlers for route collectors, DNS ARPA changes, etc. can offer much more real time control of all the services at an IXP.

IoC decouples logic from the controller. What this means is that IXPs who want to do things differently than INEX (let’s say use Cacti instead of MRTG as an example), can swap out MRTG with Cacti with one line of code (that’s assuming we write contracts – interfaces – for such handlers and a Cacti version is coded of course!). But that’s the kind of power and flexibility we’re looking to bring in.

Other features Laravel provides includes:

  • Much improved unit testing on controller actions. Right now, we spin up Apache and MySQL to test controller actions. This is no longer required with Laravel making tests easier to write, more robust and more focused with built in support for mock objects.
  • A much nicer and more structured way of creating command line interfaces rather than the quite clunky way we have of doing it currently.
  • A much more natural way to develop REST API endpoints with json:api compatible responses.

And that leads us to the front end. Right now, the front end and the back end are tightly coupled. During the development lifetime of v4, we want to move more towards an API is Everything back end with a decoupled front end.

This separation will again aid unit testing providing a more reliable and robust IXP Manager. It will allow other IXPs to create their own front end on member facing portals or, even, move to IXP Manager as their back end system but retaining investment of current member portals by adding new features from IXP Manager through API endpoints. It will also allow existing systems in IXPs to integrate with IXP Manager to provision services and ports for example.

One of the bigger tests of this plan will be the (long awaited and badly needed) revamp of the member facing area. We’re currently planning the UI / UX of this to deliver key information to members in the best way possible. This will include Bootstrap v3 which is fluid from the ground up so mobile browsers to wide screen browsers should be supported naturally.

During the early stages of v4, we’ll create the API endpoints necessary to support the member portal functions and then create a front end on that using Ember.js.

Other changes in v4 will include:

  • A switch from package management via Git sub-modules to composer and Packagist as is current standard practice.
  • Introduction of Bower for front end asset management.
  • And we’ll need a task runner for pulling everything together – for that we’ll use Grunt (although that’ll mostly be a development / release prep tool rather than an end user requirement).

So, that’s what we’re looking at! It won’t happen overnight but we’ll continue our policy of release early, release often and we’ll update the documentation and provide complete upgrade instructions at the appropriate times. Some of the above is also subject to change depending on practical experience / issues as we move towards it.

Comments, ideas, etc. are all welcome.

Peering Week Articles on trefor.net

I spent the first few days of St Patrick’s week last month in Leeds at the first of the two annual Euro-IX conferences on behalf of INEX. Trefor Davies, of trefor.net, organised a series of articles called Peering Week on his blog to coincide with it:

During Peering Week we have had 18 excellent contributions from some of the people who run the internet in Europe. This might sound dramatic especially considering that the internet is made up of sixty or seventy thousand Autonomous Networks. The contributors this week run Internet Exchanges where a greats many of these networks connect to each other.

My contribution was about our IXP management system called IXP Manager – co-written by myself and Nick Hilliard for INEX. This tool is now being used to manage two IXPs in the UK, at least five more across Europe, a couple that we know about in the US and it is now the de facto choice for IXPs in Africa and Asia – where we are working with ISOC.

You can read the full article on Tref’s blog here: INEX’s IXP Manager – tools to help manage an Internet Exchange.

I’m glad to say that the good folks at Euro-IX helped ensure I wasn’t too homesick on St. Patricks’s Day – as the days proceedings wrapped up, we were greeted by:

guinness_array_header

TallyStick, ViMbAdmin and a CSS/JS Minify Tool

It’s been a busy few weeks:

  • We launched TallyStick – a time tracking and billing tool – two weeks ago and have pushed some bug fixes and updates. So far so good!
  • IXP Manager, an open source web application to assist in the management of Internet Exchange Points (IXPs) that we built over at INEX, got a complete UI refresh thanks to Twitter’s Bootstrap;
  • Similarly, our open source email domain / mailbox / alias management tool called ViMbAdmin got a major version bump, lots of new features and a UI refresh also;
  • We also just open sourced (BSD) our (admittedly small) Minify tool which makes minifying, bundling and versioning the manner JS and CSS files that make up websites these days a breeze. Check it out on GitHub: https://github.com/opensolutions/Minify.