Category: Work

Doctrine2 Provider for Laravel 5 with Authentication Support

About a year ago, I released a Doctrine2 provider for Laravel 4. We’ve just updated this as a new package for Laravel 5.  Full details can be seen in the project’s README.md.

This package creates four bindings into Laravel’s IoC:

  • Doctrine\ORM\EntityManagerInterface (which is an instance of Doctrine\ORM\EntityManager), also available via the D2EM facade;
  • Doctrine\Common\Cache\Cache (which is an instance of the appropriate cache provider), also available via the D2Cache facade;
  • Doctrine\ORM\Mapping\ClassMetadataFactory (used in this package by the console generator commands); and
  • the Doctrine2Bridge\Support\Repository helper object which used by the D2R facade to instantiate repositories such as D2R::r( 'User' ).

An example of using the D2Cache and D2EM

D2Cache::save( $key, $value );
D2Cache::fetch( $key );

$object = new Entities\Object;
D2EM::persist( $object );
D2EM::flush();

It includes support for Laravel’s authentication system by way of a Doctrine2 based user provider with some example classes. Simply put – this allows a Doctrine2 database table storing users’ usernames and passwords to be used as the backend for Laravel 5 authentication.

Full documentation of the authentication with examples can be found here.

Available on Packagist and forkable on GitHub.

Debugging NFS Slowness

During patching for the recent GHOST bug, I updated all packages (including kernel) on a Ubuntu 14.04 file server (filer). This filer provided static content (mainly tens of thousands of images) to a number of web servers. You can see the effect in the following load graph from the filer:

Load average on the filer
Load average on the filer

You may notice from the above, that there were actually two issues. The first was solved by upgrading the filer from 14.04 to 14.10 based on a number of online references to symptoms and fixes. About an hour after this upgrade, a new form of NFS slowness manifested and, needless to say, sites that rendered in <1sec were now taking >15secs.

Diagnosing the second issue took a while longer but some tips and utilities include:

  • check /var/log and see if any log files are increasing rapidly;
  • check top and check any processes with high / unusual utilisation;
  • use iostat (apt-get install sysstat) and pay particular attention to any devices with high volumes of transactions per second. In my case it was the root filesystem rather than any of the mounted partitions exported by NFS.
  • use iotop (apt-get install iotop) and note any processes with high utilisation (in my case jbd2/xvda1-8 was at 100% and xvda1-8 is my root partition)

The jbd2 process is the ext4 journaling process. At this point you can evaluate fsck’ing your partition but I wanted to see if I could discover what was happening here. I enabled some debugging via:

# enable tracing:
echo 1 > /sys/kernel/debug/tracing/events/ext4/ext4_sync_file_enter/enable
# wait a couple of seconds and:
cat /sys/kernel/debug/tracing/trace
# and disable tracing:
echo 0 > /sys/kernel/debug/tracing/events/ext4/ext4_sync_file_enter/enable

What I found were lots of:

nfsd-2085  [001] .... 53730942.155573: ext4_sync_file_enter: dev 202,1 ino 276278 parent 149955 datasync 0
nfsd-2071  [001] .... 53730942.158743: ext4_sync_file_enter: dev 202,1 ino 276278 parent 149955 datasync 0
...

where every entry related to the same inode number (276278). We found this via:

find / -inum 276278
/var/lib/nfs/v4recovery

The solution was to stop nfs_kernal_server, remove that directory entirely, add it back and restart the nfs_kernel_server. We got the permissions wrong on the first attempt but this’ll be obvious from dmesg / kernel log messages such as:

kernel: [53731827.778104] NFSD: Failed to remove expired client state directory 8d97cccceb37641d3804a84683a9282a
kernel: [53731827.779204] NFSD: failed to write recovery record (err -13); please check that /var/lib/nfs/v4recovery exists and is writeableNFSD: Failed to remove expired client state directory 8d97cccceb37641d3804a84683a9282a

Nagios Plugin to Check Extreme Networks Devices

Over at INEX we’ve embarked on a forklift upgrade of the primary peering LAN using Extreme Networks Summit x670’s and x460’s. As usual, we need to monitor these 24/7 and we have just written a new Extreme Networks chassis monitoring script which should work with most Extreme devices.

It will check and generate alerts on the following items:

  • a warning if the device was recently rebooted;
  • a warning / critical if any found temperature sensors are in a non-normal state;
  • a warning / critical if any found fans are in a non-normal state;
  • a warning / critical if any found PSUs are in a non-normal state (or missing);
  • a warning / critical if the 5 sec CPU utilisation is above set thresholds;
  • a warning / critical if the memory utilisation is above set thresholds.

You’ll find the script in this Github repository: barryo/nagios-plugins.

A some verbose output follows:

./check_chassis_extreme.php -c  -h 10.11.12.13 -v 
 
CPU: 5sec - 4% 
Last reboot: 4007.7666666667 minutes ago 
Uptime: 2.8 days. 
PSU: 1 - presentOK (Serial: 1429W-xxxxx; Source: ac) 
PSU: 2 - presentOK (Serial: 1429W-xxxxx; Source: ac) 
Fan: 101 - OK (4388 RPM) 
Fan: 102 - OK (9273 RPM) 
Fan: 103 - OK (4428 RPM) 
Fan: 104 - OK (9273 RPM) 
Fan: 105 - OK (4551 RPM) 
Fan: 106 - OK (9452 RPM) 
Temp: 39'C 
Over temp alert: NO 
Memory used in slot 1: 29% 
OK - CPU: 5sec - 4%. Uptime: 2.8 days.  PSUs: 1 - presentOK; 
  2 - presentOK;. Overall system power state: redundant power 
  available. Fans: [101 - OK (4388 RPM)]; [102 - OK (9273 RPM)];
  [103 - OK (4428 RPM)]; [104 - OK (9273 RPM)]; [105 - OK (4551 
  RPM)]; [106 - OK (9452 RPM)];. Temp: 39'C. 
  Memory (slot:usage%): 1:29%.

OS X Built-in tftp Server

Turned out to be very useful during a recent RMA maintenance window:

sudo launchctl load -F /System/Library/LaunchDaemons/tftp.plist
sudo launchctl start com.apple.tftpd

The default tftp file path is /private/tftpboot. [Original source]

You can stop it with:

sudo launchctl unload -F /System/Library/LaunchDaemons/tftp.plist

And, speaking of tftp – there are some interesting projects on GitHub: hooktftp, php-tftpserver and ptftpd.

IXP Manager – Planning for v4

A lot has changed in the 3 to 5 years that the decision was made to use certain libraries / technologies / methods on IXP Manager.

In previous major version changes we made some serious architecture changes in one sweep. For example v2 -> v3 saw the complete migration from Doctrine ORM v1 to v2 (which was a change from the Active Record pattern to the Data Mapper pattern).

Today, IXP Manager is a very large project and to do such a sweeping migration in one go would stifle development, break something that isn’t actually broken and take a lot of time.

But, sticking with older technologies and libraries has negative effects also. It creates developer apathy (for which I can personally vouch for). It also provides a major stumbling block for bringing on new developers and contributors (who wants to learn Zend Framework 1 now which has been EOL’d for sometime?).

So, our plan for v4 is to bring in new technologies without throwing away or rewriting everything we have.

IXP Manager is a MVC application that currently uses Doctrine2 as the Model, Smarty as the View and ZF1 as the Controller. Doctrine2 is still current and won’t be changing.

Smarty will remain as the view engine for current / unmigrated functionality. But Smarty is… oh my God… soooooo bad. v4 will default to Twig which is more modern and far better structured from a programming point of view. Coupled with the new framework, it will also allow for a nicer means of skinning. For the interested, Twig has some very nice features including layouts, macros and also some nice security features.

ZF1 has served us well but it’s been EOL’d and is now quite outdated. The new hotness in PHP is Laravel, which I’ve been using to great effect for a while now. Laravel show cases some of the new and best functionality of PHP and using very modern techniques (such as IoC).

But more importantly, Laravel will let us do things in a much different and much more flexible manner for the IXPs using IXP Manager. Some of these include:

  •  Job queues: built-in and simple (to use) support for job queues via Beanstalkd and others. Queuing jobs will provide functionality that we at INEX have been looking for (and it’s also an FAQ from other IXPs) -> reconfiguring services on demand (or, at least quicker than a twice daily cronjob).

Put this together with:

  • Events: Laravel allows us to trigger events and subscribe to them.

A key example of queue and event functionality would be that a change to a VLAN interface (such as checking the route server client box) would trigger a vlan interface changed event. One subscriber to this event would be the route server configuration manager. Based on the VLAN change, this event handler can then queue events. The route servers themselves would monitor these queues and rebuild / reconfigure the route servers appropriately on demand.

Similar handlers for route collectors, DNS ARPA changes, etc. can offer much more real time control of all the services at an IXP.

IoC decouples logic from the controller. What this means is that IXPs who want to do things differently than INEX (let’s say use Cacti instead of MRTG as an example), can swap out MRTG with Cacti with one line of code (that’s assuming we write contracts – interfaces – for such handlers and a Cacti version is coded of course!). But that’s the kind of power and flexibility we’re looking to bring in.

Other features Laravel provides includes:

  • Much improved unit testing on controller actions. Right now, we spin up Apache and MySQL to test controller actions. This is no longer required with Laravel making tests easier to write, more robust and more focused with built in support for mock objects.
  • A much nicer and more structured way of creating command line interfaces rather than the quite clunky way we have of doing it currently.
  • A much more natural way to develop REST API endpoints with json:api compatible responses.

And that leads us to the front end. Right now, the front end and the back end are tightly coupled. During the development lifetime of v4, we want to move more towards an API is Everything back end with a decoupled front end.

This separation will again aid unit testing providing a more reliable and robust IXP Manager. It will allow other IXPs to create their own front end on member facing portals or, even, move to IXP Manager as their back end system but retaining investment of current member portals by adding new features from IXP Manager through API endpoints. It will also allow existing systems in IXPs to integrate with IXP Manager to provision services and ports for example.

One of the bigger tests of this plan will be the (long awaited and badly needed) revamp of the member facing area. We’re currently planning the UI / UX of this to deliver key information to members in the best way possible. This will include Bootstrap v3 which is fluid from the ground up so mobile browsers to wide screen browsers should be supported naturally.

During the early stages of v4, we’ll create the API endpoints necessary to support the member portal functions and then create a front end on that using Ember.js.

Other changes in v4 will include:

  • A switch from package management via Git sub-modules to composer and Packagist as is current standard practice.
  • Introduction of Bower for front end asset management.
  • And we’ll need a task runner for pulling everything together – for that we’ll use Grunt (although that’ll mostly be a development / release prep tool rather than an end user requirement).

So, that’s what we’re looking at! It won’t happen overnight but we’ll continue our policy of release early, release often and we’ll update the documentation and provide complete upgrade instructions at the appropriate times. Some of the above is also subject to change depending on practical experience / issues as we move towards it.

Comments, ideas, etc. are all welcome.

ViMbAdmin and Vacation / Out of Office Auto-responders

An FAQ post for future reference because, well, it’s a FAQ!

ViMbAdmin is primarily a database frontend for managing virtual mailboxes with mail services such as Postfix and Dovecot.

A frequently asked question is does it / can it support vacation / out of office auto-responders. The short answer is no.

The main way to handle OOO responders is via SIEVE (e.g. Dovecot’s version). To put a front end on this, you need a server configured for external SIEVE management and a client with SIEVE support (which ViMbAdmin does not have).

I’ve done it in three ways in the past:

  1. directly editing the .dovecot.sieve file. This is only really good for people with access to the server and some clue.
  2. Using a Thunderbird plugin allowing editing of SIEVE files. Again, clue required.
  3. For one customer, they can do it via the SoGO web interface we installed for them. This is a very nice interface for people migrating from Exchange.

We did look at fixing a Thunderbird Out of Office plugin but didn’t get very far.

What we all need here is for someone for write / sponsor a decent SIEVE based Thunderbird OOO plugin. Here’s some prior art: https://addons.mozilla.org/en-US/thunderbird/addon/sieve.

Cordova, Facebook, Android and no Eclipse

This post relates to creating an Android application with Facebook support (via phonegap-facebook-plugin using Cordova 3.5 and, specifically, without Eclipse. Most existing documentation expects you to spin up Eclipse to link the Facebook libraries – but, we’re using an automated build system with Grunt so that’s not a possibility.

Firstly – the steps to get it working:

  1. Add the Cordova Facebook plugin: 
    cordova plugin add https://github.com/phonegap/phonegap-facebook-plugin \
    --variable APP_ID="1234567890" 
    --variable APP_NAME="Test App"
  2. Edit platforms/android/project.properties and add: 
    android.library.reference.2=FacebookLib

    where n is the next available reference index. Also note the target parameter here as you’ll need it next:

  3. Update the Android project for the plugin: 
    cd platforms/andoird/FacebookLib
android update project --target android-19 -p .

     where android-19 should match the target parameter from above.

  4. Prepare and build your application: 
    cordova prepare android
cordova compile android

Development Contracts

At Open Solutions, we tend to undertake a lot of fixed price contracts to develop web applications. In fact, clients usually insist on fixed price contracts as they want to know in advance what the bill will be.

However, fixed price contracts have big negatives for both parties:

  • for the client, a fixed price contract can often limit them to their earliest ideas. Now, as a service provider, we want to be flexible and so we’re happy to chop and change as a project develops. But, this leads to:
  • for the service provider, if change and revision requests are not carefully managed agreed and billed for, the service provider could very quickly end up making a loss on the contract and thus find themselves in the position of funding their clients project!

To this end, we’ve recently been reviewing various web development contracts and have found some nice inspiration for basing our own on.

Following the success of Killer Contract, Andy wrote a plain language NDA (also available as a Gist).

Peering Week Articles on trefor.net

I spent the first few days of St Patrick’s week last month in Leeds at the first of the two annual Euro-IX conferences on behalf of INEX. Trefor Davies, of trefor.net, organised a series of articles called Peering Week on his blog to coincide with it:

During Peering Week we have had 18 excellent contributions from some of the people who run the internet in Europe. This might sound dramatic especially considering that the internet is made up of sixty or seventy thousand Autonomous Networks. The contributors this week run Internet Exchanges where a greats many of these networks connect to each other.

My contribution was about our IXP management system called IXP Manager – co-written by myself and Nick Hilliard for INEX. This tool is now being used to manage two IXPs in the UK, at least five more across Europe, a couple that we know about in the US and it is now the de facto choice for IXPs in Africa and Asia – where we are working with ISOC.

You can read the full article on Tref’s blog here: INEX’s IXP Manager – tools to help manage an Internet Exchange.

I’m glad to say that the good folks at Euro-IX helped ensure I wasn’t too homesick on St. Patricks’s Day – as the days proceedings wrapped up, we were greeted by:

guinness_array_header