Friday, May 15, 2015

Better than best effort — Rise of the IXP

This is the second installation in my Better than Best Effort series.  In my last post I talked about how the incentive models on the Internet keep ISPs from managing peering and backbone capacity in a way that supports reliable communication in the face of the ever growing volume of rich media content.  If you haven't done so, please read that post before you read this one.

It's clear that using an ISP for business communication comes with the perils associated to the "noisy neighbor" ebb and flow of consumer related high volume data movement.  Riding pipes that are intentionally run hot to keep costs down is a business model that works for ISP, but not for business users of the Internet.  Even with business Internet service, customers may get a better grade of service within a portion of an ISP's networks, but not when their data needs to traverse another ISP which they are not a customer of.  There is no consistent experience, for anyone.

However there is an evolving solution to avoid getting caught in the never ending battle between ISP and large consumer content.  As the title of this blog gives away, the solution is called an Internet eXchange Point (IXP).

IXPs are where the different networks that make up the Internet most often come together.  Without IXP, the Internet would be separate islands of networks.  IXP are, in a sense, the glue of the Internet.  From the early days of the Internet, IXP have been used to simplify connectivity between ISPs resulting in significant cost savings for them.  Without an IXP, an ISP would need to run separate lines and dedicate network ports for each peer ISP with whom it connects.

However IXP and ISP are not distant relatives.  They are in fact close cousins.  Here's why.

Both ISP and IXP share two fundamental properties.  The first is that they both have a fabric, and the second is that they both have "access" links used to connect to customers so they can intercommunicate over this fabric.  The distinction between the two is in the nature of the access interfaces and the fabrics.  ISP fabrics are intended to reach customers that are spread out over a wide geographic area.  An IXP fabric on the other hand is fully housed within a single data center facility.   In some cases an IXP fabric is simply a single Ethernet switch.  ISP access links use technology needed to span across neighborhoods, while IXP access links are basically ordinary Ethernet cables that run an average of around several dozen meters.  So essentially the distinction between the two is that an ISP is a WAN and an IXP is a LAN.

The bulk of the cost in a WAN is in the laying and maintenance of the wires over geographically long distances.  Correspondingly the technology used at the ends of those wires is chosen based on the ability to wring out as much value out of those wires as possible.   The cost of a WAN is significantly higher than a LAN with a comparable number of access links.  ISP need to carefully manage costs which are much higher per byte.  It is on account of the tradeoffs that ISP make in order to manage these costs that the Internet is often unpredictably unreliable.

So how can IXP help?

Let's assume that most business begin to use IXP as meet-me points.  Remember that the cost dynamics of operating an IXP are different than an ISP.  At each IXP these business customers can peer with one another and their favorite ISPs for the cost of a LAN access link.   

There are at least a few advantages of this over connecting to an ISP.  Firstly, two business entities communicating via an IXP's [non-blocking] LAN are effectively not sharing the capacity between them with entities that are not communicating with either of them, and so making their experience far more predictable.  This opens the door for them to save capital and operational costs by eliminating the private lines that they may currently have with these other business entities.  Secondly, a business that is experiencing congestion to remote endpoints via an ISP can choose to not use it by [effectively] disabling BGP routing with that ISP.  This is different from the standard access model used by businesses in that if there are problems downstream of one of their ISP, their control is limited to the much fewer ISP from whom they have purchased access capacity.  

The following illustrates a scenario where there is congestion at a peering point downstream of one of the ISP being used by a business that is affecting it's ability to reach other offices, partners or customers that are hosted on other ISP.

In the access model, since BGP cannot signal path quality, traffic is blindly steered over a path that has the shortest number of intermediate networks versus a path with the best performance.  Buying extra access circuits alone to avoid Internet congestion is not a winnable game (more on this in the next part of this series).

The alternative approach using an IXP would look something like the following.

This illustration shows how being at an IXP creates more direct access to more endpoints at a better price point than buying access lines to numerous ISP.  You can also see how peering with other business entities locally at an IXP can improve reliability, reduce costs and simplify business-to-business connectivity by combining it with Internet connectivity.

There is an interesting trend occurring within the growing number of managed co-location data centers. Hosted within many of these co-location data centers are IXP. Some managed data center operators like Equinix even operate their own IXP at their data centers. These data centers are an ideal place for businesses to connect with one another through IXP without the downsides that come with using consumer-focused ISP.  

This is not to say that the operational capabilities at all IXP are at a level needed to support large numbers of businesses. There is work to be done to scale peering in a manner that will give customers minimal configuration burden and maximal control.

There will even be need for business-focused ISPs that connect business customers at one IXP to business customers connected to the Internet at another IXP. Although net neutrality prohibits the differentiated treatment of data over the Internet, it does not forbid an ISP or IXP from selecting the class of customer it chooses to serve. This is much like the difference between a freeway and a parkway. Parkways do not serve commercial traffic and so in a way they offer a differentiated service to non commercial traffic.

As the Internet enables new SaaS and IaaS providers to find success by avoiding the high entrance cost of building a private service delivery network, more businesses are turning to the Internet to access their solution providers of choice. The old Internet connectivity model cannot reliably support the growing use of the Internet for business and so a better connectivity model is needed for a reliable Internet. New opportunities await.

In upcoming posts I will discuss additional thoughts on further improving the reliability of communicating over the Internet.

1 comment:

  1. Internet exchange infrastructure boom boosts African Internet growth as well...Its also increase the risk of security measures. There have been many cases worldwide which have highlighted the importance of privacy protection tools; ranging from the Edward Snowden files to The Pirate Bay take downs. While i do not condone any illegal activity, i believe that protecting your online activities should be a human right. Choosing the right VPN can be daunting but this is where we come in. If you live in Usa and haven’t thought about purchasing ExpressVPN yet, then it’s high time to do so.