Chewtoy's Technical Musings

Monkeying Around with IPv6

2011-03-17T21:44:00.000Z

Part of the reason I haven't been posting much here recently is because I am buggering around in my spare time trying to polish up the programming skills that I have been neglecting over the past couple of years.

If I ever get my current project finished I will post more about it here; however in the mean time though I came across an issue that I just had to share.

I am currently writing a networked application. To be honest I am just interested in the program logic and was aiming to completely bypass the low level stuff by using an application framework. In theory this is exactly what Twisted is for.

Things were going well until I threw a spanner in the works. I noticed that my app was only listening on an IPv4 socket and wanted to change this. It turns out that Twisted does not have any support for IPv6. There is a wiki article, and it appears that more talented coders than I have been submitting patches for at least 3 years, yet one of the most common networked application frameworks on one of the most popular scripting languages is still IPv4 only.

There are a number of ways forward here, and I considered three main alternatives.

The first is the morally right position. Open source works because of contributions from the community. If I am going to be taking advantage of this framework then I should be willing to give something back. After several hours of picking apart the various modules making up the framework I basically came to the conclusion that I just don't have the skills. Being good enough to use a module doesn't make you good enough to contribute. Given that one of my stated aims on this project was to avoid getting bogged down in the low level details, I just don't have the time or experience to do the right thing here.

The second option is to change framework. I spent another couple of hours researching this one (this blog post has a good overview of some of the alternative options). I even made a start on getting what I have so far ported to gevent but it didn't seem to match what I was doing as well as the Twisted approach did, so I aborted this effort.

This brings me to the third option: doing as little as possible to get Twisted working for IPv6 in my particular situation. I started off trying to do this cleanly by deriving IPv6 capable classes that inherit from the IPv4 classes in the standard distribution. Twisted is a big package and this looked like it would rapidly become a massive undertaking. So I gave up on the clean way and started directly replacing variables and methods in the Twisted classes. A practice known as Monkey Patching.

It turns out it only took a few lines of code to get things working to a suitable level.

import socket
from twisted.internet import tcp

def monkeypatch_method(cls):
   def decorator(func):
   setattr(cls, func.__name__, func)
   return func
   return decorator

# !!! Wind up your windows, here be monkeys !!!

tcp.Port.addressFamily = socket.AF_UNSPEC
@monkeypatch_method(tcp.Port)
def _buildAddr(self, peer):
host, port = peer[:2]
return address._ServerFactoryIPv4Address('TCP', host, port)

# !!! End monkeys. Please report to the visitor centre for cleansing. !!!
This has scratched my itch and got me past a log jam that has been holding me up for a couple of days. It also highlights a bigger problem though.

In the networking industry, especially when talking about the capital 'I' Internet it is generally accepted that IPv6 isn't just a curiosity any more, it is a real part of daily operations. My experience over the past few days has shown me that the supporting work needed in the development community to take advantage of IPv6 when it is available seems to be lagging behind. Sure it is easy to write an IPv6 compatible application if you use the low level socket API directly, but unless the high level frameworks start providing the hooks to take advantage of this there will continue to be a lack of compatible applications.

And yes, I do recognise that I am part of the problem but I'm not sure what I can do about it.

What next for RIPE post IPv4?

2011-02-09T23:30:00.000Z

This is sort of a follow up to my last post on "What next for IANA". The idea was prompted by a project that Greg is working on for an upcoming episode of Packet Pushers. Basically the idea is to get together a list of questions for the RIRs present at APNIC31 in a couple of weeks.

Having worked for a number of LIRs with responsibility for communications with RIPE over the past 15 years I looked at the list of questions and could see that maybe some of the questions weren't phrased correctly, or in some cases showed a fundamental misunderstanding of how the RIRs function. As a good netizen I left a comment pointing out the difference between allocation and assignment but in general I think that the list of questions is a good one and while I think I know how RIPE will answer, I have had only a little interaction with ARIN and virtually none with APNIC, AfriNIC or LACNIC. I am looking forward to the episode to get these other views.

I did want to have a go at an answer to this question though:

Given that ipv6 is so expansive and that enteprises are likely to only ever get one allocation from their RIR – how does your RIR plan on generating re-occuring revenue’

In the last post I pointed out that maintaining the 256 IPv4 /8 blocks is only a small part of what IANA does. In this post I will concentrate on how direct assignment of PI space is not the primary revenue stream for the RIRs.

Lets start with a simple question: What is RIPE? The proper name for the RIR commonly known as RIPE is actually the RIPE NCC (RIPE Network Coordination Centre). RIPE NCC are a not for profit organisation funded by its members (LIRs), with the purpose of administering resources on behalf of the RIPE community and according to the policies set by the community. This is an important point. RIPE NCC do not set policy, they only apply it.

What is the RIPE community? The RIPE community is basically anyone with an interest in the management of Internet resources within the RIPE region. Policies within the RIPE community are generated using a consensus model in various working groups, much like in the IETF. The policy working groups are open to everyone, not just to paying members of the RIPE NCC. If you disagree with the IPv6 PI assignment policy, you are free to join up to the Address Policy working group and submit a proposal to modify the policy. If you can argue your case and gain consensus then you could end up changing policy.

RIPE policies are published openly in the RIPE Document Store. There are a large number of documents here and alongside the policies, request forms and guidance notes you will also find documents such as RIPE NCC Activity Plan, (which gives answers to several of the questions in Greg's post), the RIPE NCC Budget and the RIPE NCC Charging Scheme.

Next: What is an LIR? That's easy right? An LIR is an ISP. Actually it isn't as simple as that. RIPE state in their FAQs:

Do I need to become an LIR?

The RIPE NCC leaves the decision whether or not to become a Local Internet Registry (LIR) up to the organisation itself. Any organisation with a legally established office in the RIPE NCC service region can become an LIR.

We advise organisations who require their own routeable block and are in need of a large amount of addresses (/21) to become an LIR. In other cases an organisation in need of addresses should first try to acquire address-space from an upstream provider to ensure the conservation and aggregation goals as promoted by the RIPE community.

For more information regarding the setup of a Local Internet Registry, please visit our new members area.

So an LIR is not an ISP. It is an organisation that requires a large amount of address space (i.e. it could be you).

Getting back to the original question: if you look at the 2011 budget you will see that Independent number resources account for €1.2M of the RIPE NCC income. This isn't pocket change; however it is actually only around 7% of the total income. The bulk of the income come from LIRs paying their membership dues. This money is not going to go away because people aren't getting extra IPv6 assignments. One thing that you always need to keep in mind is that you do not buy IP addresses. When you pay RIPE for registration you are paying them administration fees to grant you a license to use those addresses. That license is subject to your justification for needing the addresses remaining valid and you keeping up to date with your fees.

When you request direct resources from RIPE, they will charge you at a rate equivalent to the smallest size category of LIR. For the same amount you could become a full LIR with voting rights at the RIPE NCC general meeting as well as free training and tickets to the RIPE meeting. Bear in mind that with great power comes great responsibility. By becoming an LIR you are agreeing to abide by the IPv4 and IPv6 allocation and assignment policies. Alternatively you could ask your service provider to sponsor your application for these resources. LIRs are only charged €50 a year per independent resource, there is a fair amount of administration involved though so don't be surprised if they actually charge you more than this. It will still be considerably cheaper than going direct though.

Hopefully this post will make the role of the RIPE NCC clearer. They do an important job, and I personally do not begrudge paying the membership fee.

What happened to IPv5? What is IANA going to do now?

2011-02-04T18:30:00.000Z

A couple of comments that I see all the time are "What was IPv5?" and "What are IANA going to do now there is no more IPv4 to allocate?".

I'm not really going to attempt to answer the first one as many people have already done so. What I will do is tell you how to find out yourself while answering the second question.

If you have been reading the news recently you probably know that IANA, the Internet Assigned Numbers Authority has run out of IPv4 addresses to allocate to the Regional Internet Registries. From reading these stories you could probably also be forgiven for thinking that this was their entire purpose. The clue is in the name though. IP addresses are far from the only numbering resource for which a central directory is required.
Every time that an RFC document defines an option that has a pre-defined value that information needs to be captured so that there are no conflicts. TCP / UDP port numbers, IP protocol numbers, BGP AFI/SAFI values, these all need to be tracked and if you go to the IANA website you will find an easy interface for navigating these registrations. A short period of time should confirm to you that there is actually a hell of a lot of information in the registries that they maintain, and the 256 IPv4 /8 networks are actually only a tiny fraction. IANA has plenty of work to get on with post IPv4 distribution.

As a specific example of how to use the IANA database lets take a look at the IP version number. Here we see:

Decimal	Keyword	Version	Reference
0-1		Reserved	[Jon_Postel][RFC4928]
2-3		Unassigned	[Jon_Postel]
4	IP	Internet Protocol	[RFC791][Jon_Postel]
5	ST	ST Datagram Mode	[RFC1190][Jim_Forgie]
6	IPv6	Internet Protocol version 6	[RFC1752]
7	TP/IX	TP/IX: The Next Internet	[RFC1475]
8	PIP	The P Internet Protocol	[RFC1621]
9	TUBA	TUBA	[RFC1347]
10-14		Unassigned	[Jon_Postel]
15	Reserved		Jon_Postel]

So you can see here that version 4 is IP, version 5 is ST Datagram Mode (defined in RFC1190) and version 6 is IPv6 defined in RFC1752. An interesting thing to note here is that IP versions 7-9 have also been allocated. This means that when something beyond IPv6 is required we will need to jump at least to IPv10. I doubt that I will ever have to answer an end user asking "What happened to IPv7?" in my lifetime though (have I just sounded the death knell of IPv6 by making that prediction?).

So what is this IPv6 thing and does it affect me?

2011-02-02T23:14:00.003Z

It depends on who you are.

Lets start at the beginning. How much do you understand about IPv4? If you don't know your CIDR from your FCoTR then you really shouldn't worry too much. The main thing you should know is that your service provider needs to understand IPv6 and you may need to make some changes to your computer or router hardware in the next year or two. If done right the transition from IPv4 to IPv6 can be transparent to the end user (you) and there is nothing to worry about. Sit back and enjoy the media running around like headless chickens trying to make a scare story out of something they have no understanding of... Actually that is another point - the media do not understand this any better than you do. I have read some truly shocking coverage over the last few weeks making it obvious that even the tech press really don't understand how this stuff works so don't worry if it is going over your head.

The next step up is the power user. Do you understand how to configure your home router? Do you tweak your NAT settings or have a separate network segment for your open wifi network? You should probably check out some basic IPv6 tutorials. I found the Hurricane Electric certification quite good to get some guided practice on IPv6 network configuration. It isn't going to turn you into a network engineer but it will give you a grounding. You may want to get IPv6 access now using a tunnel (he.net and sixxs.net are both good options. SixXS will even work through NAT or on a dynamic IP).

If you can get native IPv6 from your provider this should always be preferred over a tunnel. Make sure that your provider give you more than a single /64 prefix. Push for at least a /56 so that you have the ability to split your network even if you don't have plans to do so straight away.

Make sure you have a router that is IPv6 capable. This might be an ebay special (my home Cisco 1801 came from ebay) or it might be a Linksys running the OpenWRT / Tomato firmware. It could be an FB2700 from Firebrick or it could be one of the new generation of consumer grade CPEs that are finally starting to appear (beware of limited features here though). For a home network stateless autoconfig (SLAAC) with router advertisements (RA) will probably do you. DHCPv6 may make an interesting extra credit assignment.

By the way: a thought came to me today. The APNIC region is the region first set to run out of addresses. Some estimate put them into their final /8 policy by July / August this year. Given that the APNIC region is the home territory of many of the biggest manufacturers of consumer electronic devices in the world. I would be very surprised if there is not a huge influx of devices from this market by the end of the year.

The next sector I want to talk about is the service provider market. You guys are doing it already, right? If you are in the SP market, or any other area where offering services across the "capital I" Internet, then you should be deploying now, if not already deployed. Visit your local operator group. The past month has seen both UKNOF and Nanog meetings. Real world experience of IPv6 was freely given at both meetings. The Internet is IP of all versions. If you do not deploy IPv6 you do not offer Internet access. Simple as that. If you need native IPv6 transit drop me a line ;)

Before I delve into a few more specific technologies just a quick comment on Enterprise networks. Your ease of IPv6 deployment is going to vary based on your network size. You have a lot of decisions to make and a lot of technologies to investigate. Think about any services you offer over the Internet first. Devices on your DMZ. IPv4 is likely to get painful in certain places of the world. Have a CEO who regularly travels to China? Your support desk will probably appreciate you making the VPN concentrator IPv6 capable so they don't have to troubleshoot running IPSec over a doubly NATed IPv4 connection...

Do not feel pressured into pushing IPv6 out to everything too fast though. When did you last need to ask your ISP for more addresses? If you have enough to satisfy your growth requirements for a couple of years and your internal communications are currently working on IPv4, why rush things? Take your time, plan carefully and get it right. You are going to have to roll out IPv6 though, and likely within the operational lifespan of the equipment you are buying today. Make sure you are buying IPv6 capable hardware. Take a hard line with your supplier and do not accept them charging a premium. IPv6 is not Internet HD, there is only one Internet.

Think carefully about how you want your network to function. Do you want to go dual stack on everything? Or do you want to go single stack? Each has trade-offs. Dual stack is the easiest to deploy and removes the need for gateway / CGN devices within your network infrastructure but it means that you need to run two stacks on your end-points, and potentially need to maintain multiple sets of access lists in your firewalls. If you stay single stack IPv4 you will probably work fine for a long time to come. You will need to account for protocol translation at the edge though. If you take service from an IPv6 enabled cloud service then you need to be careful how your users access it. Things will gradually get more painful for IPv4 users as additional layers of translation get added and support starts to get dropped as demand tails off. The remaining option is to go single stack IPv6. This will probably start painful as the majority of your traffic needs to go through NAT64 or some form of ALG. As more and more IPv6 content becomes available the pain will lessen. You will need to upgrade any legacy devices though. Windows XP is still common in the enterprise and needs IPv4 for DNS resolution. Many IP phones are IPv4 only. While you transition sites from IPv4 only to IPv6 only you will need to use translation for internal traffic as well as external traffic.

After all this waffle we finally get to the original topic I wanted to cover. Addressing and translation / migration strategies.

First the simplest. Pure dual stack. IPv4 and IPv6 addresses running on the same network segments and passing each other on the wire. The IPv4 may be on globally routed addresses or behind a single layer of NAT. This is the most commonly deployed method in service provider networks today. It pretty much Just Works(TM).

As discussed previously the type of addressing factors in here. There is no NAT in IPv6 (there is an internet draft for NPTv6 but there is a lot of resistance to NAT in IPv6 and may never make it to RFC). So this leaves a couple of options.

You can number your network using PA space from your provider. This is simple and cheap initially, but if you have a large network renumbering is not easy which ties you to your provider.

You can go all out and get yourself some PI space. This may not be cheap, although looking at the RIPE schedule of charges it doesn't seem too bad to me given the benefits (~€1300 per annum for direct assignment). This cost can be a lot lower if you get one of your providers to sponsor you. I say providers because there is a limitation: in order to get IPv6 PI space you need to be multi-homed. If you run BGP and are multihomed today PI is probably your best choice. If you are not then you need to weigh up the costs of renumbering against the costs of becoming multihomed.

The other cited method is as follows: Get yourself a PA assignment from your ISP to use for Internet communications. Use ULA addressing (equivalent to RFC1918 in IPv4) for your internal communications. If you need to change provider your internal services will remain available continuously during renumbering. This may or may not answer your needs.

The other side of dual stack is the question of how the IPv4 is delivered.

Today you generally have public addresses for the outside of your firewall or router with RFC1918 inside. In certain cases in the future you may actually find that you need to put RFC1918 space on the outside too and get access to IPv4 Internet services via a second Carrier grade NAT (CGN) within the service provider core. This is commonly known as NAT444. This probably won't affect your enterprise sites, but is already common for mobile broadband and may start to become the case for home users on consumer broadband connections too.

NAT444 generally assumes dual stack in the access network. There are also options that have a single stack access network. RFC5969 defines 6RD, which is a way of delivering dual stack to end users across an IPv4 network using techniques similar to 6to4. The converse of this is DS-Lite which allows IPv4 access across a pure IPv6 access network.

If you want to go single stack you will likely need NAT64. Read the framework carefully and be aware of the limitations. NAT64 works pretty well when going from a v6 client to a v4 server, but can be troublesome in the other direction.

Other options open to you are use of ALGs (SIP SBC, Cacheing Web Proxy, use of a load balancing appliance with dual stack on outside talking to back end servers using single protocol) but the application specific nature here is likely to be limiting.

If the fact that fundamental parts of the transition methodology are still only drafts rather than RFCs... well done. Much of what is neede for Internet IPv6 use is perfectly workable but there are a lot of less mainstream elements of deployment that are not yet baked.

Be cautious about standards track progress and equipment support but do not be scared. Many people are already using IPv6 on a daily basis without trouble.

The Internet is not over. It is just beginning.

P.S. The fact that IPv6 does not have NAT is not a security hole. NAT provides obscurity, not security. A stateful firewall with no NAT will provide much more security than NAPT with no firewalling. This is all I am going to say on this.

Fear and Loathing in IPv6

2011-01-28T17:42:00.002Z

So I flew off a bit half-cocked on Twitter last night. I read this post on the importance of PI addressing from Greg Ferro.

My reaction was classic head in the sand thinking. "PI addressing for everyone? Oh the humanity! Think of the border routers!".

The main cause of this was the heading "Service Providers want to lock you in". I work for a service provider so my eyes immediately gravitated on this heading to find out what my motivation are...

It makes it harder to move to another provider because your project would need resources to readdress a lot of equipment

My reaction: This concern does not appear on my list of benefits to PA. Any such method I use to lock in my customers also restricts my ability to win new business because everyone else will do it too. Not saying other people don't think this way, but it is certainly not a globally held belief in the service provider industry.

It uses less routing table memory in their core networks and allows them to delay network infrastructure upgrades.

My Reaction: The memory used in the global table is only one concern and far from the biggest, RAM is cheap (although you wouldn't think it looking at the Cisco or Juniper price lists). I won't deny that sometimes infrastructure upgrades are delayed to save money, but this is a very short term strategy. There is plenty of competition in the SP industry, and anyone who relies on sweating assets too far isn't going to be around long. The main concern for me isn't saving CAPEX, it is that there are fundamental issues in the control plane protocols in use.

The global routing table is already set to grow at an accelerating rate over the next few years as IPv4 depletion drives de-aggregation. Doubling up this de-aggregation by mirroring (or exceeding) it in the IPv6 table concurrently will lead to growth that it may not be possible to address just by adding memory, and as the number of prefixes grow the complexity of filtering and the risk of a single prefix that can cause global reachability issues also grow.

Most of you probably already know that the global BGP routing infrastruture is starting to creak a little. Rob Shakir of Cable and Wireless recently gave a good presentation on what is required for better BGP error handling at the UKNOF18 meeting. (by the way, he is also soliciting feedback on his associated Internet Draft)

More robust and scalable protocols are needed, and taking these protocols from current early drafts (I have discussed some of the potential candidates such as LISP and ILNP on this blog before) to global deployment in production grade networks is something that will take years. This needs to happen anyway but increasing the rate of IPv6 PI assignments alters the timetable.

Enough defending my initial reaction though... After exchanges with Greg on Twitter and Skype and on more careful reflection I find that the main reason the post made me feel uncomfortable is that it is pretty much spot on. After all, if not PI then what is the alternative?

Today enterprise networks largely achieve provider independence and avoid renumbering by means of NAT. This does not exist in IPv6. This has lead to numerous calls for NAT66 as a requirement but as yet the IETF has stood firm against it.

Pretty much everyone in the IETF wants to avoid NAT66 and retain end-to-end addressing as a fundamental architectural requirement. Many protocols rely on this principle and embed addressing information within their protocol messages. Technically there is a layer violation here but the overloaded semantics of the IP address make such violations difficult to avoid. In order to allow such protocols to function across NAT the device performing the translation needs to understand the protocol and be able to modify any embedded addresses the same way that it modifies the outer header.

In the IETF world where the endpoints have the responsibility to maintain communications state and security, playing with packets on the wire like this is frowned upon. It is not necessarily the simple rewriting of the addresses in the outer header that is objected to - it is the requirement of network layer devices in the path needing to understand application layer protocols that they object to because it puts obstacles in the way of innovation.

In the enterprise world much of the duty of state and security tracking is delegated from the endpoint to border middleboxes such as firewalls. The user *wants* his firewall to understand all the protocols passing through it. If the firewall doesn't understand a protocol in this world it should be dropping it anyway. NAT66 looks good here.

This difference of viewpoint is at the heart of why any attempt to introduce NAT66 is strongly opposed. From the IETF groups dealing with Internet communications, which are also the groups with the responsibility for NAT in the IPv4 world, NAT is evil because it removes end-to-end addressability and hampers the development of new services. For the enterprise user it is a useful tool as part of their operational toolkit today and losing it is painful.

What if the enterprise community banded together and stood up at an IETF meeting with one voice shouting "We demand NAT"? The IETF is not the IEEE, the ITU or ISO, it is an open community working for consensus and anyone is welcome be they operators, vendors, academics, enterprise network admins, or just interested individuals. None are turned away and the only requirement for participation is that you take the time to join discussions, review documents, etc. It would be perfectly possible for a group of enterprise network engineers to band together and put their point across and maybe even get a standard approved. That wouldn't make it the right choice though for the reasons given above.

Meanwhile, the danger of PI assignments is to the routing infrastructure and while it is concerning, it is also a problem that is fairly well understood and work is already underway to find alternatives. As the IAB stated in RFC5902

Concerning routing scalability, although there is no immediate danger, routing scalability has been a longtime concern in operational communities, and an effective and deployable solution must be found. We observe that the question at hand is not about whether some parties can run NAT, but rather, whether the Internet as a whole would be willing to rely on NAT to curtail the routing scalability problem, and whether we have investigated all the potential impacts of doing so to understand its cost on the overall architecture. If effective solutions can be deployed in time to allow assigning provider-independent IPv6 addresses to all user communities, the Internet can avoid the complexity and fragility and other unforeseen problems introduced by NAT.

I have posted before on the importance of standing back and looking at the big picture. I should really have taken my own advice here and considered further before firing out tweets...

I also need to bear in mind another fact that I have mentioned before, the Internet has always "only just worked". Taking the infrastructure to the brink of failure is always what it takes to get new developments deployed. It is the reason that many people are only just thinking of IPv6 migration now: until getting IPv4 is painful why change things? It is a fundamental outcome of the "if it ain't broke, don't fix it" principle.

In conclusion I concur with Greg and with the IAB that for IPv6 deployments in enterprise networks today PI is the only workable solution. Admitting this still makes me feel uncomfortable.

The IPocalypse is coming. But is it news?

2011-01-26T23:27:00.000Z

Momentum seems to be gathering on IPv6 deployment. Which is good given that we are likely to see the exhaustion of the IANA pool of IPv4 addresses any day.

The thing is that the more mainstream news articles I see butchering the explanations (this is a particularly bad one) the more I think "Why should the user care?". The only constructive aspect of this media coverage is that it shames network admins into getting IPv6 transition plans into place. One of the things I have always loved about networking is keeping up with the technological developments in the industry. I like to think that I don't need to be shamed into action. The fact that my peers may need this makes me despair sometimes.

Frankly I see the fact that we are relying on media hysteria to drive development really reflects badly on us network administrators. As a group we often seem to forget that our end-users do not think the way we do. The vast majority of end users don't even really know what a bit is, let alone why using 128 of them is better than 32 of them. Does a sales person need to know whether they are using DHCPv6 or SLAAC for their address assignment? If you mention DS-Lite to them their first thought will be of the Nintendo console, not a transition technology.

When my 9 year old son searches Google at home using IPv6 he gets exactly the same service as when he searches using IPv4 at school. IPv6 doesn't make his searches more relevant or make completing his homework any simpler. It is the same service to the end user, and if done right should be totally transparent.

I was using telephones successfully for years before I learnt anything about the signalling methods used in the public telephony network. Telesales people continue to bring in business to their employers every day without knowing if their calls are signalled using C7, H.248 or SIP; whether the codec used is PCM or CS-ACELP. So why do we think that end users need to know the difference between IPv4 and IPv6?

My opinion is that it is our job as network admins to make sure that things work the way that our users expect. We do not need to expose anything below layer 7. It is not our job to scare the users with tales of the IPocalypse or to try and force them to learn how the sausage is made when they have no desire to know.

If you work in the Internet Service Provision market or develop networked applications and you don't have IPv6 deployed at least in a lab then you are behind the curve and need to pick up the pace. If you are an enterprise network admin and you don't understand IPv6 then hit the books now so when you need to roll out you aren't taken by surprise. Make sure all kit you buy is IPv6 ready and insist that you should pay no extra for IPv6 features. You probably still have a year or two before you need to deploy anything.

I am a firm believer in the "If it ain't broke, don't fix it" principle. IPv4 is not broken today; however any network plan that doesn't include IPv6 as a major component within 2-5 years is broken. Fix it.

I guess in summary I would like to issue a challenge: aim to deploy IPv6 without your users even needing to know. Don't wait until you are shamed into action and don't spread the FUD.

"Fair" use?

2011-01-11T08:03:00.000Z

I have been neglecting the blog for a couple of months, which I intend to rectify when I get back on top of things, but I just couldn't let this one go.

I recieved an SMS message from T-Mobile yesterday telling me that from the beginning of next month the fair-use data limit will be dropping to 500Mb. Per month. Previously owners of Android phones were on the "Internet on your phone plus" service which has a fair use of 3Gb per month.

I have nothing against fair use policies to the right definition of fair; however this policy does not seem to use the right definition of fair.

In my experience of running networks the 80:20 rule is usually a pretty good guideline. If you look at your data consumption per user and set your fair use volume at around the 80th percentile then most of your users will be unaffected and it will just be the outliers that you need to either throttle or charge extra. This seems fair to me.

Of course one property of this model is that as data demand rises the 80th percentile figure will also rise. It would be pretty unusual with this model for the fair use figure to drop, especially a drop of the magnitude that T-Mobile are making.

Mobile data demand is growing exponentially and this is largely driven (much like the rest of the Internet) by video consumption. What do T-Mobile have to say about this?

"If you want to download, stream and watch video clips, save that stuff for your home broadband." [see here]

Guess we won't be seeing always on media consumption devices such as the iPad on T-Mobile anytime soon...

So who is T-Mobile's fair use data limit being fair too? It seems to me that it is being fair to the network and to the shareholders by discriminating equally (but not fairly) against all users.

Advanced Exchange Topics

2010-11-01T10:51:00.000Z

So I was a guest on the Packet Pushers podcast again this week. The topic was Internet exchanges and Greg, myself and Stephen Wilcox covered topics such as what peering is, what exchanges are and some of the technical and non-technical benefits of joining an exchange.

We briefly touched on the subject of multilateral peering using route servers. This is an area that I find very interesting but was a bit too niche a subject to go into detail on during a general introduction to exchanges. Today I noticed that a second revision of an Internet Draft aiming to provide a specification of what Internet exchanges require in a route server was published. The timing seems too coincidental here. Seems like a subject hint to me...

A quick aside before I dive into the main body of the post. Two of the authors of the draft – Elisa Jasinska and Nick Hilliard – both gave presentations at the 16th UK Network Operator's Forum (UKNOF16) giving technical updates on the AMS-IX and INEX exchanges. Both are interesting talks and available from the UKNOF site. If you are involved with the UK networking industry I would highly recommend the UKNOF meetings, they have a lot of very interesting content. I haven't managed to make it along to a meeting in person yet but I try to watch as much of the meeting as I can on the live stream.

So route servers.

The peering LAN at an Internet exchange is generally some sort of Ethernet broadcast domain. The exact technology doesn't really matter. This could be built using bog standard spanning tree, vendor specific ring technologies or newer technologies such as VPLS (and it won't be long until there exchanges out there built on TRILL or 802.1aq). The exchange gets an allocation of IP address space (the RIRs have specific policies for allocating to exchanges, distinct from the LIR or PI direct allocation policies). Each customer connected to the exchange will be assigned an address (most exchanges will carry IPv4 and IPv6 these days. This may be on the same LAN or the families may be segregated into different VLANs).

The traditional way to exchange routes over the exchange is for each individual to look at the members list, and to choose a number of names on the list that you exchange traffic with. If the other party is amenable to peering then you move on and set up a BGP session using the IP address, AS number, AS-SET / route set, MD5 Secret, etc. Once this session comes up you will be exchanging traffic with each other directly over the exchange fabric. Job done. Time to move on to the next peer.

This works pretty well and it gives you a good level of control over your peers. On some of the larger exchanges it can lead to you having hundreds of eBGP sessions on your peering router and keeping track of sessions going up and down can present quite a significant management overhead.

What about if the exchange themselves were to install a router? You peer once with the exchange and get the benefit of only having one or two BGP sessions. These sessions are stable and easy to manage. If you operate a restrictive / selective peering policy then this is not necessarily that useful because route filtering can become complex. Another downside of doing this is that the exchange AS ends up getting included in the AS Path of the routes (this is why DE-CIX ranks so highly in the CAIDA AS-Rank listings).

So we begin to see that standard eBGP behaviour is not necessarily the best match.

The nexthop information needs to be preserved (we only want the control plane crossing the exchange router - if we try and push the data / forwarding plane that way then things are going to get swamped very quickly).

The AS_PATH needs to be preserved. We don't want the exchange appearing here. A discussion point that is currently open with the IX route server draft is how to handle the AS4_PATH. If one peer has AS4 support but another doesn't the route server is going to intelligently handle the use of AS_TRANS. AS_PATH preservation may require modification to the peer router as well as the route server. If your BGP implementation verifies that the first AS in the path is the configured peer AS then it isn't going to like the advertisements coming from the route server.

There should be support for distribution of routes intelligently based on community tagging. If you give the users the chance to restrict distribution of their routes you will pick up people with selective routing policies. If everyone has to announce their routes to everyone else then many people will just pass on the option. This is another area where AS4 paths get interesting. the standard ASN:COMM format of communities is well established; however extension of this to support AS4:COMM communities is less well supported.

The route server needs to pass on all routes. Not all routes will be allowed to pass to all peers. If your best path to a destination has a restrictive distribution policy you will shadow routes which should be more widely distributed. This could be managed through multiple RIBs or through holding multiple active paths active in the RIB. There is quite a bit of detail on ways to do this in the draft, I recommend reading the references there rather than me covering it here.

The other really important point is the ability to scale. The standard vendor supported bgp implementations such as Cisco, Juniper, etc. do not support route server operation. Exchanges running route servers today are generally running open source solutions such as Quagga, Bird or OpenBGP. In a lot of ways this has been very successful; however there has been a lot of pain along the way. When you are running an exchange of the size of LINX or AMS-IX there are a lot of peers and a lot of routes. This should be coupled with the fact that almost every message gets multiplied (an update from one peer then needs to be passed on to every other peer that can see that route). Putting this on an open source, community developed platform and stressing areas of the code that few other people use was always going to be a bumpy ride. But this is an example of where the not-for-profit member owned model of the exchanges works well. When they hit these problems the various exchanges got together (through Euro-IX) and have funded development of these features. This gives us more stable exchanges as well as more robust open source routing software. Seems to be win-win to me.

After all that deep tech, here are a couple of videos to end on.

Euro-IX have put up a video on how the Internet and peering works:

Personally I prefer this explanation:

Why Mobile is Different

2010-10-21T23:41:00.000+01:00

Network neutrality is a topic I have discussed a number of times on this blog. Generally I don't have a problem reconciling my belief in a free and open Internet and commercial concerns that come from the fact that I earn my wage from the proceeds of selling access to said Internet. I think that the commercial concerns of an ISP or Telco are actually helped by the phenominal growth of the 'Net.

These days most of us access the Internet in one form or another using our phones and when we don't get the same level of freedom that we do on our wired connections we cry foul. Is it fair for us to do so?

In this post I will discuss some of the differences between mobile and fixed data and some of the high levels physics involved. I will focus on the types of optical networks used at the core of todays network. A similar comparison between copper and wireless is left as an exercise for the reader.

I think that the fundamental differences between fixed and mobile essentially mean that unlimited mobile data is unsustainable.

Lets start off with the similarities and differences between the way that these technologies encode data. The light and radio waves used are both forms of electromagnetic radiation; however the transmitters in an optical network component and in our mobile phones are fundamentally different.

Firstly there is directionality.

A mobile phone transmitter sends data out in all directions simultaneously. This lack of directionality all phones in an area will have overlapping signals which limits the bandwidth available to each device. In some cases the signal can bounce off of obstacles and interfere with itself leading to areas with particularly high or low levels of signal. When transmitting without direction, the amount of power at the receiver varies according to an inverse cube law. This means that a large number of towers are required to keep the power

requirements of the handset low.

Optical transmitters in fixed line devices light up fibres using lasers. The fundamental properties of a laser are that the signal generated is coherent (meaning it has a very precise and constant frequency) and is also highly directional. These properties together mean that you can pack a number of signals together within a pretty narrow frequency band. Signals also travel a lot further before they attenuate. Another benefit of fibre is that the individual fibres are very small - even with a protective sleeve only the size of a human hair - signals with the same frequency on different fibres will never interfere meaning that a huge amount of bandwidth can be concentrated into a very small space.

Lets consider the medium through which the signal propagates next.

Optical fibre is a relatively recent invention and is the result of a huge amount of scientific research. Many types of fibre exist and they can be structurally modified at the molecular level to provide an ideal medium for a specific frequency of light. Fibres are available that can limit the attenuation and dispersion of the signal. Because of this modern DWDM systems can put 160 signals with frequencies differing by 25GHz on a single fibre.

The medium for propagation of mobile data signals is the atmosphere. There isn't much you can do to modify the properties or composition of the atmosphere. People tend to get upset if you try.

Strike two for fibre. Materials science is an active research area and I am sure we have not seen the ultimate fibre optic cable. We can play with encoding types to compensate for attenuation in the atmosphere but we cannot fundamentally change things as we can on fibre.

The theoretical amount of data you can encode on a wave is related to the frequency of that wave (I am over simplifying here c.f. Nyquist - Shannon sampling theorem for more info)

Mobile phone signals are generally in the 800MHz-2.2GHz range.

Optical signals on DWDM fibre systems are generally in the 190-200THz range. The application to the discussion at hand is pretty obvious.

Mobile has distinct benefits. I am typing this while connected by wifi and have been checking email on my phone simultaneously. When it comes to data transfer speeds though there are physical limitations with RF frequency signals in the atmosphere that will always restrict supply. Basic economics tells us when demand is high and supply is limited the price goes up. If we want to keep our mobile affordable and accessible some other method of removing demand will need to be found and the obvious answer here is to limit functionality.

I don't like this conclusion but it seems unavoidable. As a great engineer once said "Ye cannae change the laws of physics cap'n!". If the growth trends we are currently seeing continue then it won't be long until we hit a ceiling. I might not like the thought of a wired Internet and wireless schminternet but I find it hard to see another alternative.

Weaving a Better Fabric

2010-10-11T23:18:00.000+01:00

So last week I had the opportunity to attend Juniper Network's J-Tech Club event. An event where engineers from Juniper customers are invited to get together and drink the Kool-Aid. Technical presentations on new and future developments from the guys from Sunnyvale.

Unfortunately some of the most interesting content was under NDA so I can't write about it here yet. The keynote presentation was a presentation on an Internet Draft document though so I consider that fair game.

Distinguished engineer Rahul Aggarwal has been heavily involved in the development of MPLS technologies such as Traffic engineering, MPLS OAM and P2MP LSPs. Recently he has been looking at a better way of doing wide area ethernet fabrics; namely a replacement for VPLS that distributes MAC addresses through BGP rather than using independent local data-plane learning on each node. This technology is outlined in https://datatracker.ietf.org/doc/draft-raggarwa-mac-vpn/

The presentation was interrupted a number of times and I think it is fair to say that many people in the room were dubious at the proposition. If there is any protocol I would trust to distribute a table of destinations in the 10^5 order of magnitude it would be BGP; however just because something might be possible doesn't make it a good idea. As I said to Rahul during the presentation - many of the methods used to manage routing in large IP networks do not apply to MAC addresses as there is no way to aggregate them. And putting ACLs in place is also a problem. You can list all MAC addresses individually or you can restrict to certain manufacturer OUIs but that is about it.

I have read through the draft. I think it could probably be made to work but am still not convinced. Here are a few thoughts I have on the draft:

First off I will include the terminology section of the draft.

MES: MPLS Edge Switch
CE: Host or router or switch

MVI: MAC VPN Instance

ESI: Ethernet segment identifier

Another term used frequently in the document is "Ethernet Tag". My understanding is that in most cases this will equate to an 802.1Q VLAN. I am not sure what else it may equate to... S-TAG perhaps?

Before addresses can be advertised into the network, they need to be detected somehow. The draft suggests a couple of methods to do the initial MAC discovery. Standard data-plane learning and modified data-plane methods (e.g. a modified version of LLDP allowing a device to advertise reachable MACs). The second is an interesting idea and could get around some of the convergence time issues. I trust BGP to find a stable topology eventually, but I don't expect it to do it quickly. Section 9.1 only seems to allow support of control plane learning if the CE is a host. This seems to be a bit limiting.

There are some interesting restrictions around route distinguishers and route targets. RDs need to be type 1 (i.e. using the loopback IP of the MES). RTs mandate the use of 2-octet ASN based communities. Not sure on the recommendation for networks with a 4-octet ASN.

The draft allows ethernet segments to be multi-homed and I have to admit that the ESI concept required to do this seems massively complicated. I probably just need to re-read things a few more times though.

The split horizon rules seem to be the same as the potentally flawed rules in RFC4761; however the announcements of MACs via BGP stop the network from stabilising on a non-optimal topology.

The final concern I have is regarding a bit of a corner case which is a bit of a hobby-horse for me. The same MAC being present in different locations for different VLANs. When the customer is in control of the network this isn't uncommon. The usual cause is the use of HSRP or VRRP. My first concern is that there are references to doing a MAC lookup if there is no Ethernet Tag assocated with the MPLS label. As long as the MAC lookup referred to is a <VLAN ID, DST MAC> tuple this shouldn't be an issue. Similarly, in section 16 on MAC moves the logic should be that if a MAC update being seen on an <ESI, Ethernet Tag> tuple where the ESI differs from the existing value known for that <MAC, VLAN> tuple then the MAC move process should be triggered.

Will this work for large datacentre networks with hundreds of thousands of MACs? I have my doubts. I do think that this is potentially a more stable solution than VPLS for a network with thousands of MACs though.

Centralising the Control Plane

2010-10-06T00:10:00.000+01:00

So I have been looking at some of the presentation files from NANOG 50 and there are some real Gems in there.

First off a simple reference paper. The first day is for tutorials but sometimes there are some great general purpose sessions. The presentation by Philip Smith from Cisco on Service Provider BGP certainly falls in this category.

None of the material is going to be new to most people involved in managing the backbone of a medium to large ISP network... but for people that haven't had that chance it is a great summary of best practices for building your BGP network (or trying to figure out why someone else did it that way). I know I have a few readers here who are interested in service provider networks. I can confirm that I have seen and used almost all of the design techniques in this document in a number of production networks.

Another thread I have seen in the presentations is to do with OpenFlow. First we have Tom Scholl from nLayer presenting on building Scalable Peering Routers. There are many cases where budget top of rack hardware is admirably suited for applications like this. The main stumbling block is the software. Tom does a good job of outlining the pitfalls and where things don't quite work. The more conspiracy-theory minded folks might even think that the vendors are deliberately crippling the software in their low end gear in order to force us to buy more expensive kit... surely not...

Of course if these theories are well grounded then we should expect the major vendors to avoid OpenFlow like the plague. The ability to put your own innovative features into the platform without waiting for the new software release? (and possibly more compelling is being able to do it without having to pay a license for the right to use).

Network vendors have traditionally used a monolithic model with all software coming from them. Juniper have been making steps recently to allow SDK access to JunOS in order to allow approved apps to run on the platform (the apple model). If OpenFlow takes off we could have fully open control plane logic and the ability to get in there and fix what is broken or build new protocols ourselves. This sounds fantastic to me. Especially considering a project recently that has heavy MPLS control plane virtualisation that just isn't available in any of the major operating systems.

Which feeds into the next NANOG presentation I wanted to mention: Google's Open Source LSR. Generic MPLS support is pretty lacking in the open source router space thus far. This project does a lot to address that. There are still a few things that are missing that mean that I can't really answer my own problem with this. It is getting close enough for me to consider pitching in myself... although I would need to find the time to get familiar with the code and also to dust off my coding skills before doing anything on those lines...

I am very interested by OpenFlow. Looking at the site they seem to be missing a virtualisation type though. They have a model with one controller per device... Then they have a model with a controller managing mulitple devices. My particular interest is in a large number of controllers managing a single device. I will post again on this after reading more, I am sure.

On the subject of "one controller - many devices", this is a technique that I am generally not too fond of; however that is because most times I have seen it suggested it has been meant to propagate old traditional telco connection oriented paradigms (e.g. MPLS-TP). I am generally a fan of the distributed model with each node making its own decisions. In my experience the larger the system the more chance of something breaking. Centralising the control plane gives you a fewer larger systems. I would rather put time into automating the management of a large number of nodes, each with a smaller impact radius, than into consolidation into large boxes each with the capability of taking down a greater proportion of the network.

There are a lot of factors in these decisions though. This is where my multi-controller desire comes in. Imagine a big chassis switch with a number customers. You run a controller per customer in a VM, each of which has control of a couple of ports (or perhaps down to logical port level) You have the hardware resilience, redundant power, etc of the chassis hardware combined with the knowledge of full control plane isolation. A bug with Cust-A's NAPT won't take down Cust-B's OSPF, in fact Cust-B doesn't even have the NAT module loaded.

Juniper already have a limited version of this on the T-series with the JCS platform; however the JunOS limit of 16 logical systems per physical RE is a pretty low number and prevents you bringing virtualisation to smaller customers.

Is anyone listening?

2010-09-28T21:18:00.000+01:00

Public domain image from Wikipedia page for "Three Wise Monkeys"

Been a while since I posted anything on Android. Have to say that generally I am loving the Froyo update that finally made its way to my T-Mobile HTC Desire a couple of weeks back. I have a couple of complaints / annoyances though...

Firstly there is the fact it took so long. HTC Sense has a couple of nice features but nothing I can see that is worth an additional 3-4 months delay. Will these vendors and the carriers ever give up on this crap? Please?

Secondly I am seeing some odd app misbehaviors that are cleared by a reboot of the phone. There doesn't appear to be excessive memory usage so it isn't a memory leak. The most annoying is Swype deciding that it isn't going to interpret any gestures. Swype is a fantastic input method, but without gestures it makes a pretty third rate keyboard.

The HTC keyboard has enabled speech recognition, which is cool; I can't quite bring myself to dictate notes while on a crowded train though (which seems to be where I spend a lot of my time).

The last, and currently most frustrating is the radio silence over the Google Listen App. I love this app. It is a good podcasting app that with a little work could be great. They were making regular updates up until the announcement of Google TV at the I/O conference. The keynote showed Listen being used to deliver video podcasts to the TV. Back in May I commented on the listen-discuss mailing list:

[...] the recent I/O keynote for Google TV showed listen being used for delivering Video podcasts to your television.

I hope that Listen doesn't fragment and this ability comes to phones too when it becomes available.

Unfortunately it seems that not only has video support not made it to the phone, there have been no further updates to the audio capabilities either. The developers haven't been seen on the list at all and the bug list seems to be growing as new phones are upgraded to Froyo.

Hopefully when Google TV launches Listen will start getting updates again. I am not holding my breath though.

Dvorak: An Interlude

2010-09-23T21:26:00.000+01:00

I'm taking a little break from my dives into protocol development for a couple of weeks.

I have decided for the sake of my hands (and to be honest another of my random whims) to make the switch to the Dvorak keyboard layout.

History lesson. QWERTY came about through a combination of engineering and marketing requirements. Ergonomics did not factor into it at all. The primary concern was avoiding mechanical jams while typing. The letters where placed to avoid limitations with a specific typewriter design not used since the 19th century. The engineering based layout was then modified to make it easy for the salesman to find the letters required to type the word 'typewriter' all on the top row.

The fact that people can learn to type at speed using QWERTY is a testament to human adaptability and has nothing to do with the layout itself.

In the early years of the 20th Century Dr. August Dvorak and Dr. William Dealy investigated the effect of QWERTY and concluded that by applying some simple principles there were a number of improvements that could be made in order to make touch typing easier to learn, less prone to errors and faster. The Dvorak layout was patented in 1936 but at the time it was seen as too expensive to switch from QWERTY and it didn't take off.

When computer use picked up in popularity the ideas of Dvorak started to gain a little traction due to the ability to switch layouts without expensive mechanical changes. The theory was the same, but some of the keys changed due to additional keys and changes in usage. Dvorak was published as an ANSI standard in 1982 giving it some recognition.

More recently another alternate layout has appeared called Colemak. Colemak has some impressive stats and has more keys in common with QWERTY making the transition easier. I was tempted by this; however the greater levels of support for Dvorak in standard OS builds won me over in the end.

So having made the decision to switch I have been typing exclusively in Dvorak while at home for the past week. I have been using a number of excellent tools including, Tuxtype; Dvorak7min (available in all good Debian/Ubuntu repositories); Klavaro; keyhero.

I am practising for a couple of hours a night and my speed is approaching 20wpm. Low compared to my QWERTY speed but gaining quickly. I am almost ready to switch during my day job.

Physically I am relying on memory and don't have Dvorak keycaps. I have a cheap keyboard with switched caps but am not actively using it. I am considering some stick on letters as well.

The keyswap option is a fun simple project. One of the benefits of a flat non-ergonomic model is that you can move the keys without having to worry about the keys coming out level. The model I used had keyed home keys, which was a pain. I had to leave F and J in place which gave me nowhere to put U and H. I just scratched off the letters on those keys. I probably could have done something more fancy but was more concerned with getting it done quick.

Being in the UK poses a couple of issues too. Punctuation and even key positions are slightly modified. I have gone for a simple variation with " and @ switched and the other usual UK oddities.

So far I am favouring moving shortcuts to the new positions. I may break down and remap hjkl to dhtn in vi though. Having J & K on one hand and H & L on the other is a little weird.

An excellent reference site I have been using is: http://dvorak.mwbrooks.com/

And finally a link to my current performance on Key Hero:

All in all I think this will be a successful experiment. It is good to mix things up from time to time.

True Saviour or Hyperbole?

2010-09-14T22:52:00.001+01:00

An interesting paper was published in Nature recently looking at the mathematical properties of the AS connection graph of the Internet and whether this could be used to save the network from collapse.

I am going to take a brief look at the paper and its conclusions, comparing what is actually in the paper to some of the exaggerated claims that have been made on other websites. There seem to be some first glance similarities to some of the desired properties of the ALT overlay mapping network. I also have some comments on whether the connectivity metrics chosen are the best and have a couple of ideas about variations I would like to see investigated.

Overall I think this is an interesting start on some important research, but I do not feel that it is deserving of a lot of the "This map will save the Internet" hype that it has been getting.

The source paper makes some fairly ambitious claims. It expands on the authors' previous works on greedy routing within hypothetical networks in hyperbolic space and applies it to the Internet. I won't try to explain the way that they map the Internet geography into hyperbolic space (I'll leave that to them). Greedy routing is
essentially "fire it off in the general direction of the destination". This sort of routing can be dangerous when applied to current production networks, if you hit a dead end and have to back up you tend to end up looping until your TTL expires. In this paper the authors go into a lot of detail of how they perform the mapping but
they don't actually go into the practicalities of building a routing architecture using these principles. For this reason the billing of the paper in various online locations as "saving the Internet from collapse" or as "foiling internet instability" are over-egging the pudding a little. Something about the paper that is very encouraging is that the expected parameters of the model are extremely close to the actual observed parameters of the public Internet. Having consistent empirical models for network connectivity is a definite move forward from traditional "rule-of-thumb" methods of interdomain traffic planning. The authors are planning further papers on the practicalities and I am looking forward to them but for the moment this is just an interesting starting gambit.

When looking at the map and thinking about how this could be applied practically my mind immediately went to LISPs ALT Alternative topology. As I explained in a previous post, the ALT is essentially an overlay network which is organised to maximise the ability to aggregate prefixes. Choosing who to put at the root of the ALT was one of the main concerns that I had with using this technology in a production environment. If the ALT hierarchy were to be linked to efforts to maintain an ongoing geometric map of the Internet then this seems to be an ideal way to choose the nodes with greatest degree of aggregation. Those nodes closest to the centre of the map (i.e. those with the greatest degree of connectivity) are likely to also be the ones with the greatest capacity to aggregate. Although this could also lead to further concreting of the importance of the traditional "Tier-One" carriers; a club that is already nearly impossible to join. It is also interesting to note that the hyperbolic map does not tend to put the national incumbents a priviledged position. Traditionally efforts to aggregate the Internet have been based on geographic mapping rather than geometric mapping. A geometric topography is likely to be a lot more practical in a decentralised network like the Internet than any geographic approach would have.

While the paper does a good job of mapping the internet based on degree of connectivity in the AS graph it does not take many useful metrics into account. In the real world choice of a path is rarely only decided based on natural AS-path length type flows. Traffic engineering is used to override default decisions based on alternate metrics such as latency, bandwidth and cost. Some of the geographic placements of nodes also seem to be off in the graph. Telianet mapping to Russia for example, or Cogent to eastern Europe. The supplementary information sheet for the paper reveals that the geographic mapping is done using a combination of Digital Element IP Mapping Technology and whois data. Some of the results tend to give me a gut feeling that the databases are not all up to date. I have other bad experiences with GeoIP type lookups being badly wrong in the past. I would be very interested to see a mapping project such as this link up with a project like the RIPE Active Measurements project in order to extend the mapping to include "quality" factors in the definition of connectedness.

To summarise: I think that the paper is a very interesting start in an area of research that could help complement other avenues such as those being investigated within the routing research group of the IRTF. This appears to be backed up by the fact that the topography shown in the hyberpolic map seems to be compatible with locator mapping overlay networks such as the ALT. I feel that in order to be deployable in real world network it needs to pay a bit more attention to other important traffic engineering factors such as network latency. I will be looking for the follow up papers on this subject with interest.

Going with the Flow

2010-09-13T21:31:00.003+01:00

When the packet header for IPv6 was decided much of it was familiar from the IPv4 header. Sure, some of the fields changed size and others changed their name but in general there wasn't too much change. One thing that was completely new was the Flow Label.

At the time of publication of RFC2460 the definition of the Flow Label and the intended use were fairly loosely defined and the field was essentially set aside for experimental purposes. The original idea in
the RFC seems to have been for identifying traffic that requires special treatment. The vision was for either stateful establishment of flows using RSVP or for definition of required characteristics by including a hop-by-hop option.

RFC3697 refines the definition of the Flow Label with stateless semantics. In RFC3697 speak a flow is a sequence of packets that the source considers to be a flow. This allows sub-flows with the same 5-tuple. to be identified as different flows. This is good for load balancing. RFC3697 tightens up some of the properties of the flow label, perhaps too much according to some.

So the field has slightly overloaded and mostly compatible semantics. Since the publication of RFC3697 even more uses for the flow label have been suggested. The land grab on these 20 bits of the header seems to be generating a bit of noise in the 6man working group at the moment so I thought I would get my thoughts together and put a post up here here.

One of the first suggestions for the flow label was that it could be used for QoS treatment. I have to admit that I'm a bit confused by this one. I tend to favour the Diffserv QoS model. I am certainly not a fan of hop-by-hop options which tend to push packets into the slow path on the router.

The general theory goes that if you signal a QoS channel (either with RSVP or by use of a hop-by-hop option on the initial packet of the flow).

As long as flow labels are sufficiently unique the SRC Address, DST Address, FL 3-tuple should be suitable for this form of traffic pattern matching. If a host were to reuse a flow label value, or a malevolent router were to rewrite other flows to this single flow label it could potentially inject other traffic into the traffic class. I am dubious as to the real world application of this idea but if an implementation becomes available I'll take a look at it with an open mind.

Using the flow label for load balancing is the most obvious and immediately aplicable use.

The IPv6 Header stack makes things flexible and extendable but it also makes forwarding in silicon harder. The layer 4 information such as SRC/DST port are hidden away with a variable offset deep in the packet. This makes traditional 5-tuple based load balancing trickier and could potentially limit scalability in the future.

If the flow label can be relied on to be a suitably unique value on a per-flow or even per-subflow level then it makes a perfect input to a load balancing algorithm.

The only down side is that the RFCs define that a packet which is not part of a defined flow should have the flow label set to zero. Constants are not going to cut it for providing entropy to a balancing algorithm. A better default would be some sort of pseudo-random value, or some hash of the 5-tuple. The requirement from RFC3697 that flow labels not be re-used for 120 seconds could be a problem here.

One suggestion is to use a similar method for FL assignement as is used for TCP ephemeral port allocation, using a combination of a counter and a hash function. (see ID draft-gont-6man-flowlabel-security)

If information is encoded into the flow label then there is a possibility that passing this information outside of an administrative boundary may lead to unwanted exposure. Or perhaps source nodes could use steganography to encode data into an apparently random field and all of our important datas could escape onto teh intertubes!

Yes. This is how security people think.

And because of this people want to be able to remap or zero flow labels at the network border.

There seems to be a belief that service providers will also want this capability. To be honest I don't see this. Service providers will want flow label distribution suitable for load balancing. As long as this is present I can't see any call for rewriting the flow label. Writing values into zero labelled fields will be required anything more that that seems like too much work.

In conclusion I am watching the development of updates to the flow label with great interest. And I have to admit that the fact that the semantics of the flow label are still not set is a little concerning. As is anything to do with the low uptake levels of IPv6 to be honest...

Unexpected Consequences

2010-08-31T22:13:00.000+01:00

On Friday, a little experiment carried out using RIPE infrastructure caused a bit of confusion in the global BGP table. This isn't the first time something like this has caused routing problems. In fact one of the most surprising things about the global routing table is how stable it tends to be given that it is essentially dependant on mutual trust between thousands of different network providers.

This particular incident provides good examples of a number of the issues in the BGP routing system.

First we have the cause. RIPE injected a prefix with an experimental attribute. This attribute was an optional transitive attribute. The rules for processing unrecognised attributes are quite clear in the RFC (latest is RFC 4271 but the text is almost unchanged from RFC 1771):

Paths with unrecognized transitive optional attributes SHOULD be accepted. If a path with an unrecognized transitive optional attribute is accepted and passed to other BGP peers, then the unrecognized transitive optional attribute of that path MUST be passed, along with the path, to other BGP peers with the Partial bit in the Attribute Flags octet set to 1.

This behavior is part of what makes BGP such a flexible and enduring protocol. This behavior is absolutely fundamental to the operation of 4 byte ASNs in today's Internet. Without this deploying 4 byte ASNs would have involved coordinated upgrades across the Internet core.

So why did this break during the experiment on Friday?

Well it turns out there was a bug in the Cisco IOS XR software. The original attribute advertised from RIPE was correctly formed; however when an XR device (such as the Cisco CSR-1 Core router) processed the prefix it mangled the attribute, and when it advertised it on to its peers the length field of the attribute was mangled, leading to the peer resetting the session with a "malformed update" notification.

Here we get to the crux of it. This is not the first time that a bug in a specific Vendor's routers (Cisco this time, Quagga has been at the root of similar issues in the past) has caused significant disruption to the global routing system. It can be seen from the RIPE report that the actual impact was fairly limited when you look at the number of prefixes affected; however some of the networks affected were actually carrying a large volume of traffic. The AMSIX Internet exchange in Amsterdam saw a traffic drop of around 80-100Gb/s at the time of the disruption.

So why did this cause disruption? Many people believe it is because BGP is overreacting. There are a number of attempts to add a soft notification to the BGP spec. In the current deployed versions of BGP the only way to deal with a malformed update is to send a NOTIFICATION message to the peer and then reset the session. In cases where the peer believes that it has good data this can result in a cycle of bringing the session up, starting to exchange routes and then generating a new NOTIFICATION and reset when the malformed route is sent again.

There are two sides to the argument. One side says that the hard reset behaviour is too harsh when the implications are so high. The other side says that if you detect garbage from your peer, do you really want to trust that the rest of what you recieve is good?

This specific case actually puts a strong argument forward for the latter stance. The malformed update had an incorrect length field. You have a TCP segment which potentially holds multiple BGP messages. You read an UPDATE with a malformed prefix. What do you do? If you discard the whole segment then you might be discarding multiple messages, and you will be out of sync with your peer. If you discard only the bad message, how do you know when the next message starts? The length field is bad so there is no reliable way to detect the next message. If the length has been mangled upwards you will skip past it. If the length is mangled downwards then potentially the payload of the unrecognised attribute could be formed to include an apparently valid message. This could be a major security vulnerability.

Personally I lean towards the hard failure with NOTIFICATION. Who knows how long the IOS XR bug could have hung around undetected if the mangled experimental attribute had been ignored rather than having caused an issue?

Another interesting angle to this is explored over on Ivan Pepelnjak's blog. The ability of any BGP speaker to attach arbitrary large transitive attributes to routes could potentially be a vector for large scale DoS attacks. A lot of providers are running their BGP routers fairly close to the wind when it comes to free memory. Inject enough of these prefixes and you could probably knock over a number of peering routers. There is no allowance in the RFC to block these attributes; however there is allowance to block the entire prefix.

I think it would be a useful extension from the vendors to allow blocking prefixes based on unknown transitive attributes. I would prefer if it could be done by size of transitive attributes rather than just whether they are unknown. Being able to say "reject any prefixes with more than 1k of unknown attributes" would be a lot less disruptive to new protocol extension deployments. Blocking the prefix is a much better action than acceptig the prefix and dropping the attribute. The former has recognisable effects (destination may not be globally reachable), the latter will cause more subtle failures and require more time to diagnose.

Of course the other thing that is sitting in the back of my mind is the whole question of securing the BGP system. I can remember reading about initial attempts at SBGP / SO-BGP over ten years ago. Yet they are still not available. The most security the BGP has is protection of the TCP sessions using the MD5 authentication option. This protects individual sessions but provides no protection at all to the content of messages advertised to peers. Requiring origin signatures could go a long way to protecting BGP. Full PKI authentication of routes will be very expensive on resources (probably requiring large optional transitive attributes to carry the signatures... taking us full circle).

One thing we can't get away from is that we are going to continue to need larger and larger volumes of RAM in our DFZ routers to keep up with both routing table growth and the addition of any new attributes. This is bound to keep the vendors happy given the ridiculous premiums they charge for routing system RAM.

To finish: the following video, while long, is an excellent overview of the development of BGP from one of the original creators. Well worth a watch.

Managing a Split Identity

2010-08-31T00:37:00.000+01:00

So last time I did a brief run through of the ILNP draft. First off, a bit of a correction. I incorrectly said that ILNP is a member of the Map/Encap family. This is not the case. With ILNP the Locator / Identifier split is clear within the packet headers, it is not hidden away by adding a layer of indirection.

The draft that I was working from in the last post was a high level overview. This time I'll go into a bit more detail and take a look at three supporting drafts. Rather than building a new mapping database from scratch, ILNP uses the DNS to store the locators and identifiers as DNS resource records. In order to support the multi-homing and mobility aspects of ILNP methods are required to inform correspondants of locator changes and to verify authenticity of communications.

I'm going to take these in order. They are all actually pretty brief drafts. Including the standard legal boilerplate and references section they barely break the 10 page barrier.

First up is the DNS draft. The document starts out by pointing out that with DNSSEC (RFC 4033) and Secure DNS Updates (RFC 3007) the tools are in place for systems to update their information in the DNS and for them to rely on the information recieved from the DNS. This may be a little optimistic about the actual deployment of the technologies, but I think a little leeway can be granted hear. DNSSEC deployment has been really picking up over the last year or so. So what is the ILNP specific requirement on the DNS? Basically the draft is asking for four new resource record types. ID, L32, L64 and LP.

Lets have a look at each of these in turn.

First the ID RR. This takes the format:



<owner-name>  IN  ID  <preference> <ID>

The distinction between a host name and an owner name seems pretty subtle, and I think in most cases the owner will simply be the end system; however in some situations this may be an intermediate router. In some cases it may be a collection of machines. The inclusion of a preference will allow you to say something along the lines of:



www  IN  ID 5 1234:1234:1234:1234

     IN  ID 5 1234:1234:1234:1235

     IN ID 10 1234:1234:fffc:9184

In this situation the LHS is a service rather than a host. This can already be done of course with multiple A records, but the addition of a preference here is quite nice.

In the example above I am assuming that the identifier in the DNS server will use an IPv6 address like textual format for the display of addresses. It will of course be an adjusted IPv6 notation because there are only 64 bits rather than 128. The draft doesn't actually have anything to say about the textual representation, it simply states that it is an unsigned 64 bit integer.

Next come L32 and L64. I will deal with these together because they are very similar. The primary difference between them of course is that one it 32 bit and the other is 64 bit. Nothing is said about textual representation again; however I assume that the L32 will be represented using dotted quad like an IPv4 address and that L64 will use colon separated 16-bit hex fields like IPv6.

Like with the ID field, these records also contain preference values. An interesting extension here is that the draft states that the preference should be compared between unlike record types, so a node that has both L32 and L64 locators could do something like the following



gandalf IN L64 10 dead:beef::1

        IN L32 20 222.173.190.239

Remote hosts capable of ILNPv6 will see the better preference and only hosts that are ILNPv4 only will use v4. Likewise if the host has known bad v6 connectivity (e.g. if it is behind a tunnel) then it would be a simple matter to reverse these preferences in which case any dual-stack remote hosts would prefer v4 connectivity.

The final record type is LP (a vinyl comeback?). The LP record is an indirection type. The RHS of the LP record is not a directly specified locator, it is a pointer to another owner name. One example use might be to set up an owner name for a network, which contains all of the locators. You can then set up individual hosts within the network with only two records, an ID and an LP, rather than having to configure each possible locator for each possible host. The LP record also has a preference, allowing you to define additional records for an individual station if you want to (at either higher or lower preference than the LP record).

The draft makes mention of using wildcard records. Wildcard records tend to make me feel a little uneasy, so I doubt I will ever configure things that way. I'll take their word for it.

The final point I found interesting about this draft was in the usage example section. It is stated that there is some evidence that not all resolvers pass on the full contents of the Additional Data section of the DNS reply. The draft advises that when people ask for one of the ILNP resource records, that any other ILNP records are sent back as additional information. This makes sense because you don't want to be doing multiple lookups when setting up a connection. The more data you can get in that first packet the less time it is going to take to get your session up and running. The recommendation is to ask for everything you will need in the initial request, and if any of the information doesn't come back to ask again for just that information.

Another thing to keep in mind is that L32 and L64 records will likely have low TTL values in order to maximise the mobility capabilities. ID and LP records are not as likely to have low TTL values. If you ask your local cache for information they may well only have ID / LP records so it is worth asking again for the L32/L64 records because they will likely have to recurse for that information.

The ICMP draft has a little more meat on the bones when it comes to understanding the operation of ILNP.

The ICMP and Nonce drafts are tightly related and it is difficult to cover one without the other. I am going to look at the ICMP draft first because it is the shorter of the two.

The ILNP ICMP draft defines a new ICMPv6 message type for Locator Update Messages. This message type is sent when an ILNP capable node detects a change in its locators. It sends a new ICMPv6 Locator Update Message containing the list of known locators so that the far end node can update its correspondent cache and recieve messages from the new locator set. The first field within the message payload is a message length. This is an eight bit field allowing for 255 locators. I would imagine that this will be sufficient for some time to come ;). Each 64-bit locator has a 16-bit preference value against it in the message. There are also some reserved fields in the message which serve to pad the messages to 32-bit boundaries and also allow for future expansion.

These messages MUST only be sent to hosts involved in ILNP sessions (and therefore known to support the message time). They also MUST include the Nonce IP destination option for authentication purposes (see below). They can also use AH if additional authentication is required. Nonce is mandatory even if AH is used.

Next up is the Nonce draft. Before I start I have a little admission to make. I have grown up in the UK just north of London. Now, I am aware of the use of the term Nonce within mathematics and security to mean a one time number; however for the first 20+ years of my life my only experience of the word nonce was the UK slang meaning. This means that whenever I see the word the first thing that comes into my mind isn't security - it is a sex offender. This also leads to the concept of a correspondent cache being a bit of a source of amusement, because every time I think of new sessions needing to be added to the nonce register I start visualising packets as scout masters and catholic priests... Anyway, that's enough of the aside. Time to tackle the nonces...

Like with the ICMP draft, the Nonces draft specifically applies to IPv6 sessions. The nonce is inserted in packets as an IP Destination Option header. The Nonce is generated per session, and is unidirectional. This means that for a given correspondent cache, if there are multiple sessions taking place there will be multiple Nonces stored. The length of Nonce to be generated and specific generation methods are not covered. There is a link to RFC 4086 and it is stated that ILNP capable nodes must(sic) be able to support 32 and 96-bit nonces.

Packets with more than one Nonce option SHOULD be discarded.

When an ILNP capable host opens a new session, the first step should be to take a look at the DNS. The required destination should be looked up in order to find any ID/L32/L64/LP records. If these records are found then a new Nonce should be generated and included in the initial session setup (SYN/SYNACK) messages.

When opening a new session, if the ILNP capable Network layer software detects an Nonce option it treats the connection as ILNP capable and will zero the locator fields of the packet before passing it to the transport layer. If the network layer is not ILNP capable then it MUST reply with an ICMP Parameter Problem message. If an ILNP capable network layer recieves a new setup with no Nonce option then it MUST treat the connection as a classic mode connection.

The Nonce option is required in the initial packets for session setup. After this the session should be present in the correspondent cache, and as long as the locators being used stay the same there is no need to include the Nonce option. If a node detects a change to the local locators then an ICMP Locator Update Message needs to be sent with the updated locators. This packet will need to have the Nonce option included for authentication of the update. The Nonce option should also be set on any other packets being sent for a short while (at least 3 RTTs) so that packets can be authorised and accepted in the case of out of order packet delivery and during the processing time required to update the correspondent cache.

If a packet is recieved from an unknown correspondent then it MUST be discarded unless it has a valid Nonce option. A valid Nonce option on a data packet will not cause updates to the correspondent cache. This can only be updated by an authenticated ICMP Locator Update Message.

I was already getting the impression last time that ILNPv4 was a bit of a red headed step child, hidden away in the attic and kept away from polite company. Reading further into the supporting drafts does nothing to get rid of this feeling. The ICMP and Nonce drafts barely even mention that IPv4 exists. The DNS draft throws us a bone by including the L32 resource record.

One interesting thing I note in the Nonce draft is that all ICMP messages related to ILNP sessions MUST be authenticated using the Nonce. This seems to be a problem to me. Not all ICMP messages are end-to-end. Sometimes a mid-point box needs to send an ICMP message back to the source (e.g. path MTU discovery, or TTL expiry).

It will be interesting to see if the ILNP drafts have potential impacts on things such as traceroute or PMTUD.

While I quite like the architecture of ILNP, the more I read about it the harder it seems to deploy...

Down, down, down the Rabbit Hole

2010-08-23T23:31:00.000+01:00

Time for the next entry in my investigation into building a more scalable Internet. Identifier Locator Network Protocol.

Similar to LISP ILNP is another member of the Map and Encapsulate family of scaling proposals. Saving the world through tunnels. Chasing the white rabbit of a stable core topology.

Enough Alice references. Lets get on with reading the draft.

ILNP is based from various ideas that were bouncing around IPng working group around fifteen years ago. While ILNP supports both IPv4 and IPv6, the implementation is a little cleaner in IPv6 so its probably better to start there.

Essentially ILNP takes the existing IPv6 header and adjusts the semantics of the addresses fields. The 128bit IPv6 address is split into two 64 bit fields. The Locator and the Identifier. Given that the RFCs recommend /64s are used for links this doesn't sound like much of a change so far. Although it is controversial given how strenuously some people fight against the idea of standard /64 link addresses... (there is yet another argument going on currently in the IETF IPv6 working group around the use of /127 prefixes for point-to-point links).

The subtle distinction is that an IPv6 address is a point-of-attachment identifier and it identifies a network interface. In ILNP the Locator is a subnet identifier, and the Identifier is a host identifier. While I like the idea of a host identifier, I think tying it to the hardware address of a single interface could cause problems. Sure it makes it easier to maintain uniqueness; however it also means that if that particular interface is removed from the node then the Identifier changes. Of course there are also the various arguments that people make about using global scope MAC addresses as an identifier could be a privacy issue, support for RFC4941 Privacy Extensions is included.

This is all pretty straight forward, and for the IPv6 case has the immediate benefit over LISP that there is no encapsulation overhead. Things are a little less straight forward when considering IPv4. Obviously there isn't room in the IPv4 header to install a pair of 64bit identifiers. For ILNPv4 the IPv4 header becomes the 32-bit source and destination locators, with an additional encapsulation header inside the IP header containing the ILNP identifiers.

So what about higher layer protocols? This is where it gets interesting. In ILNP the upper layer protocol is only supposed to bind to the identifier. Lets take a second to consider this.

Lets start with a TCPv6 connection from [2001:db8:1:1::dead:beef]:80 to [2001:db8:2:1::deaf:beef]:43126.

In traditional IPv6 the sockets at each end would be identified by the 128bit address + the 16 bit port number. In ILNP the sockets are bound to [::dead:beef]:80 and [::deaf:beef]:43126. If the second node loses its connection to the 2001:db8:2:1::/64 subnet then it can continue the session using its address on the 2001:db8:2:2::/64 subnet without having to tear down and reestablish.

This is nice.

Of course to fully take advantage of this ability updates are required to hosts as well as new ICMP messages to allow for the locator updates to take place are required. This is not something that can be done straight away; however it is something that can be done incrementally and there is no flag-day requirement.

ILNP mapping relies quite heavily on DNS (I haven't read the DNS draft yet and will save that for a future post). Personally I don't have a problem with this. I know that some people do, and there are certainly bootstrap issues to be taken into account. To those that say that we are overloading the DNS I would point out that DNS has had the concept of record classes for a long time. Adding resource records that don't correspond with the traditional IN class is not going beyond the scope of the system.

On mobility, the suggestion is that each host would maintain a correspondent cache. By including a random nonce in the cache, the (identifier, nonce) tuple maintains uniqueness even when correspondent identifiers are not unique. If a correspondent changes locator then knowledge of the nonce can be used to authorise the hand-over.

ILNP has some interesting implications for NAT. Removal of the locator from the transport layer binding basically means that it doesn't matter that much if the locator changes in transport... So this could restore end-to-end communications in the world of IPv4 NATs. Equally it also opens the gates to the concept of NAT in IPv6. Which isn't quite as nice. But then if NAT doesn't break communications then maybe it stops being evil...

All in all I quite like what I have read of ILNP so far. I have to admit that it doesn't seem to enforce route aggregation in the locator space as strictly as LISP though so it wouldn't necessarily deal with the prefix creep plaguing the DFZ. I think by making multi-homing and mobility easier it does remove a lot of the drive to fragmentation, but need to implement host changes to fully realise these benefits could defer the gain for quite a while.

Next up: DNS and ICMP and Nonces (Oh My!)

LISP from the Horses Mouth

2010-08-23T23:30:00.000+01:00

Three excellent videos of Dino Farinacci explaining LISP to Google. I love the Google TechTalks channel on Youtube. There is some fantastic content in there.

LISP Part 1: Problem Statement, Architecture and Protocol Description

LISP Part 2 - Mapping Database Infrastructure and Interworking

LISP Part 3 - Deployed Network and Use-Cases

When is Video Prioritisation Good vs Evil

2010-08-17T23:06:00.000+01:00

Given the current bruhaha over the whole Google/Verizon "classification of classes of Internet traffic" thing, I thought it might be time for me to do a post on QoS and traffic queueing.

I will focus on the queueing methods for congested access networks because that has the most relevance for the neutrality debate. QoS has its place in the core of the network but it generally only affects packet ordering in a minor way and should rarely drop anything.

So what does QoS actually mean?

QoS generally refers to a whole suite of techniques used to provide certain performance guarantees for a certain type of traffic. For some traffic, such as VoIP, the most important thing is that traffic arrives on time – a little delay is not a problem. For other applications some variation in delay can be tolerated as long as the traffic does get through eventually. Internet traffic generally falls into the "best effort" category – there are no guarantees at all on the delay bounds, or indeed whether the packet will arrive at all.

To deliver QoS requires three main components. Classification, marking and scheduling.

Classification is the act of looking at an incoming packet and deciding how to treat it. This could be as simple as looking at which interface it entered the router, it could be pre-marked traffic with an agreed marking in a header field, it could use simple 5-tuple type matching (src, dest address, protocol, port) or it could use DPI to look well into the packet payload. Once you have done this inspection a configured class of forwarding behaviour can be selected.

The position of the marking action varies. Sometimes it is done at ingress, sometimes at egress. Basically marking is a shortcut to make sure that routers further into the network can rely on your markings in the header to do their classification rather than having to do detailed packet matching again. The best place to do deep classification is as close as possible to the ingress edge. The marking could be using standard IP header fields such as the TOS byte / DSCP, or it could be the EXP bits on an MPLS network, or some other layer 2 marking scheme such as 802.1p.

These two stages are pretty straight forward and fairly low impact on the traffic itself. This is not true of the next stage.

The simplest form of scheduling is a simple FIFO queue.

One of the simplest ways to envision scheduling is to think of a leaky bucket. with a FIFO queue you have packets dropping into the bucket at random rates. Sometimes they will be bunched together, sometimes they will be spaced apart.

If the packets arrive faster than the leak in the bottom of the bucket they will start to build up. Packets leave the bucket in the order they arrive. If the bucket is empty then packets will drop out of the bottom pretty much as soon as they arrive. If they arrive in bursts and are held temporarily in the bucket then they will leave onto the wire at even spacings.

In many cases the rate that the bucket leaks will be the rate of the underlying interface. Sometimes the bucket will be manually configured to leak at a slower rate than the underlying interface. This is known as traffic shaping. It is important to remember that a traffic shaper does not affect the rate of any single packet on the wire. It only affects the spacing between packets. Any individual packet will always travel at wire speed. When a gigabit interface is shaped to 300Mb/s, the individual frames will always travel down the wire at 1Gb/s, the shaping only affects the macroscopic traffic flows by spacing the packets out to give an average rate of 300Mb/s. There are generally two methods of rate limiting traffic. Shaping is generally the most traffic friendly, there is also policing. With policing any traffic arriving too quickly will be immediately discarded or remarked. This is harsher on the traffic flows, but light on network resource usage. Often policing is done on ingress and the customer can be advised to shape their traffic to within the policer profile to avoid drops.

The next level of scheduling complexity would come with so-called "fair-queuing". Fair queuing tries to maintain a macroscopic level of fairness by giving parallel flows of traffic an equal crack at the scheduler. In the example shown above, if packets one and 3 were part of the same traffic flow, then packet 3 would be held up because it is part of a recently serviced flow. Packet 4 would sneak in front and the packets would leave the interface in the order 1, 2, 4, 3.

An alternative method of giving per-flow fairness in a congested network is with Random Early Detection (RED – originally outlined in this paper by Sally Floyd and Van Jacobson). This technique is sometimes incorrectly referred to as Random Early Discard because the most common way to deal with congestion detected using this method is to drop it. This is not mandated by the paper though. Marking the packet in some way is just as acceptable.

Basically with RED you keep an eye on the average fill level of your bucket. The fuller the bucket, the more likely a packet is to be selected for RED treatment. The original paper (and the Cisco implementation) state that the weighted average utilisation should be used. The Juniper implementation of "drop profiles" use instantaneous queue utilisation and is more sensitive to small traffic bursts.

By randomly discarding or marking packets in the aggregate you ensure fairness through the principle that a flow with more traffic will have more packets in the queue and therefore more chance of being affected. One of the major benefits of this method is that the random factor leads to less likelihood of multiple flows backing off simultaneously and synchronising, leading to an oscillating traffic pattern (for an in depth look at TCP including synchronisation effects see this ISOC paper from Geoff Huston.)

This is all well and good, but how do we extend this model to deal with the multiple classes of traffic we defined earlier?

Basically we set up a system of buckets to feed into other buckets. Each of our traffic classes gets its own bucket to hold packets from that class. When it comes to choose a packet to drop onto the wire the interface queue will select the next packet from the buckets using a priority selection mechanism. Often voice traffic will be strict high priority, meaning that if there is anything in the queue it will always be serviced first. This is because the most important metric for voice is generally the latency, so you want packets to be queued for as little time as possible.

Any other queues will be processed according to priority. If the High priority queue has a priority of 20 and the best effort queue has a priority of 10, then for every packet transmitted from the best effort queue, two will be sent from the high priority queue.

An important thing to bear in mind here is that usually the priority queues are not shaped (the voice queue might be policed – because the voice queue will always be processed first it can lead to the other queues being starved if no limits are set). If a queue is empty then the other queues on the system will still be able to use the full bandwidth of the interface.

The next step is to imagine a broadband congested access network. If you think of the queue described above, you could apply this to an IP interface with multiple customers behind it.

If we consider what would happen in this situation if video traffic was classified into the high priority queue, things do not look good. A couple of people watching HD video streams could potentially slow down the web browsing of thousands of other users on that same port. That isn't going to fly in any network.

Lets consider an alternative.

In this model each user gets a bucket which is filled according to priority. These user queues can then be served in a round-robin basis, giving each user a fair crack at the access network bandwidth.

Personally I don't see a problem with using this last form of queueing on a broadband connection. If we use the example of video traffic being mapped into the high priority queue. A well established video provider such as Youtube will be put into this queue. The performance will be consistent whether you are watching the video exclusively, or if you start doing some browsing in the background. If you go to some new video service that has not yet made it into the classification list, it will get equal priority with other traffic such as web. This service is only disadvantaged if you use it concurrently with the high priority service. If it offers something new and unique then there is no disadvantage. At worst it is on an equal footing with the rest of the best effort internet, which is exactly the situation that will be in place under strict Network Neutrality.

This is why I have no problem with prioritisation of single defined services, so long as the prioritisation is only amongst a single user's traffic. If the best effort service offers better features than the prioritised service it will still win out with the users.

There are other options here that are more distasteful though.

Putting specific services into a lower than best effort priority class would be a bad thing. If this new innovative video service were marked down to below web traffic then it would be at a large disadvantage, no matter the offered features. Marking down services in this way just would not scale though. There is no way that some traffic de-preferencing list could keep up at an Internet scale. Sure this behaviour is possible, and sure it would be a bad thing. But in the real world it also just wouldn't happen.

Today at the meeting of the London Internet Exchange (LINX) there was a debate on Network Neutrality. One of the contributors stated: In engineering we generally say "If it ain't broke, don't fix it". The question is what is broken that we need to fix? I am still not sure that I see anything broken.

Travelling in Strange Directions (or is RFC4761 broken)

2010-08-10T20:55:00.000+01:00

After a week or so of thinking "How could they make such a bonehead mistake" I have hit the RFCs and come to an interesting conclusion. I think RFC4761 (VPLS using BGP) is broken.

RFC4762 (VPLS using LDP) is pretty explicit about forwarding traffic between remote PE devices.

Since every PE is now directly connected to every other PE in the VPLS via a PW, there is no longer any need to relay packets, and we can instantiate a simpler loop-breaking rule: the "split horizon" rule, whereby a PE MUST NOT forward traffic from one PW to another in the same VPLS mesh.

It appears that RFC4761 implements a less straightforward split horizon rule.

4.2.5. "Split Horizon" Forwarding

When a PE capable of flooding (say PEx) receives a broadcast Ethernet frame, or one with an unknown destination MAC address, it must flood the frame. If the frame arrived from an attached CE, PEx must send a copy of the frame to every other attached CE, as well as to all other PEs participating in the VPLS. If, on the other hand, the frame arrived from another PE (say PEy), PEx must send a copy of the packet only to attached CEs. PEx MUST NOT send the frame to other PEs, since PEy would have already done so. This notion has been termed "split horizon" forwarding and is a consequence of the PEs being logically fully meshed for VPLS.

Split horizon forwarding rules apply to broadcast and multicast packets, as well as packets to an unknown MAC address.

Here is the root of the problem. For some reason it was decided in RFC4761 that known unicast frames could be relayed by a PE device. This shouldn't be a problem, right? After all, it doesn't allow flooding or broadcast to be relayed, so that should break any potential loops, right?

Actually, no. Consider the following network:

The VPLS PE devices are all configured with a common MAC aging time of 5 minutes. The devices connected to the PEs have an ARP timeout of 10 minutes.

The devices connected to PE1 and PE3 have both communicated with the device behind PE2 within the last 3 minutes. They have not communicated with each other for 7 minutes.

The devices connected to PE1 and PE3 coincidentally both try sending each other traffic at almost the same time. The devices still have cached ARP entries for each other so do not need to send a broadcast ARP request; however neither PE device has an association for the remote MAC address. The frames are flooded. PE1 sends a frame from 0000.5e00.0101 to 0000.5e00.0103 to both PE2 and PE3. PE3 sends a frame from 0000.5e00.0103 to 0000.5e00.0101 to both PE1 and PE2.

When the frame from PE1 arrives PE3 installs 0000.5e00.0101 in its forwarding table against the pseudowire to PE1. Likewise when the frame from PE3 arrives at PE1 it installs 0000.5e00.0103 against the pseudowire to PE3. The interesting part is when PE2 receives the two frames. It already has valid entries for these MAC addresses in its forwarding table. The RFC only says that unknown unicast frames should not be relayed. When the known unicast frames arrive they are sent back out according to the forwarding table.

PE1 shortly receives another frame from 0000.5e00.0103, this time over the pseudowire associated with PE2. It updates its forwarding table. This duplicate frame is seen by the host and discarded. At PE3 the same thing has happened. PE3 now holds an entry associating 0000.5e00.0101 facing PE2.

When PE1 responds to the packet from PE3 the packet is sent unicast to PE2, where there is still a valid entry to PE3 and it happily relays the frame to its correct destination. The reply from PE3 to PE1 also bounces off of PE2.

Frames between PE1 and PE3 now have a stable path across the network. Traffic is not taking the optimum path, but it is flowing. Unless one of the devices sends a broadcast frame the traffic between their MAC addresses will continue to be relayed via PE2. Because there is no TTL in Ethernet the only symptom to the end devices is that the round trip time has increased. If the initial flooded traffic was a ping then the customer may have noticed the duplicate arrive.

It seems that some of the frustration I have been feeling towards a vendor is not entirely justified. They have implemented the RFC as written, the behaviour is seemingly boneheaded; however it is consistent. Of course given that the vendor in question is also the employer of both authors of the RFC means that the frustration is not entirely misdirected...

Neutrality Quick Bite

2010-08-10T07:10:00.000+01:00

I haven't really had much time to look at it, but an interesting new development in Net Neutrality
Verizon Google Legislative Framework Proposal. Interesting idea. This is essentially public lobbying, right? Legislation is often written in draft by lobbyists before being amended by politicians and passed into law. The chances of the US legislature taking this proposal as is and voting it into law is zero. I want to hear whether a) legislators are interested and b) what amendments they propose before getting too interested in this.

One key point: They don't think wireless should have regulated neutrality. They do however think that it should have transparency requirements. I think this approach can work because there is more competition in the mobile space.

Which reminds me of a thought I had a couple of years ago.

A while back some Australian ISPs got slammed for claiming that Net Neutrality was an American problem. Actually I largely agree with this. Here in the UK there is a much wider selection of DSL providers. If you don't like the service of your ISP you get a migration reference, pass it to a new provider and bish, bash, bosh on the same line you are on a different SP network. This competition provides strong incentive to deliver what the customer wants rather than forcing what you want down the customer's throat.

Perhaps a better way to address the neutrality issue is to regulate to make wholesale network access a requirement, then if people don't like the retail offering of a big player, someone can start up a competing service over the same copper, this could be a local cooperative or an alternative player that wants to expand into an area without having to build out network.

There are probably plenty of reasons why this model works in the UK but not in the US. Feel free to point them out in the comments...

Alternative Topology

2010-08-09T23:29:00.001+01:00

So in the last scaling the Internet post I did a brief overview of LISP. It was pretty shallow in detail because at the heart of it LISP is just an encapsulation method like any other. The clever stuff is in the interactions with the mapping protocols.

A number of different mapping protocols are being investigated with greater and lesser degrees of complexity. In order to get systems out there and experience with the protocols, a base mapping protocol has been written called "LISP Alternative Topology (LISP+ALT)".

Quoting from the abstract:

An important design goal for LISP+ALT is to allow for the relatively easy deployment of an efficient mapping system while minimizing changes to existing hardware and software.

While LISP+ALT has its faults, it is felt that having a complete and functional mapping protocol in place is an important step in order to gain experience with using the protocols in real networks (as explained in the RRG recommendation draft)

LISP has a standardized mapping system interface, in part to allow reasonably smooth deployment of whatever new mapping system(s) experience might show are required. At least one other mapping system (LISP-TREE), which avoids ALT's problems (such as query load concentration at high-level nodes), has already been laid out and extensively simulated. Exactly what mixture of mapping system(s) is optimal is not really answerable without more extensive experience, but LISP is designed to allow evolutionary changes to other mapping system(s).

So what is LISP+ALT and how does it work?

Packets travel between endpoints. When a LISP ITR (ingress tunnel router) receives a packet it needs to map the destination endpoint identifier (EID) to a routing locator (RLOC) so that it can encapsulate the packet and forward it across the LISP network to the ETR (egress tunnel router).

ALT defines a method of building an overlay network using GRE tunnels. This overlay network is built with a strong hierarchy to ensure that EID advertisements are aggregated as much as possible. By doing this the routing table of any individual LISP+ALT router should only hold specific EID prefixes for nearby networks and a smaller number of aggregated routes. The overlay network is used only as a control plane for the forwarding of LISP map-request messages. No data plane traffic is forwarded over this network (actually the draft does define the capability to forward traffic for unknown EID-RLOC mappings rather than dropping the initial packet [known as data probes]; however this behaviour is experimental and the recommendation is to disable this by default.)

Example ALT Topology

Here is an example topology. The logical connections between LISP+ALT routers are made using GRE tunnels. Exchange of information related to the LISP Endpoint Identifiers (EIDs) is done using BGP.

A LISP ITR in AS960 receives a packet destined for 10.1.64.25 which is part of the network 10.1.64.0/24 originated from AS9. The ITR in AS960 forwards a Map-Request into the ALT overlay network. It follows the aggregated route to the top of the hierarchy, when it gets to AS10 the prefix gets de-aggregated and the Map-Request makes its way back down the chain to AS9. Once the packet reaches AS9 the EID to RLOC mapping can be looked up. Because the Map-Request contains the RLOC of the ITR, the Map-Reply message is sent back over the LISP network directly rather than passing over the ALT overlay.

When the first packet received by the ITR there was no mapping available, the default action in this situation is to drop the packet. When no response is received the host will try again, and this time the Map-Reply should have been received and added to the local cache. The ITR is now able to encapsulate the packet and forward it to the correct ETR across the LISP network.

In a network with frequent communication between a relatively small number of source and destination EIDs this is a pretty good match. Some limitations though are:

Dropping the initial packet is fine for TCP (retransmitted); however not so good for UDP (e.g. lost syslog messages)
The hierarchy of the ALT means that a Map-Request could take a very long path to the egress ALT node. Delay could mean more than just an initial packet drop.
The ALT draft indicates that the Autonomous system numbers for ALT would be assigned independently from AS numbers used in the global Internet. Could cause problems as most routers only support a single AS number.

DFZ and get rid of its benefits.

My opinion? I think the LISP+ALT solution is clever and quite elegant; however I think it is too far from current network practicalities to be widely accepted. Coupling ALT with the AIS to get rid of the requirement for central aggregators could potentially be an interesting alternative. I will take a look at some alternative mapping methods for LISP but next in this series I will be taking a deeper look at ILNP.

Paved with Good Intentions

2010-08-04T23:07:00.000+01:00

I'm going to take a step back from the actual consultations for a post and have a look at the big picture. The big picture is important. In my opinion it is not stopping to look at the big picture that has got us into this mess.

There have been rumblings about net neutrality for a long time. It was always there, people generally thought it was probably a good idea but people seem to be doing the right thing anyway, so what is the point?

Then comes "The Comcast Affair". I don't think anyone would try and justify Comcast's actions in this instance; however I just want to play devils advocate for a moment and set out a series of events that could (almost) lead me to implement something similar.

As an engineering department you are approached by a product manager: "Guys, we just can't sell bandwidth to consumers at these prices. How can we bring the price down.". You have a bit of a look at the network utilisation, the number of current users, do a bit of number crunching... "Well looking at today's traffic patterns our userbase are using significantly less than we have provisioned. Assuming traffic patterns remain constant I think we could survive with a contention rate of 20-30:1 without negatively impacting the customer's performance. This would bring our costs down by a significant margin but we would need to keep a very careful eye on the traffic mix to make sure this doesn't come back to bite us.". The product manager turned off when he heard "significant margin".

Move on 6-9 months. The lines have been bumbling along with similar traffic patterns. You are still generating the reports but no-one outside of engineering is looking at them. After a while you don't even have time to look at them yourself. Or maybe the guy who wrote the script moves on and no-one notices when his cron job generating the report stops.

Move on another 6-9 months. The pipes in the broadband network are starting to run a bit hot. When you ask for upgrades you get a surprised look "We paid for an upgrade 3 months ago. Why are you back so soon? We don't have the customer revenue to justify an upgrade right now." Maybe it is time to revisit those reports.

When you look at the report you notice two things. Firstly you notice that the average usage per customer has gone up significantly. Next you notice that if you ignore the top 5% of customers the average hasn't actually changed that much. What are those top 5% up to? Time for some traffic analysis.

You dig out your favourite traffic analysis tool (in a broadband access network this probably won't be Wireshark. It will probably be something based on Netflow. NfSen is a personal favourite in this area). You notice that these subscribers seem to have a massive number of TCP connections on ports 6969 and 6881-6889. These connections go all over the world. This doesn't look like a typical user traffic pattern, are these guys compromised and part of a botnet? A quick Google search shows not. BitTorrent. That's one of them thar peer to peer networks that people use to steal music and movies. Stuff like this wasting bandwidth was why we stopped carrying the binary news groups a few years back and now it is back causing problems for customers again.

Internet congestion management has traditionally held one concept fundamental. The best way to be fair to traffic is to give each flow equal precedence. If you give each flow an equal crack at the bandwidth then any drops are random and no-one can accuse you of discrimination. These peer to peer users are gaming that. In a congested network the subscriber with 10 connections will get twice the bandwidth allocation of a customer with only 5 connection. To paraphrase Orwell: "All subscribers are equal, but some are more equal than others."

So we have a small number of subscribers grabbing a disproportionate share of the limited bandwidth. How do we spoil their fun? What about a simple port block?

And so the game is on. Each time you do something to block the traffic the user does something to avoid the limits. This is what I like to think of a "whack-a-mole network management". It rapidly escalates to the point where you have installed DPI and start spoofing RST packets when certain protocols are detected.

The problem is that deciding to stand against the bandwidth leeches was a blinkered choice. By focusing too closely on the problem we ended up moving away from the reasonable path one step at a time, and by the end of it we were way out in the wilderness with no hope of justifying our position.

When I found myself in the position of tackling access network congestion a few years back I approached it from a different angle. I tried to keep an eye on the big picture and rather than questioning the intentions of the users, I questioned the underlying assumption that per flow fairness is the right approach. When it comes to the access network I still question this assumption. If you have a congested access network and you can't upgrade it, then you need to ensure that access fairness is on a per subscriber basis rather than a per flow basis. This allows your customer's to do whatever they feel like. They lose the ability to game the system. Maybe it isn't a case of "everybody wins", but at least it doesn't end with you looking like the loser.

Almost all of the cases where people have claimed a "violation of net neutrality" could be explained by wrong-headed technical decisions made by overworked engineers. One of the biggest problems with network neutrality is that most engineers don't get enough time to look at the big picture. People are paid to keep networks running by whatever means necessary, and people are rarely willing to stand up for a principle when doing so will mean threatening revenue, that way leads to a P45. One case often cited of business driven blocking is VoIP services over 3G networks. Personally I am not convinced. Knowing a little about how mobile networks are built, an also having personally experienced GPRS networks with round trip times counted in seconds rather than milliseconds, I tend to think that the reason mobile networks don't allow VoIP is because they know it won't work and that it will generate a deluge of support calls. I do think that if this is the case they should just come out and say it though.

Personally I think the people best placed to ensure network neutrality are those of us that design and operate networks. I also believe that unless we are careful we are also the people most likely to break things. I believe that we need to take the time to step back and look at the big picture from time to time. I believe we also need to remember the KISS principle. A clever configuration might be good for your job security but it probably isn't good for the sanity of your ops team.

What we need to do is to foster an environment where giving people time to looking at the big picture can help the bottom line. If only we had an independent organisation focused on the good of the Internet that could maybe give us some standard naming conventions for Internet access types so that we could better inform consumers of what they are buying. Maybe we could call it the Internet Society. If only these things existed so that instead of governments focusing on legislation to use as a stick to keep evil ISPs in line, they could instead help develop the carrots that could encourage people to do TheRightThing™.

If government want to help then the best thing they can do is sponsor independent research and provide funding or incentives for community outreach. A combination of competition, clearly defined terminology and transparent metrics will create a market where there are business drivers to do things right.

Here endeth the sermon (for now anyway...)

Maintaining a Balance

2010-08-04T00:17:00.002+01:00

Only five weeks to go until the deadline for submissions on the Ofcom Net Neutrality Consultation. I need to pull my finger out and crack through this...

The problem is that whenever I start I seem to get steamed up about it. The wording in the documents is always leading.

Take the following:

Even in the longer term, as next generation networks are deployed, there may continue to be congestion problems particularly in wireless networks.

Why is there congestion in access networks? Is it because the technology can't keep up? No. It is possible to put terabits of information over a single fibre. 10Gb/s routed interfaces are mainstream. 100Gb/s interfaces are starting to hit production. This limiting factor here comes back to one of the networking truths: "Good, Fast, Cheap. Pick any two." People want fast, people want cheap. People are often willing to compromise on quality or in order to reduce the price. Congestion in the access network exists because people are willing to buy a contended service if the price is right. People buy £6.99 per month for a DSL line because they don't have the £££ available for an uncontended TDM or Ethernet access circuit. In business, where reliability is often a more important factor than price, it is not hard to convince someone that a leased line is generally the right choice.

I think the emphasis needs to shift when discussing this subject. By casually dropping phrases like "congestion problems" we are implying that access networks are badly designed. Congestion doesn't occur because of bad design, it occurs because the design parameters have to allow a certain amount of congestion in order to meet their economic limitations. The ways to address this are: raise prices or figure out how to handle congestion in a fair and transparent manner.

In response, network operators and internet service providers (ISPs) are making greater use of traffic management techniques. These can allow them to handle traffic more efficiently, to prioritise traffic by type, to charge for guaranteed bandwidth or to block or degrade the quality of certain content. Whilst traffic management potentially offers some benefits to consumers there are also concerns that firms could use traffic management anti-competitively. The increasing use of traffic management also raises questions about consumers’ awareness and understanding of the impact that traffic management has on their broadband service.

It isn't just ISPs that are implementing these types of measures. It is actually pretty common for this sort of thing to be deployed within enterprise networks too. Stopping staff clogging up the tubes by watching dancing kitteh videos is essential to many companies. Blocking access to sites such as Twitter and Facebook from work PCs is seen by many business IT managers to be an essential tool to avoid lost productivity. Does this count as anti-competitive? Where do you draw the line between this corporate traffic management and ISP traffic management?

What about when these private enterprise links are part of a service provider VPN? What about when those links carry a mix of Internet traffic and intranet traffic? Should this network still be subject to neutrality requirements? If not, then why not? If so, then how do enterprise network administrators ensure the availability of their mission critical applications?

This is a very complicated area. Going back to the networking truths: One size never fits all, and It is more complicated than you think. I am not sure of my opinions on this and networking is my life. Are we supposed to believe that some politician, with only a rudimentary grasp of how to fill in an online expenses form, will be able to craft a piece of legislation that will promote neutrality while not stifling innovation? The whole concept seems laughable to me. Legislation is a very blunt instrument. While I would happily take a hammer to many of the misconfigured firewalls out there, I can't feel comfortable with the idea of politicians or quangos poking holes through ACLs without any understanding of the subtle arts of traffic engineering or capacity planning.

The more I think of it the more I see it this way:

Traffic management is a layer 8 problem, primarily driven by politics and religion. If we build legislation to shape and limit traffic management we are essentially creating a layer 8 control plane, and in effect justifying the existence of the practises by default.

Maybe this new layer 8 control protocol needs to be RFC3514 compliant. Traffic management measures with the evil bit clear SHOULD be implemented. Traffic management measures with the evil bit set MAY be implemented but MUST generate a warning message of the format:

%WARNING: Network Karma Reduced - Consider alternative employment

Moving on in the consultation document we come to:

Proponents of ‘net neutrality’ argue that traffic management by network operators and ISPs could lead to discrimination, in turn harming what they see as essential features of today’s internet . The debate ranges widely including questions such as whether citizens have a ‘fundamental right’ to a neutral internet, or whether ‘net neutrality’ promotes economic competitiveness and growth. These are important questions, but also ones primarily for governments and legislators.

I will grant that this could lead to discrimination, is that really the basis for legislation? I would have thought that a minimum requirement would be is likely to lead to discrimination. I come across issues that restrict the end-to-end principle on a daily basis. Protocols that are blocked or forwarded at a low priority. The thing is that the problems I see are very rarely caused by some malicious traffic management policy designed to discriminate. The most common problems I see are due to mis-configured firewalls that block valid traffic because the administrator that configured them had heard that ICMP was bad. I see problems with NAT implementations that can only handle TCP and UDP. Other protocols silently fail. I see problems due to out of date software on ALGs that interpret newer protocol versions as invalid and terminate sessions. I see DNS servers that return A records for holding pages when they should return NXDOMAIN.

Does a right to neutral internet access mean that these problems will all be swept away? Of course it doesn't.

On the other side of the argument, I see a lot of traffic that I would prefer not to. As soon as you connect a machine to the Internet you will start receiving connections from all over the world. Attempts on port 1433, on port 445, on port 80, on port 25. Packets being originated from machines infected with viruses and trojan horses. Traffic with malicious intent. Should this traffic be given the same right to neutrality? A service provider that blocks such traffic would be providing a service that is safer for their customers, generate fewer support calls and fewer network abuse incidents. This seems like quite a nice niche business model, but it is incompatible with true neutrality.

The final parts of the opening part of the Ofcom document give a statement of requirements:

Against the background of this wider debate, traffic management raises two key questions for Ofcom, in relation to our duty to promote the interests of citizens and consumers in carrying out our functions. These are:

i) What stance should Ofcom take on any potential discrimination?

ii) What is the best way to deliver consumer transparency?

After all the information given to outline the problem, the actual goals seem to be surprisingly narrow. What stance should they take, and how to deliver transparency. Given the complexity of issues surrounding traffic management and the target audience of the transparency requirements (consumers), the second question will be by far the harder to answer.

The Ofcom document is actually significantly longer than the EC document, and I have only covered the first 5 pages so far. More to come later.

Living on the Edge

2010-08-02T23:32:00.000+01:00

Time for the next installment of my scaling the internet series. It is time to move on to some specifics.

The first proposal to the RRG that I will be taking a look at is LISP. As mentioned in my last post, LISP is one of the proposals to investigate the possibility of splitting the Locator and Identifier semantics of the IP address (in fact LISP even stands for Locator Identifier Separation Protocol). While LISP isn't the officially recommended technology, it is the one that has major vendor support from Cisco, where the proposal came from. The protocol is supported in IOS 15.0 in regular routing platforms, and has also been implemented by Facebook internally.

So what's it all about?

In LISP the Internet is segmented. At the edge (although the definition of edge can be fairly fluid) everyone uses IP like usual. In this region an IP address is still an interface identifier. To all intents and purposes devices in this edge region do not see anything different. This is a good thing because it was decided long ago that the Internet should not have any more flag days, all further deployments should be incremental and backwards compatible. As use of the protocol grows the location of the LISP edge can move further out, starting in the ISP core, moving to the customer edge and potentially even to the individual host.

When a packet is forwarded through a path that crosses a LISP network, the first LISP router (the ITR, or Ingress Tunnel Router) will encapsulate the packet. The destination identifier will be used by the ITR to select an associated ETR (Egress Tunnel Router). Routing to the ETR is via an aggregated RLOC (Routing LOCator). RLOCs are largely analagous to routing prefixes but because they are never used as endpoint identifiers many of the drivers for routing table fragmentation, such as multi-homing and mobility, do not apply in this space. This reduced fragmentation pressure means that a DFZ formed of RLOC entries will have fewer entries. The ITR encapsulates traffic to the relevant ETR. Core network devices will just see the encapsulated packet. The encapsulated packet looks just like a standard IP packet, so the core of the network can use standard IP forwarding without change. Only the ITR and ETR need to be LISP aware.

In order to perform the mapping from an EID to the destination RLOC a mapping system is required. A number of mapping systems have been suggested using different methods. These methods each have their advantages and disadvantages, and to some extent can be considered separately from LISP and applied to other map and encap style locator / identifier separation protocols.

The map and encap approach has advantages and disadvantages. Some of the advantages have been mentioned, the end systems and core routers can remain unchanged for example. The DFZ routing table can be made more stable by reducing the drivers for fragmentation and churn.

In the disadvantages column we have issues such as the additional overhead of an extra set of headers. In the network core the MTU is generally pretty high, so this is not necessarily an issue. The mapping method chosen can also contribute to the disadvantages. The ETR RLOC chosen may not be the closest to the destination EID, leading to a longer path through the network than would have been taken with hop-by-hop routing. This is sometimes known as network stretch.

There are also issues with mapping related to timing. Does the ITR need to maintain a full set of EID/RLOC mappings at all times? If so then we are essentially placing the same requirements on ITRs as we are currently seeing in the DFZ. If it isn't acceptable or scalable in the current core, can we really maintain it in the ITR in future? Possibly if the ITR is specially chosen custom hardware, but in many networks it is likely that ITR and core functions will often sit on the same box. If we decide that the ITR does not have to store the information at all times then it needs to have some form of cache and a lookup method.

The question then is what do you do if you don't yet have the correct RLOC for a packet? Do you hold it and wait for a mapping response, or do you drop it and wait for the initiator to retransmit, hopefully after your map request has been answered? The second is generally favoured because when working with large bandwidths buffer space is not cheap. A large number of packets to uncached EIDs could quickly cause a denial of service. Obviously any drop of initial packets is going to slow communications. Especially if the communications profile involves many short duration connections.

One of the design decisions in LISP has been taken for a reason that I have been meaning to blog about for a while.

Firewalls. The bane of the end-to-end principle.

In the current Internet firewalls and NATs are unavoidable. And in certain cases the assumption you have to make is that they are broken. PMTU doesn't work because there are so many routers on the internet that block all ICMP because someone thinks that ping gives too much information about their network. Technically advanced but not widely deployed protocols such as SCTP do not gain wide deployment because firewalls and NATs need to be updated to support them, and there is no reason to expect that this will be done in any reasonable time scale.

Where does this factor into LISP? LISP uses UDP for its encapsulation. One of the primary reasons is for compatibility. Choosing another protocol would make it much harder to ensure unhindered passage across today's Internet. Another reason for the choice was to make sure that LISP would be compatible with ECMP (Equal Cost Multi-Path) implementations which largely use the 5-tuple (SRC IP, DST IP, PROTO, SRC PORT, DST PORT). TCP and UDP play well with these load balancing decisions. Other protocols may not be understood and will not be spread as well across link members.

For performance reasons LISP specifies a zero UDP checksum. This is a perfectly valid decision on UDP in IPv4; however with IPv6 the UDP checksum is mandatory. There was quite a heated debate on the LISP lists about this at the time. While I can see both sides of the argument, I believe that the choice to make UDPv6 checksum mandatory was the right one. If LISP wanted no Checksum the right decision would have been to use something other than UDP at the transport layer. In theory IPv6 shouldn't need the 5-tuple for load balancing on ECMP, it should be possible using the 3-tuple (SRC IP, DST IP, FLOW ID); however current early implementations of IPv6 cannot be relied on for a (pseudo)random flow id. In theory a 3-tuple balancing implementation would balance traffic agnostic to the higher layer protocol, but in reality it would give little benefit because most flows would have a zero flow id.

Next up I will be looking at the current leader in the mapping protocol race. ALT.

Simplifying the Directions

2010-07-26T23:26:00.000+01:00

So there seems to be a consensus that it is a good idea to separate locator and identifier information. What does this actually mean though? I feel an explanation coming on.

So. Back to basics. I'll keep it simple and stick with IPv4. IPv6 has a few subtle differences but is generally the same. Every IP packet has a source and destination address. An IP address is an interface identifier, it identifies the point of attachment to the network, not the host (this distinction is important when considering mobility). IP routing is generally done on a hop by hop basis based on the destination address. When a packet arrives at a gateway the destination address is compared with the local routing table using a longest match algorithm. If the destination is not on the local network then the first couple of hops will probably be following a default route. The packet will eventually reach the core level of routers running without a default (the default free zone or DFZ), routers that hold the full Internet routing table and carrying 300-400k routes in their FIB. As the packet nears its destination and hits the destination AS the matches will get longer and it will eventually hit the egress router and arrive at the destination network interface and be passed up the stack to the host.

This is all simple enough. This is the way that the Internet has worked for decades. The problem is that the DFZ routing table is growing at a rapid rate and this rate is likely to increase over the next couple of years as addresses become scarce and prefixes are split and traded.

Lets take a step back and look at the DFZ routers. Do these routers have 300-400k interfaces? No. Are there 300-400k destination AS numbers to go along with those routes? No. If you look at the routing table prefix by prefix, and take a look at the next hop IP address, next hop autonomous system number, as path length, etc. you will find that the number of distinct paths through the network is vastly reduced.

The number of active AS numbers is far smaller than the number of active prefixes in the table. Even if we assume multiple partitions for each AS due to traffic engineering the table would still be vastly reduced. As RFC1925 says: It is always possible to add another level of indirection. The set of routes in the Internet can be split into multiple groups of destinations (call them locations). The number of locations is smaller than the number of prefixes and also more stable.

Abstracting the location semantics from the identifier semantics in the IP address seems to be a good idea but how do you do it? That is an ongoing argument. And I will looking at the leading proposals next time.

A Gathering Storm?

2010-07-20T21:33:00.000+01:00

This should be the last part of my read through of the European Commission consultation into Network Neutrality. Most of the contentious bits are behind us now.

Section 4.3 Market Structure. This section is fairly straight forward. The Internet is not a collection of tubes. It is a complex interconnected mesh of individual autonomous units with an array of contractual relationships governing the borders. No shit.

Imagine travelling from London to Prague by land. You will need to travel through a number of different countries. You will use road networks with different rules. You will pass through border checkpoints with different levels of security. Someone else leaving at the same time for the same destination may end up taking significantly more or less time to get there. In some locations you may be expected to pay to use a road. Paying may or may not get you better service. Where is the consultation on transport neutrality?

The normal snipe at peering sneaks in at the end.

The flows of traffic over and between the networks that ultimately deliver content to the end user are complex; they sometimes involve peering arrangements that do not involve monetary flows, since traffic is presumed to be balanced. While peering agreements appear to be the norm between internet backbone operators, network operators are also entering into such agreements with internet companies even though traffic flows might be asymmetrical.

A lot is made of balancing traffic flows and of symmetrical traffic. Volume of traffic is not the only metric in a peering arrangement. A peering arrangement between commercial providers is a commercial decision. For some people minimising latency is more important than volume of traffic. For others they peer as widely as they can as a marketing exercise. At the end of the day a peering arrangement will come about when two networks think that exchanging traffic directly will be less costly than sending that same traffic via a third party. Peering with asymmetric flows is nothing new. It has been going on as long as I have been in the industry. When I was first setting up peerings for the NetDirect network at LoNAP I don't think there was a single symmetric session.

Question 10: Are the commercial arrangements that currently govern the provision of access to the internet adequate, in order to ensure that the internet remains open and that infrastructure investment is maintained? If not, how should they change?

4.4 Consumers — Quality of Service

From the point of view of end users, including domestic consumers, the private sector and public administrations, the core issue is one related to transparency and quality of service.

While transparency and quality of service are certainly concerns high on the list of many people I don't think I would agree that they are the core issue. Most people, especially in the current climate, would put price far ahead of concerns about transparency.

The rest of this section focuses on the ability of consumers to switch providers and how it may be limited by minimum contract terms or lack of competition. This doesn't really seem to be related to any of the questions in this section.

Question 11: What instances could trigger intervention by national regulatory authorities in setting minimum quality of service requirements on an undertaking or undertakings providing public communications services?

Question 12: How should quality of service requirements be determined, and how could they be monitored?

Question 13: In the case where NRAs find it necessary to intervene to impose minimum quality of service requirements, what form should they take, and to what extent should there be co-operation between NRAs to arrive at a common approach?

Question 14: What should transparency for consumers consist of? Should the standards currently applied be further improved?

The final section is 4.5 — The political, cultural and social dimension

The internet has become a vital platform for the political, cultural, and social participation of European citizens. Any policy decision concerning the way in which the internet functions must be framed keeping this basic premise very firmly in mind.

This is a good control to maintain. Any regulations imposed on the Internet should be formed to maintain free speech and openness. I don't suppose someone could let the guys negotiating the ACTA treaty know about this requirement could they?

Question 15: Besides the traffic management issues discussed above, are there any other concerns affecting freedom of expression, media pluralism and cultural diversity on the internet? If so, what further measures would be needed to safeguard those values?

That is the end of the European consultation. There were a couple of points that I would consider misleading, and some sections that seemed fairly irrelevant to the topic at hand; but in general it was pretty straight forward. Next up I will be working through the Ofcom consultation document before attempting to build my responses.

Just In Time

2010-07-19T23:39:00.000+01:00

The Internet is a finely tuned machine. The most surprising thing about it is that it keeps going. For years the Internet has been repeatedly patched just before it breaks. I advise you to read this paper, giving an overview of some of the disasters narrowly avoided in the past as well as a feel for the future. Of course that paper is four years old now, so some of those concerns are already upon us.

X-day is now less than a year away. Poorly designed congestion control mechanisms have led to governments to start stepping in to enforce Net Neutrality.

As I posted last week, I am going to be looking in more depth at the investigations and recommendations of the Routing Research Group.

At IETF 77 in March, the RRG co-chairs Tony Li and Lixia Zhang presentation the conclusions reached after a number of years of discussion around the subject of building architectures for a more scalable internet. There were a large number of proposals made, approaching the problems outlined from a number of different directions. I will not be examining all of them here, I will be focusing on those recommended by the group.

There are several primary problems to be addressed. The growth in the size of the routing table is the main issue. Some of the factors pushing growth include requirements for multi-homing, traffic engineering and mobility. When discussing mobility it is important to understand the scope, this is not talking about full dynamic mobility with session handover, this is the ability to move an entire subnet from one location to another (i.e. moving between providers without having to renumber). These problems exist today in the IPv4 Internet and they will also be an issue as the IPv6 routing table grows so any solution needs to apply to both protocols.

As the available free address pool reduces in size fragmentation effects are causing the average prefix length to increase. Existing prefixes are being split into multiple smaller chunks leading to a larger routing table. The first recommendation (Aggregation with Increasing Scopes, AIS). AIS is actually a collection of methods that can be used to aggregate routing information. These include route aggregation within the FIB to reduce the size of forwarding tables within individual routers, virtual-aggregation which can reduce the size of the RIB as well as shield edge routers from churn associated with a larger number of smaller prefixes and finally the aggregation can extend to inter-as routes. These techniques are a controlled method of reducing information. This reduces the resources required; however it can also reduce the optimality of the routing decisions. When a technique makes a network path longer in this manner it is termed "network stretch" or "stretch factor". The more aggressive you are with your aggregation the longer the stretch factor. In some cases you may even reduce the routing information so far that forwarding is no longer deterministic. AIS is a useful set of tools and if used pragmatically will probably be an essential part of running a network over the next decade or so. This is not a long term solution though and should probably sit in the realms of "necessary evil" along with technologies like NAT.

To make a lasting impact on the problem we need to rethink how we address the Internet. One of the recurring themes amongst the proposals is the concept of Locator and Identifier separation. Right from the beginning the Internet protocols have overloaded the semantics of the address. An IP address is an interface identifier; however it is also used as a node identifier and a network locator. It is debatable whether the concept of node identifiers or host identifiers are within the remit of the RRG. Such a concept probably exists at the session layer of the OSI model which is a layer that isn't really defined in the Internet model. True mobility with seamless session hand-over would need additional host support and in my mind is outside of the scope of the RRG which should concern itself with the network layer.

In a way it would be nice to just go back to the drawing board and redesign our network communications from the ground up based on what we have learned over the past few decades. We could probably build a bloody good set of protocols that way. But noone would use them because they wouldn't be compatible with the existing deployed internet. One of the primary requirements of the proposals to the RRG is that they must be deployable in an incremental manner. That they mustn't break communications between existing hosts.

There are many proposals around the split between locator and identifier; however there are two front runners. The working group have recommended ILNP as the preferred medium-long term solution to the scaling issue; however the industry have put a lot of effort behind the LISP proposal. Which of these will win out is yet to be seen; however the fact that Cisco have put LISP into IOS15.0 and that Facebook have a working LISP implementation, give LISP a strong position even though it isn't the favoured solution of the group. I will cover more detail on the concept of locator/identifier split as well as some specifics of both of these proposals in future posts.

The final conclusion presented by the group is that more work is needed to address renumbering issues.

Next time. An overview of Locator / Identifier Split and a brief look at the headline capabilities of LISP and ILNP.

The image above is of course from the mighty xkcd.

Clearing the Tubes

2010-07-16T00:08:00.002+01:00

So, time for part two of my EC net neutrality consultation read through.

Section 4.2: Traffic management/discrimination

Traffic management is acceptable as long as it is transparent to the customer. I can certainly agree with that concept. The question is though is how do you visualise the complex world of statistical multiplexing and capacity management for your average consumer broadband user?

I'm having a bit of trouble parsing this paragraph:

There are also constraints on prioritisation of traffic which stem from the nature of the internet as a network of many different networks. This means that prioritisation can work only if all the interconnected networks that link the provider to the consumer of content/services agree on the methodologies and tools to implement such prioritisation. In practice, therefore, up to now prioritisation of particular types of traffic seems to be an option only at a certain level of the internet architecture, when the traffic reaches the network of the internet service provider (ISP) to which a particular content provider or website is connected.

I think they should have stopped after the first two sentences. The remainder of the paragraph seems to be contradictory. If prioritisation is only effective when done end-to-end then what benefit from prioritising at the content provider end? And why would prioritisation at the customer end be any less effective? Given that the link to the customer is likely to be the choke-point, this is actually the place that is most likely to benefit from prioritisation.

I find the whole of page 6 of the consultation document hard to read. It is mostly background material and in places tries to get quite technical. It would greatly benefit from a bit of technical editing.

e.g.

Traffic management can take different forms and have different purposes. Traffic may be managed to ensure a certain quality of service (for voice calls) or to ensure life-critical or real-time services that might otherwise be jeopardised by bandwidth-hungry applications, e.g. for access to emergency numbers.

So traffic management can be used to ensure quality of voice calls. Or to ensure quality of access to emergency numbers (generally a voice call too, no?). Nothing here is false; however it is poorly worded and confusing.

The paragraph continues:

In future, traffic may also be managed to ensure that legal obligations are met in some Member States, particularly for example with regard to illegal content. Traffic could also be managed to ensure for example that heavy users downloading bandwidth-hungry content such as films or software will not degrade the service for other users.

A get out for illegal content is fair enough. Interesting that they include films here though. Video is set to be the most common type of traffic on the Internet, if it isn't already. Allowing the bulk of traffic to be considered nuisance traffic suitable for traffic management doesn't seem particularly neutral.

The next couple of paragraphs get to the meat of it. First the explain that when a provider hosts both the service and the customer it can provide a higher level of "managed service". It then goes on to paint a picture of access providers and content providers forming a partnership to deliver managed services across network boundaries. Apparently "stakeholders" are concerned that this could undermine delivery of standard best-effort internet services. True. What this doesn't say is that sometimes that may be what the customer actually wants. If a particular service is important to their business then they may require their provider to prioritise that traffic over other traffic.

If I squint I can see the other side of this argument. Certainly a level playing field helps new services come forward. But a level playing field can lead to a boring game. I think that innovative companies will always be able to find a way to break through. I think that we should be free to innovate on the net and that legislating to halt the progress of useful technologies such as QoS is more likely to hold things back in the future.

The above concerns relate to offers by network operators of services offering enhanced quality of service. However, concerns that the openness and neutrality of the internet may be at risk have also arisen as a result of specific instances in which restrictions have been placed on end users' ability to access particular applications over their internet connection. Examples are the slowing of speeds for streaming of video content or for peer-to-peer applications (allowing the downloading of content from one user to another), as well as restrictions imposed by mobile operators on voice over internet protocol (VoIP) services offered by third parties.

This is an area I have more sympathy for. If there is congestion being caused by a customer then throttle the customer. Don't make an arbitrary decision about which protocols you as an operator feel should be less important. The important thing is which protocols are important to the customer.

In a way this is caused by some of the end-to-end principles that are being applauded here. How do you manage congestion in a fair manner? Well the end-to-end approach is to randomise which traffic you discard. The queue management process that does this is named Random Early Detection or RED. The paper is very interesting and well worth a read but be warned, it is quite heavy stuff.

RED was designed for a specific purpose, to avoid the problem known as TCP Synchronisation you need to try and ensure that you do not cause multiple TCP sessions to start backing off (and then building back up) at the same time. The best way to do this is to drop randomly. The problem here is that the system is designed to be fair to individual flows. If you look at the typical congestion case in a broadband access network you will find that the outlying system abusers, the sort of people that use more than their fair share of the bandwidth, will generally be spreading their traffic across a large number of flows.

I would argue that in order to be fair to subscribers, rather than to flows, you need to put the end-to-end principle to one side. Feel free to match multiple flows to the single subscriber, and then to give each subscriber a fair share of the bandwidth. Ignore the flows and the individual protocols.

To address the last point mentioned, VoIP on mobile networks, this isn't necessarily just a power play. Anyone that has paid attention to the performance of 3G and GPRS data networks will know that the jitter and latency can be abysmal. VoIP isn't really practical on the current generation of mobile data networks. The mobile operators are quite possibly reducing their support overhead by disallowing VoIP protocols. Troubleshooting performance problems is far more resource intensive than simply saying no.

I am probably being too generous to the mobile operators here though.

In closing, the final paragraph of this section sums up as follows:

The practices and behaviour of network operators and ISPs with regard to internet traffic are difficult to monitor and assess. Particularly as operators start to invest in Next Generation Networks and increased traffic management capabilities, clarity may be needed as to what constitutes ‘reasonable traffic management’ and what might be considered as unacceptable both by regulators and consumers. It is therefore important to undertake such monitoring/assessment, and that approaches in this area lead to comparable results across Europe.

I certainly agree that transparency of traffic management should be a priority. In areas where there is a monopoly I agree that regulation may be required to maintain acceptable practices. Where there is competition I believe that regulators should step away and let the consumers vote with their feet based on the available performance data.

Questions for later:

Question 4: To what extent is traffic management necessary from an operators' point of view? How is it carried out in practice? What technologies are used to carry out such traffic management?

Question 5: To what extent will net neutrality concerns be allayed by the provision of transparent information to end users, which distinguishes between managed services on the one hand and services offering access to the public internet on a 'best efforts' basis, on the other?

Question 6: Should the principles governing traffic management be the same for fixed and mobile networks?

Question 7: What other forms of prioritisation are taking place? Do content and application providers also try to prioritise their services? If so, how – and how does this prioritisation affect other players in the value chain?

Question 8: In the case of managed services, should the same quality of service conditions and parameters be available to all content/application/online service providers which are in the same situation? May exclusive agreements between network operators and content/application/online service providers create problems for achieving that objective?

Question 9: If the objective referred to in Question 8 is retained, are additional measures needed to achieve it? If so, should such measures have a voluntary nature (such as, for example, an industry code of conduct) or a regulatory one?

It May be Neutral, but is it Cricket?

2010-07-14T22:08:00.000+01:00

There are a couple of related consultations going on at the moment that I have been meaning to get a handle on and respond to.

Both Ofcom and the European commission have current consultations on the subject of net neutrality. This is a sticky subject. The term is used in a number of different ways by different people. It can be a very positive thing giving consumers transparency and freedom, or it can be a chain around the service provider stifling their ability to innovate. Working in the internet industry I can see both sides of this argument so I am interested in reading how the two bodies interpret the question of neutrality and think I might be able to bring some relevance to a response. And of course I also want to be open with my thought processes so I am going to use the response documents and the ideas they raise to write a few entries here while I compose my response.

Given that the Ofcom consultation is essentially a response to the launch of the European Commission consultation I am planning this as follows. First I will read through the European consultation document. Next I will read the Ofcom document. I will then work on my responses starting with the Ofcom consultation, then moving on to the European consultation.

First up: European Commission Questionnaire for the Public Consultation on the Open Internet and Net Neutrality in Europe.

So page 2. Section 2 of the document "Net Neutrality and the Open Internet". The first three paragraphs are typical stage setting marketese. Paragraph four raises the first comment that feels slightly uncomfortable:

Concerns have been raised that the openness of the internet, and therefore its benefits to society and the economy, may be undermined if network operators seek to treat traffic differently, for example on the basis of its origin, destination, the type of service or content that is being transmitted, or other criteria.

This seems pretty open to interpretation. Traffic management technologies are built into the protocols of the Internet at a very low level. Fields like the IP TOS / DSCP were developed for a reason. There are situations where an internet may become congested and in such situations there are legitimate reasons why you may want to throw away one type of traffic before another type. Even if you don't throw anything away you may want to dequeue certain traffic earlier to avoid jitter. I'll park my concerns on this issue for the moment. I'm sure I'll get a chance to return to them...

A couple of definitions to remember in the next paragraph. "Network Management" and "Traffic Management" seem to be interchangable terms for any technique used to prioritise one type of traffic over another. "Net freedoms" refers to "ability to access and distribute information or run applications and services of their choice". Interesting. They seem to mandating that users should be able to run services. That could be interesting and have implications for homing subscribers behind NAT solutions.

The final paragraph of this section outlines the objective of the consultation: "how best to preserve the open and neutral character of the internet". Am I the only one still bothered by the capitalisation here? An internet, the Internet? Most people aren't aware of the origin of the term internet and are only aware of the global public Internet. I should probably give it up. Interesting that the aim is to maintain the character of the Internet, not to ensure fairness to citizens.

Section 3 (Background) lists a number of relevant existing directives. Mostly articles of the "Universal Service Directive". Reading them makes my head hurt. I have no doubt that I will need to revisit them through the rest of the document and will avoid quoting them here.

On to the meat of the consultation. Section 4.

We start with 4.1 "The open internet and the end-to-end principle". I find the wording of this section a little disingenuous.

The end-to-end principle is one of the central design principles of the internet. According to this principle, whenever possible, communications protocol operations should be defined to occur at the end-points of a communications system, or as close as possible to the resource being controlled. In practice, this means that network operators treat packets equally, regardless of origin, content type or destination.

This makes it sound as if the end-to-end principle is rigidly defined somewhere in a design manual for the internet. No such document exists. The closest I can find is in RFC1958 (note that this is an Informational RFC. It is not a standard, and the abstract clearly states "This is intended for general guidance and general interest, and is in no way intended to be a formal or invariant reference model.").

Section 2.3 covers the end-to-end principle and states:

This principle has important consequences if we require applications to survive partial network failures. An end-to-end protocol design should not rely on the maintenance of state (i.e. information about the state of the end-to-end communication) inside the network. Such state should be maintained only in the endpoints, in such a way that the state can only be destroyed when the endpoint itelf breaks (known as fate-sharing). An immediate consequence of this is that datagrams are better than classical virtual circuits. The network's job is to transmit datagrams as efficiently and flexibly as possible.

This certainly advises against maintaining state in the network regarding individual network flows; however it in no way suggests that every datagram, or every protocol, should be equal. In fact it goes on to state:

To perform its services, the network maintains some state information: routes, QoS guarantees that it makes, session information where that is used in header compression, compression histories for data compression, and the like.

The end-to-end guideline as applied to internet protocol design absolutely allows the network to identify individual communications for the purposes of QoS. RFC2474 (which is a standards track document) explicitly defines methods of traffic discrimination in the Internet in a way that is compatible with the end-to-end principle:

Differentiated services enhancements to the Internet protocol are intended to enable scalable service discrimination in the Internet without the need for per-flow state and signaling at every hop.

With this in mind, if I move on to the final paragraph in this section:

However, as the volume of traffic passing over communications networks has increased, and new forms of data services have developed which change the business models underpinning the provision of services over those networks, a number of cases have emerged involving the differentiated treatment by network operators of services or traffic which have led some interested parties to question whether the principle of the openness or neutrality of the internet may be at risk.

Based on the previous RFC quotes, I would posit that the claim that there is any inherant "openness or neutrality" to the Internet is misguided.

On the other hand I would say that when providing a general purpose connection to the public Internet the only reason for traffic differentiation should be for the purposes of congestion management.

There are three questions in this section:

Question 1: Is there currently a problem of net neutrality and the openness of the internet in Europe? If so, illustrate with concrete examples. Where are the bottlenecks, if any? Is the problem such that it cannot be solved by the existing degree of competition in fixed and mobile access markets?

Question 2: How might problems arise in future? Could these emerge in other parts of the internet value chain? What would the causes be?

Question 3: Is the regulatory framework capable of dealing with the issues identified, including in relation to monitoring/assessment and subsequent enforcement?

I will leave these open for now and return to them after my response to the Ofcom consultation.

Tomorrow I will continue with section 4.2.

Scaling the Internet

2010-07-12T21:57:00.000+01:00

OK. This is something I have been meaning to start for a while now and just haven't got around to. As a network architect, part of my duty is to keep an eye to the future. I need to know where the network may be going in the next five to ten years. That is a long time in the Internet; however I have been doing this for about fifteen years now and it is actually quite scary how little things have changed at the network core. Sure the number of the prefixes in the DFZ has gone up, and the IPng project finally seems to be gaining some momentum. Tag switching seems to be more popular than I initially thought it would be after first reading about it in the pages of Network magazine. And lets not forget the first time I encountered the web via a gopher gateway. This will never catch on, too chaotic. The menus of Gopher are much better.

So now that we have established a bit of my predictive pedigree, what do I hope to achieve here. The Internet has a number of crisis points approaching.

So here is the famous exhaustion graph from Geoff Huston's IPv4 Address Report. This is the first crisis looming. Plenty people have written about it, I'm not going to focus on this issue here. If you don't know about it then you are probably too late to the party. But this is far from the only problem approaching.

When I first set up a router with the full default-free routing table I used a Cisco 4700M with 64Mb of RAM. This was easily man enough for the ~45k routes in the DFZ at the time. In the years since then the DFZ has been rapidly expanding. There are a number of reasons for this expansion. One is the exhaustion issue: as the amount of space reduces and LIRs modify their allocation policies people are being allocated smaller blocks. Another driver is PI space. As networks become more and more core to the ability of companies to do business they feel that they need to have their own provider independent space, so that they can multi-home, they can migrate between providers. In some cases it is even because people have heard that these IPv4 addresses are getting scarce, time to get some while the getting is good. The land grab is already starting and it is only going to get worse. As policies change and it becomes possible for companies to return or transfer unused address space, older large blocks will start being split into smaller blocks. There aren't that many big blocks of address space to allocate, but there are still plenty more prefixes that can be generated by splitting the space into smaller and smaller chunks. As the number of prefixes grows exponentially, so does the memory usage in DFZ routers and the processing power required to scan these tables for freshness and bring forwarding tables into sync.

So here we get to the point of the post series that I am aiming to start. A number of years ago the Internet Research Task Force (IRTF) sponsored a working group named the Routing Research Group. The remit of this group was to investigate both the reasons for routing growth and possible architectural solutions to handling the Internet routing table of the future. This group has now essentially made its recommendation.

The favoured long term recommendation made at the 77th IETF meeting was ILNP.

What I am aiming to do in this series of posts is to look at the goals of the RRG, and to look at how the proposed solution ILNP achieves these goals, and also to take a look at how the most developed alternative (LISP) could answer the same issues.

I have not yet set myself a schedule for this series of posts but I am aiming to put out at least one post a week.

Cisco Versus IEEE QinQ

2010-07-12T12:48:00.004+01:00

This post is in response to this packetlife post on 802.Q tunnelling. It is a great introduction to the subject but having been doing this stuff for years now there are a few things that I would like to add to the explanation.

I would like to start with terminology. Like most layer 2 vendors, Cisco developed their particular implementation of this technology before the IEEE standard was fully baked. The IEEE standard for this is 802.1ad-2005, which is an amendment to 802.1Q-2005. The official name for this technology is "Provider Bridges"; however vendors have used a number of different terminologies some of the more common being 802.1Q tunnelling, VLAN Stacking or Q-in-Q.

Another area where the wide range of competing pre-standard vendor specific implementations introduces complexity is with the Ethertype used. The well known Ethertype for 802.1Q is 0x8100, the well known Ethertype for 802.1ad is 0x88a8. Before this value was chosen vendors implemented a wide array of Ethertypes, including re-using 0x8100, using 0x9100, 0x9200, 0x9300 and more.

Treating the labels a stack or tunnel, and re-using the 0x8100 Ethertype introduces a painful problem. 802.1ad does not do any rewriting or encapsulation of MAC addresses. The {DST MAC, SRC MAC} tuple stays the same from end-to-end. For most cases this is not a problem (although it does increase the CAM usage in the provider core). Where it starts to become a problem is with control frames. When a switch generates a spanning tree BPDU, it sends it to a well known destination MAC address (01-80-C2-00-00-00). When you build a network using Q-in-Q you do not generally want the customer ports to take part in the backbone spanning tree. Most switches, if they see a frame come in for the all bridges group address, will punt the frame to the processor. With Q-in-Q it is desirable to forward STP BPDUs (and other L2 control frames) across the network, but it is not desirable to punt these to the CPU at every hop.

With a well-known Ethertype it is trivial to avoid unnecessary processing. You make the decision to punt to the CPU based on the {DST MAC, Ethertype} tuple. If you use the same Ethertype for the outer VLAN tag as you do for the inner (customer) VLAN tag then this won't work.

Cisco have an extension that works around this. Layer 2 protocol tunnelling. Essentially the way it works is as follows. A frame is recieved on an ingress port (dot1q-tunneling port) to a well known MAC. If Layer 2 Protocol Tunneling for this MAC is enabled (e.g. l2protocol-tunnel stp will enable MAC 01-80-C2-00-00-00) then the MAC address is rewritten to a Cisco proprietary multicast MAC address (01-00-0C-CD-CD-D0). This frame can then be forwarded across the backbone network as usual. When it gets to the egress port frames to the well known MAC have their destination MAC rewritten to the original well-known value for that protocol.

This is a good solution if you have an all Cisco network but, as with any proprietary technology, you need to be aware of potential problems if you have a multi-vendor network.

Another thing to be aware of here is whether you are generating local BPDUs. Cisco recommend turning on BPDU filtering on the edge ports. This has two effects. If you are doing layer 2 protocol tunnelling for STP then this has no effect on the ingress traffic; however if you are not doing this tunnelling then this will make sure that customer BPDUs do not eat your CPU cycles. The other effect is that it stops you from sending out your own BPDUs on the port. If you send local BPDUs as well as outputting the customer's tunnelled BPDUs you could cause some confusion.

The final issue I wanted to touch upon in this post is MTU.

I have written before (at length) about MTU terminology problems here on this blog. The system MTU setting on Cisco catalyst switches is a good example of this confusion. When using standard untagged Ethernet, or 802.1Q single tagged VLAN traffic the system MTU is the maximum size of the Ethernet client data field (i.e. the payload). When you configure 802.1Q tunnelling on a catalyst switch this will add an additional 4 bytes of header data to the frame; however this additional 4 bytes is not counted as header information for the purposes of the MTU calculation and is instead classed as payload. So in order to maintain the same behaviour for tunnelled traffic as you have for non-tunnelled traffic you need to up your system MTU by 4 bytes.

The image above is taken from Wikipedia, follow this link for the full size image as well as Author and Licensing information

Musings on Human Computer Interaction

2010-04-08T21:48:00.000+01:00

I love my G1, and have been using it heavily for the past 18 months. In the past I have loved Palm and Psion touch screen devices. I think that the touch interface is fantastic for selection of graphical interface elements and object manipulation.

But I just cannot get on with touch-screen text input.

I don't know what it is. I taught myself to touch-type back when I was at University (is it really approaching 20 years ago?) A free DOS copy of Mavis Beacon Teaches Typing on a magazine cover disc and a couple of weeks in the house alone during summer break and I was set. Since then I have played with other input methods. I used to do a lot of blogging (before blogging was a verb) using Graffiti input on my Palm Vx. I found that I was making a lot of mistakes and going back to correct them was interrupting my flow. It was especially bad when I tried to write entries while drunk, but that is probably not a good yardstick. Fortunately there were a number of good folding keyboards for Palm devices and this was perfect for writing on the move.

When I first got my G1 there was no onscreen keyboard in Android. The first keyboardless Android phone was still a few months off and the physical keyboard was the only way to go. A lot of people slammed it (the HTC chin makes it awkward, the fold-out mechanism leaves the screen crooked). Personally I found that I got used to it very quickly, and although I may not be able to type on it as quickly as I can on a full sized keyboard, I can generally keep up with a train of thought and don't find myself getting frustrated by not being able to get the ideas down fast enough. The touch keyboard is another matter entirely.

I am probably guilty of not using it enough. The ability to just flip out the G1 keyboard has spoiled me. I have used the Android on-screen keyboard for a number of SMS and IM posts. I generally steer well clear of it if I am going to write anything of length though because I just get frustrated.

I have since tried alternative methods of input too. Most recently I have been using shapewriter. Although it is generally easier to enter text using shapewriter I have found that some of its guesses are a long way off the mark. I can enter the shapes quickly enough; however I am having to scroll through the suggestions list after the majority of words, and sometimes it will take me two or three tries before the word I am going for is even in the list. I also find that the symbol input is awful.

So far I have never done any quantitative comparisons of my typing speeds using different input methods. My qualitative yard-stick is frustration. When I am entering the text, am I able to keep up with my thoughts? Do I find myself having to interrupt what I am thinking in order to go back and correct mistakes, or having to hold thoughts in order to keep up? On this test every touch screen text input method I have used has fallen down.

Recently I have started to wonder whether it isn't the speed that is the issue. Is it perhaps something more fundamental about the visual nature of entry? When I touch type I have no need to actually pay attention to where my fingers are. This is largely muscle memory which with patience could probably be reproduced on a touch screen; however with touch text input there is also the added factor that there is usually some degree of fuzziness involved. With gesture based input it is usually a case of draw, select from choices, move on. This is a lot more mentally involved that a simple keyboard. I think this is likely to be my problem. I have always found it difficult to switch modes. If I am programming for example, or doing another task involving similar amounts of concentration, then my ability to communicate verbally drops to almost zero. If you talk to me when I am concentrating you will usually get a furrowed brow while I adjust my thought processes to parse what you have said followed by a slow deliberate response while I switch modes. Once I am back into speech mode things will get back to normal; however I will have a hard time again when I try to switch back to concentration mode. This is just a quirk of how my brain works and I am largely used to it. I wonder whether this has something to do with why I can touch type, but just cannot adjust to touch-screen text input.

The keyboard has been around for centuries. I checked Wikipedia earlier and the typewriter could possibly have been around since 1714. Something that has been around for so long with only minor iterative changes has to speak somehow to something fundamental in the human psyche. I think those who try to dismiss the physical keyboard are ignoring something very important.

There have been numerous innovations to improve the speed of entry using keyboards over the years. Moving from mechanical devices needing considerable force to operate, to electronically assisted devices which reduce the travel and force required allowing for faster input. This is pretty much where the computer keyboard takes over from the typewriter.

There are other innovations since this that have been tried on the computer but never seem to have made it into the mainstream. For a long time the input device of choice for live transcription (e.g. courts, subtitling of live events) has been some form of chorded keyboard. This allows for text entry without having to move your fingers off of the keys, you simply have to press a certain number of keys simultaneously. While there have been a number of attempts at chorded input devices for personal computers the method has a steep learning curve. Anyone can sit at a standard qwerty keyboard. If they don't know how to touch type they can simply use the hunt and peck method. It may be slow, but it is an easy entry point. With a chorded keyboard you don't have this. You need to have a certain minimum skill level before you can even do basic text entry. But if you put the effort in to learn then you can reap enormous benefits.

Recent "innovations" in touch screen text input seem to want to have the cake and eat it. But as far as I am concerned the cake is a lie. At best a touch keyboard is simply a standard keyboard, but without the feedback of a physical set of keys with travel. At worst it is a distraction as you try to figure out how to do simple things like punctuation, or even have to try trail and error simply to get it to recognise the gesture you made as the word you intended. I have yet to see a touch-screen keyboard that even comes close to a physical keyboard in terms of speed accuracy or comfort.

Some people seem to think that I am nuts for even thinking of trying to write long-form blog posts or similar while on the move. Perhaps I am, but sometimes I have something I need to get off of my chest that Twitter will just not satisfy. And frequently I find myself stuck in a cramped train carriage where there is certainly not enough room to get a laptop out. To be honest there often wouldn't be enough room to get an iPad out. This is partly why I doubt I will shell out for an iPad. I will probably end up getting some form of cheap tablet for home casual browsing though.

I fear that the on-screen keyboard will become the norm. The lowest common denominator. It will be good enough for most people so there will be no drive to develop anything better for the people it doesn't suit.

I would love to have a convenient (bluetooth, single handed operation) chorded input device; however this seems unlikely to happen. At least at any reasonable price.

What I would really like in the mean time is for Google to get around to integrating support for bluetooth input devices into the stock android kernel. At the moment there is a 3rd party bluetooth keyboard driver in the market. I have heard that it works well; however it only supports older keyboards using the bluetooth SPP profile. Newer keyboards use the bluetooth HID profile and apparently (as of Eclair at least) this is not yet supported.

I have heard nothing about this being rolled into FroYo (I believe the main change in FroYo is the separation of the core apps from the OS kernel). I really hope this makes it in by Gingerbread at the very latest.

The iPad already does this, and appears to do this well. The numerous Android tablets will not be serious competitors without this ability when it comes to getting money out of my pocket. ChromeOS tablets will probably be better in this regard; however I don't like the always on assumption here. I spend too much time out of signal while on the underground. I want full featured local apps as well as the cloud.

So what's the MTU on that?

2010-02-24T09:49:00.005Z

Gotta rant on this to clear it out of my system.

The term 'MTU' is used too much.

A lot of people seem to think that MTU and Maximum Frame Size (MFS) are interchangeable. When you are in an environment where you are providing layer 2 services across a layer 3 network the distinction here is very important.

So, a quick recap.

The Ethernet standard at time of writing (IEEE802.3-2008) defines the "MAC Client Data Field" (which I will call MCDF from here on). This is the part of the frame that gets the user data and is wrapped up in the standard 18 byte Ethernet MAC header. The standard defines three compliant sizes that could be supported by devices. 1500 bytes for plain vanilla ethernet, 1504 bytes for 802.1q frames with a VLAN tag and 2000 bytes for "Envelope" frames (this is all in section 3.2.7 of the document which should be available from the IEEE).

The frame sizes that need to be supported for these three requirements are therefore the allowed MCDF size plus the 18 byte Ethernet header. i.e. 1518/1522/2018.

So the MCDF is just another name for the MTU, right? Wrong.

MTU is such a pervasive term currently, with different people using it in different ways, it is hard to track down concrete definitions. I'm going to go all the way back to RFC791. On page 6 MTU is defined as:

"The maximum sized datagram that can be transmitted through the next network is called the maximum transmission unit (MTU)."

From this definition the term MTU is used at the IP layer and refers to an IP datagram. If we take this back to the MCDF this equates to: MCDF 1500 = MTU 1500, MCDF 1504 = MTU 1500, MCDF 2000 = MTU 1500 (bear with me for this one).

The extra 4 bytes in the 802.1q case are for the VLAN tag, which is inserted at layer 2. The IP layer MTU is unchanged. In the 2000 byte case things are a little grey. The intent from the IEEE is that end-stations should never be exposed to any more than 1500 bytes for their layer 3 traffic for compatibility reasons. The extra space in the MCDF is for additional encapsulation (e.g. MAC-in-MAC, S-VLAN, arguably MPLS).

But I can get 9Kb Jumbo frames, right? Maybe. Most suppliers will allow this; however there is nothing in the standards to cover it. The extended 2000 byte MCDF does allow jumbo frames compared to the original spec but the standards state that it shouldn't be passed on to the end user.

When you are delivering Ethernet services to customers you need to be very careful with your terminology. If you offer a transparent service and do not mandate how the customer uses it (i.e. 0, 1 or 2 VLANS as the customer prefers) then be careful not to guarantee an MTU of more that 1500 to the customer. Your network probably has limits defined by the equipment manufacturer and these limits are probably defined in terms of either the MFS or MCDF (Cisco is particularly confusing in this regard. On Catalyst switches the system MTU is closer to the MCDF; however the first VLAN is free, meaning a system MTU of 1500 allows for 1522 byte VLAN tagged frames. If you use 2 VLAN tags, e.g. dot1q-tunnel, then you would need to raise the system MTU to compensate as a setting of 1500 would reduce the IP MTU to 1496).

It essentially boils down to two options:

Guarantee the customer 1500 and make sure that you can pass frames of at least 1526 bytes.
Give the customer the MFS or MCDF value and leave it up to them to calculate the MTU for their encapsulation

But in all cases be absolutely clear about your terminology. If your customer requests an MTU of 1600 and you deliver an MFS of 1600 they probably won't be happy...

Bzzt... crackle Bzzt... I think there is a loose connector here somewhere...

2010-02-18T09:00:00.004Z

So the buzz about Buzz seems to be subsiding after the initial backlash. This post isn't going to cover any privacy issues, there have already been plenty of posts on that subject.

One thing I have been using Buzz for is to probe the relationship between my apps account and the google account with the same username.

Many moons ago I had a gmail account as a side effect of signing up for iGoogle and Reader. I knew it was there but was running my own exim/courier install on a Linux box so never felt the urge to use it.

After a couple of hard disk deaths I was getting tired of maintaining my own mail (its not my day job any more and I don't really want to be spending my spare time doing a job I'd left behind). I initially forwarded to Gmail using a simple aliases file but when google launched Apps for domains (the free standard edition) I jumped at the chance to point my MX records straight at the Goog.

Initially I had two personal domains pointing to the same apps account; however this wasn't really doing what I wanted. So I signed up a second account and hosted the domains separately. Things were getting a little crazy here with 3 accounts but I just set up forwarding rules to get all mail in one inbox and all was peachy.

I then got my G1 Android phone and linked it to one of my 3 accounts (only version 2+ support multiple accounts, in version 1.0 that wasn't even an option).

The account I was using was only for apps. I soon found out that this caused a few problems with the phone, not least because paid apps in the market use google checkout, which doesn't support apps accounts.

Fair enough I thought and linked a gmail account to the apps account. This is where the fun started.

The gmail address suddenly became my primary ID. I would sign into things with my apps email and the username would show up as the gmail address. Not a big problem at the time, it just looked like a cosmetic issue, slightly annoying but no big deal.

Next came Latitude. I was jonesing for this as soon as I heard about it, and the archives already have plenty of my frustration over T-Mobile deciding not to launch it in the standard build. I didn't notice it at first, but my apps account and my gmail account (which up until now I had considered as a single entity) showed up separately when I linked up with people.

I fiddled around with it for a while and eventually came across a post in one of the google mobile support forums suggesting that the way to fix it was to create your google account with your apps email address rather than the default gmail address.

So I de-link from my apps account and sign up again. All seems to go smoothly and I now have a single username for all Google services. As an experiment I use a different password. My apps account keeps its password making it clear that while I have a single username I still have multiple accounts.

I have also found that I have two ways to get to gDocs, the standard version acknowledges my paid storage, the apps version doesn't. I have two sets of contacts. The non-apps version has been pretty pointless because there is no way to sync the two. I also seem to have two gTalk accounts with the same username but different contact lists. The non apps one only seems to work in the iGoogle side bar and is pointless because XMPP will route all messages to the apps account.

This is all slightly annoying but still just cosmetic.

Then Buzz launches. I have public Google profiles on all the accounts I have actually used so I show up straight away. To be honest everything I do on Google is public anyway (other than Latitude which is friends only) so the privacy issues didn't bother me.

Buzz isn't launched for apps thus far, so I logged into my @gmail account to have a little play. Just for giggles I go to the profile page while logged in with my apps account and make a comment. It works. I now have a semi-functional Buzz experience using my apps account. There is no Buzz folder in my apps mail so I have no way to link external streams and no easy way to post (I generally use force=1 on the mobile web app).

When I post using the Buzz feature on Maps 4 it seems to be posting from a completely separate account with a private profile and while I can see the buzz in my stream, it doesn't show up on any public buzz timelines on my profiles.

What I find interesting is that it seems that the buzz features are all on my non-apps account; however to launch buzz for apps it seems they will need to do some pretty close integration between the apps and non-apps accounts. Certainly more integration than they seem to have been able to do before.

I can see a few possibilities:

The apps and google accounts are fully integrated
When Buzz is enabled on apps and I lose the existing stuff. I also gain a new apps public profile.
Buzz for apps ends up as a Kludge that links a single non apps feed into your apps profile

A lot of people do what I initially did, and link to a gmail address for non-apps features; while other people don't use non-apps features at all and then there are people like me with a single username and a fragmented experience. I don't see how they can solve this for people in all 3 categories. The second option comes closest I guess but means I have to keep maintaining two profiles, two contact lists, etc.

Interesting times.

Edit (18/2/2010 17:17): As if by magic Google Dashboard has been updated. I can now access features such as Buzz connected sites via my dashboard.

Still no access to Buzz via my apps mail though, so no easy access to friends timelines, etc...

I have put up a quick buzz explaining how I enabled my apps account too.

Edit (19/02/2010 20:40): It seems Google were listening when this subject came up on TWiG last week and Gina Trapani got some inside scoop on the separation of Google Accounts and Apps accounts.

http://smarterware.org/5271/google-gmail-and-google-apps-accounts-explained

Musings on the iPad

2010-01-28T08:35:00.003Z

Everyone and his dog is blogging about this so I might as well join in. I am not an Apple fanboy. I look at their devices with awe. I have a great deal of respect for the company and for Steve Jobs. I just don't generally buy anything they produce.

I don't think the iPad announcement met the pre-launch hype but then I don't think anything could have. I am currently unsure what its market is but I know it isn't me. A large screen ipod touch? I'll stick with an ipod classic thanks. An ebook reader? Great for indoor use but if you are going to be out in direct sunlight you can't beat e-ink. (More on ebooks coming later in this post). Living room laptop? Give me a camera for internet video calls (with my wife and son in Australia at the moment skype from the couch is an important part of my day).

I have never been a fan of god boxes. One box that does everything generally doesn't do anything as well as a dedicated device would. For people who don't want to carry multiple devices around this might have value but then this isn't a phone, so you are still going to have to carry 2 devices around anyway...

Even so I can see that for a lot of people this is good enough and it sure is purty so I have no doubt it will sell by the bucketload. Tent hire shops near apple stores have busy time ahead. I recommend you to Stephen Fry's blog for a less jaded view of the box.

The area that gives me most concern is ebooks (or ibooks?). The announcement says they will use epub which is great. The proprietary format DRM hell is the reason I will never buy a Kindle. Epub is a nice format. Essentially it is a zip file with xhtml and images in it. You can download them from Waterstones, WH Smiths, my local library service, Google Books, Project Gutenberg and many other places. Publishers already have a lot of content in epub format and almost all of it has Adobe "ADEPT" DRM as used in their Digital Editions reader software. I don't think Apple can ignore this existing content so I expect that ADEPT protected content will be supported on the iPad. What I don't expect is for them to incorporate ADEPT into the itunes book store it is more likely to be a proprietary DRM something like their FairPlay system.

I think the ipad will be great for electronic reading in the long term. It will get the last few publishers which haven't embraced digital books on board. It will get the publishers to publish more and earlier (currently it is quite common for the ebook launch to be delayed until the paperback is available).

In the short term it will be painful. There will be exclusive deals done (want the latest harper collins publications on your e-ink device? Sorry, ipad only). Apple will be dealing direct with the publishers meaning that other booksellers will lose on sales. I am not a big fan of Waterstones but I do like the opportunity of being able to browse the dead tree catalogue in one of their stores before heading home and buying the digital version through the website.

A lot of people are saying that Apple will change the game with ebooks the same way they did with music downloads. I think the situations are different. Momentum has been building around ebook distribution for a few years already. Amazon and the Kindle have already given the ebook market the closed DRMed kick up the arse it needed to get it moving. Apple wading in now is more likely to move things backwards.

My prediction is that it will dramatically increase the selection of content at the expense of pushing the level of openness back a couple of years. Having Amazon and Apple available as the evil face of big corporation will help reduce over zealous DRM in the long term.

Free (for all) Peering

2009-11-20T17:23:00.004Z

Given that people keep bringing up the old "Peering is free Internet Access" chestnut, I thought I'd run through a quick pricing exercise.

Let's imagine that you live in London Docklands. You want some of this free Internet access you've been hearing about recently (e.g. this recent podcast: TWiG 12).

First off, you need to get into one of the many Colocation centres around London where there is a public internet exchange point. Lets choose Telehouse North.

A 100Mb/s ethernet link to Telehouse from somewhere in Docklands is likely to cost you around £2500 to install, and then £500 a month to keep running.

Now to peer at an IXP you need to talk BGP. This means you need your own IP address range, and an autonomous system number. The easiest way to get these things is to join the local internet registry for your region. For London this would be RIPE. Checking the 2010 fee schedule the sign-up fee is €2000 with an annual fee of €1300 for the extra-small registry class.

Once you have this sorted out you can sign up to the exchange.

Current LINX Membership fees for 100Mb/s ports are: £1800 annual membership fee. £160 per month for a 100Mb/s port.

The install fee for a 100Mb/s port in Telehouse is: £550

Lets add that up:

Link to Telehouse:	£2500
RIPE Membership:	£1800 (exch rate 0.9)
LINX Port:	£550
Total Install:	£4850
Link to Telehouse:	£500
Ripe Membership:	£100
LINX Membership:	£150
LINX Port:	£160
Total Monthly:	£910

So assuming a usage pattern of 60Mb/s you are looking at an initial outlay of nearly £5000 and getting your internet access at around £15 a month per Megabit/s. And you won't get all the sites you want either. You will only get access to other LINX members that want to exchange traffic with you.

Does peering still look free to you? I'd rather take a limited service cheap DSL service at home myself.

I have made a lot of assumptions in this email and the situation I have described is not realistic but I hope it has indicated that even though peering may not involve any cash changing hands between peers it does involve cost to all parties involved.

Peering (a.k.a SFI) certainly makes very good sense when you are doing a lot of internet traffic. It is not free though. The costs do come down quite a lot when there are very large volumes of traffic but they never go to zero.

Ping and OSI Annoyances

2009-11-17T08:55:00.004Z

OK. Its been a while since I had a rant here but the wording in a couple of stories grabbed from my google reader stream this morning have sparked me off again...

First off, Lifehacker posted a recommendation for a tool called Pingtest. The guys who wrote this are the same guys behind speedtest.net I am sure they know the limits of ping and have made sure this will be reliable; however I can't applaud perpetuating the myth that ping is a reliable performance monitoring tool rather than a simple connectivity checker.

Secondly. Juniper and O'Reilly have just released a new book on green networking named "The Sustainable Network". This in itself isn't bad, and I intend to check it out using my Safari account. What annoyed me about the press release is the sentence: "(The Global network)'s ... future depends on ... industry innovation at all layers of the OSI model".

The OSI 7 layer model is a conceptual model rather than a practical one. It is still a useful learning tool but network designers and protocol architects will generally move past using it for actual deployments pretty rapidly.

The core Internet protocols have been around longer than the OSI model. TCP/IP layers do not correspond with the OSI layers. Even the ISO protocol designers had a hard time implementing the OSI model in real life and often utilised "pass through" funtions to skip pass layers they couldn't shoehorn in.

Network innovation doesn't depend on the OSI model. It depends on finding the right recursive layering model and evolving the control plane so that the current exponential growth curve of the default free zone can be flattened out.

For more on these subjects see:

Day, John - Patterns in Network Architecture (Prentice Hall) (website at http://pnabook.com/)

Various - Proposals to the Routing Research Group on addressign routing table scaling

Layering in the Modern Internet

2009-08-28T07:30:00.002+01:00

OK, so I'm still reading Day's "Patterns in Network Architecture" and it's gradually bringing areas of what I do into better focus.

Open any networking book and you'll immediately be shown the classic OSI 7 layer model. As models go it does a pretty good job. It doesn't match the architecture of the Internet protocols though. Most of the books will point out that the OSI model came later. Few actually point out that the OSI model isn't flexible enough to model a scalable global Internet.

In simple networks you have services, hosts, interfaces and links. If you have two hosts communicating you have service communication encapsulated in TCP/UDP/etc. IP doesn't have a host layer (see HIP/SHIM6 for a couple of proposals) but DNS allows a mapping of a host name to an interface identifier at the host communication layer (I.e. IP address). Interface level communication is trending towards Ethernet. This leads back to the simplified view: TCP is transport/session layer; IP is network layer; MAC is data-link layer.

The Internet has a massive variety of configurations. A better (but still vastly simplified) model could be: Hosts talk using IP and could be relayed at a number of aggregation levels such as: Network (e.g. IGP area), inter-area (e.g. BGP confederation member or RR cluster), inter-entity (e.g. External ASN). Going back 10 years or so these boundaries would primarily just be defined by routing at the IP layer but now there are real moves to encapsulating protocols at these boundaries too. Within a service provider network the IGP is simply a bootstrap protocol for bringing up the MPLS infrastructure for intra-entity communication, BGP for intra-ASN MPLS (this is usually only seen in larger cores, this can also be used for inter-ASN MPLS but such uses are not generally for Internet applications). Inter-ASN communication today uses the same addressing scope as inter-host communication but there is active work to change this (See proposals under discussion in the routing research group such as LISP).

Some of these encapsulations allow for flexible mappings for services. An example would be that the ethernet layer in the first example could actually be running over the MPLS layer of the second example. For this reason I think that any inter-ASN method needs to support carrying more than just IPv4/IPv6 (the thought of configuring IPv4-over-Ethernet-over-MPLS-over-GRE-over-IPv6-over-LISP in the future leaves a bad taste in my mouth)

Recursive layering is a powerful concept but care must be taken that there is a real architectural reason for the layer. Layers should be created to solve problems, not mask them.

Save us Obi-Wan Torvalds, You're Our Only Hope

2009-08-11T21:38:00.004+01:00

OK, random thought time.

I'm currently reading the extremely interesting "Patterns in Network Architecture: A Return to Fundamentals" by John Day. In it he points out that one of the fundamental benefits that caused the internet protocol suite to win out is that it was designed by operating system people, rather than telecomms people. He also puts forward that when addresses started getting low we missed a chance to go back to these principals and design a new scalable protocol using contemporary thinking on IPC, virtual addressing and the like and instead we just threw header bits at the problem. I'm not sure I'm 100% convinced by all of the arguments in the book but it is certainly thought provoking.

Now back to the title.

We all know what happened when Linus Torvalds got the itch to improve on the Minix OS. Many of us also know what happened when he got the itch to improve on the available distributed version control tools available a few years back. I am wondering whether he is itching as much as I am over the lack of take-up of IPv6 and the number of features still missing to facilitate migration from a v4 Internet to a v6 Internet.

If he is, and if Day is right, then an itchy Linus plus some time to code could be the best solution to the closing v4 exhaustion problem.

Does 21CN go far enough?

2009-08-10T10:24:00.001+01:00

I read a post on a mailing list that got me to thinking. Recently there have been a number of live events (e.g. Wimbledon, The Ashes) which have been broadcast live online by one means or another (I personally quite like the video scorecard the BBC are streaming alongside the Ashes coverage). These events have caused a noticable rising of sustained traffic over business grade DSL during office hours (i.e. people tuning in while at work).

The perennial argument against IP Multicast is that the massive growth in IP video content is primarily on-demand where multicast doesn't give any gain; however the peaks seen during live events, as well as other research shows that there would still be significant bandwidth savings if live content could leverage multicast proplerly.

This moves on to my thoughts about 21CN. The primary DSL delivery methods in 21CN are WBC and WMBC (there is also IPSC for access to 20CN IPstream services). Largely these services mirror the IPStream service in that they are L2TP delivery of PPP services. Because these services are delivered to the ISP as a point to point L2 service this means that the ISP has to aggregate traffic within their network and streams have to be replicated before being fed in to the BT network. No cost saving to the ISP generally equals no driver to do any work on multicast services.

Of course a lot of this cost saving argument is moot if the ISP are charging their customers correctly. If you are being charged on a usage basis by your supplier, then you have to charge on a usage basis to your customer. If you charge flat rate and then get pwned when the customers actually use what they are paying for then you have no right to complain...

An alternative L2 for this could have been a VPLS or VLAN based ethernet broadcast network. In this type of deployment (similar to many cable deployments but not usually used for DSL) multicast traffic could be sent out using ethernet level multicast (being careful not to flood, otherwise this traffic would be sent to users who don't want it). This method of delivery would have a lot of positives but it is also very different from existing strategies. Much of the resistance to this idea would be from inertia (I certainly cringed when the idea popped into my head); however the more I think about it the more I like the possibilities for future expansion. The biggest scare factors would be direct customer to customer traffic at ethernet layer (could be mitigated by private vlan like split broadcast domain) and broadcast / flooded traffic.

CPE support could also be an issue.

A Fresh Cupcake

2009-05-22T13:35:00.003+01:00

A new cupcake update just appeared on my mobile. Appears to be a security update rather than feature update.

Official build strings: Firmware version 1.5, Kernel Version: "2.6.27-00393-g6607056 san@sandroid #1" Build number: CRB43

User agent: "Mozilla/5.0 (Linux; U; Android 1.5; en-gb; T-Mobile G1 Build/CRB43) AppleWebKit/528.5+ (KHTML, like Gecko) Version/3.1.2 Mobile Safari/525.20.1" "-"

Same webkit version, so not a browser vuln... minor change in kernel version...

The Emperor's New Transport

2009-05-20T23:14:00.004+01:00

All of the hype in networking at the moment seems to be surrounding the move to packet based transport with particular emphasis on Ethernet.

I have to admit to being confused.

Don't get me wrong, I'm a packet guy through and through and can totally understand the benefits of moving to a packet core. What I don't understand is the obsession on using Ethernet to do it.

One quick pedantic point I want to get off my chest: Ethernet PDUs are "Frames" not "Packets". That's just a terminology point though and just annoys me rather than being an argument for or against the applicability of Ethernet in the Packet Transport network.

Ethernet is a great LAN protocol. I have been working with it for years and have a pretty good understanding of it (there's a reason one of my unofficial job titles is "Switch Bitch"). But using the vanilla 802.1d and 803.3 standards it doesn't scale well.

Ethernet is a CSMA/CD technology (Carrier Sense Multi-Access with Collision Detection). These things are all important when considering the most basic setup with a number of devices communicating over a dumb hub/repeater.

I real modern day networks the CD part has largely been rendered irrelevant due to the prevalence of micro-segmentation and full-duplex links; when operating in full duplex you don't get collisions.

Lets imagine our next generation all-singing all-dancing packet transport network. Each node in the network is a packet switch. If you try and build this sort of network with Ethernet switches it won't work; spanning tree simply does not scale this way. Future enhancements such as TRILL (see RFC5556) will allow further scaling at the Ethernet layer but the protocol designers only see this scaling up to high 100s of nodes in the network. Plenty for a metro, but when talking of a National or International network that's not so much.

Another solution that seems well liked is MPLS-TP. I have nothing against this. I've been working with MPLS in the IP world for years and it generally does what it says on the tin. I tend to prefer an intelligently signalled MPLS network over an NMS configured MPLS-TP network but reducing complexity in the core allows for faster and cheaper boxes so it is forgivable.

If the node is an MPLS LSR then forwarding decisions are made based on MPLS labels, not on MAC addresses and the links between nodes are point to point (meaning the MA in CSMA/CD becomes redundant, leaving us with CS - what L2 protocols don't have carrier sense?). I suppose technically you could put a L2 switch between the nodes but why would you want to?

I still believe that in a point to point network a point to point protocol makes more sense. PPP over HDLC as done traditionally for POS certainly has its problems, the main one being unknown packing overhead due to having the escape the SOF/EOF flags. If you use PPP over GFP though the downsides pretty much go away. I would much rather see an international network built using some flavour of PPP over WDM than Ethernet over WDM.

Ethernet just doesn't seem to fit the requirements and the standards groups all seem to be rapidly bolting new features on (e.g. MAC-in-MAC, Ethernet OAM, 1588 Synchronisation) to make it more and more like the networks it is trying to replace...

I'm going to follow up on this post but I'll leave it for now with a question:

Is my standpoint Bill Gates saying "No-one will need more than 640K" or am I the little boy saying "The Emperor has no Clothes!"?

An Explanation for the Lack of Latitude in T-Mobile Cupcake

2009-05-20T23:07:00.004+01:00

I found a reasonable explanation for the lack of latitude in the T-Mobile G1 version of the Android software.

On a thread on the Google Mobile support forum user GerardK writes:

I'm guessing that the following is the code of practice to which Christopher is referring: http://www.mobilebroadbandgroup.com/documents/UKCoP_location_servs_210706v_pub_clean.pdf Code of Practice for the use of passive location services in the UK

They're probably afraid that because this software is integrated with the phone that it is classified as being provided by the network operator rather than a 3rd party and it requires the implementation of some things that Latitude probably doesn't do yet, including...

Random notifications to subscribers by SMS that their location is monitored

Provide information on how to contact the location service provider's customer support service by phone (yep that's right, Google's phone number)

Provide a link to the code of practice

Comply with the Data Protection Act and the Regulations of Privacy and Electronic Communication

The code also includes some special regulations relating to locating children including a requirement to declare someone's relationship with a child and parental veto over who can locate them via a password-controlled parental master account and parental consent to use the service in the first place.

On the surface this seems like a good reason for T-Mobile to request Latitude is disabled in the G1 software. The fact that Vodafone haven't disabled it in the G2 reopens the debate though.

Either T-Mobile is correct in thinking that Latitude (as an integral part of the platform provided by T-Mobile) is covered by the passive location services code of conduct and Vodafone are in breach of that code, or Vodafone are correct in thinking that Latitude (as a third party service disabled by default and explicitly enabled by the end-user) is not covered by the code and T-Mobile are being arses.

IANAL so I can't really say which party is in the right here. What I can say is that at the moment I wish I was a Vodafone customer and had a phone that doesn't have crippled functionality :(

I Can Haz Cupcake

2009-05-08T17:06:00.003+01:00

Looks like I was overly pessimistic when I predicted a June release for Cupcake. My phone randomly rebooted this morning and when it came back up it was reporting an update available. The Cupcake has landed.

Official build strings: Firmware version 1.5, Kernel Version: "2.6.27-00392-g8312baf android-build@apa27 #27" Build number: CRB17

User agent: "Mozilla/5.0 (Linux; U; Android 1.5; en-gb; T-Mobile G1 Build/CRB17) AppleWebKit/528.5+ (KHTML, like Gecko) Version/3.1.2 Mobile Safari/525.20.1"

Useful stuff so far: Video camera seems to work pretty well. On screen keyboard is great for quick URL entry, etc. Find in page and select text within the browser seriously kicks arse. Various visual changes. I miss the little green droid on boot :( I also preferred the old icon for the browser (now a very generic looking globe). The ability to mark messages for bulk actions in the mail client is nice.

All in all a nice update. Some of the features shouldn't have taken this long to add, but are well appreciated now they are here :). Now if only they could release that flash player that was demo'd a few months back...

Better Late than Never

2009-03-04T10:17:00.006Z

The latest UK Android release is now available on T-Mobile G1s...

We've been running on kila_uk-user 1.0 TC5-RC8 116470 since back in November, the US got their RC33 update a month back... and now we have kila_eu-user 1.1 TMI-RC9 128600.

We get "Check for updates" and "Save attachment to SD Card" for MMS messages. No voice search, and as far as I can tell no latitude...

About as meh as I was expecting unfortunately :(

EDIT: The update includes Google Maps 3.0, which supposedly supports latitude... It seems to be lacking the menu items necessary for adding / approving contacts and publishing data. It may just be boneheadedness on my part, but I'm still not convinced latitude is there.

EDIT2: The updated browser seems to fix a lot of the frustrations I had with unicode text input. For reference, the new user agent is: "Mozilla/5.0 (Linux; U; Android 1.1; en-gb; dream) AppleWebKit/525.10+ (KHTML, like Gecko) Version/3.0.4 Mobile Safari/523.12.2"

Will RC33 equivalent hit UK?

2009-02-06T07:50:00.004Z

Some worrying wording in the announcement of Android voice search over on the Google Mobile blog.

"For those of you with a G1 in the US"

The fact that they limit the announcement could indicate that voice search won't be hitting this side of the pond (with the problems the iphone app had with English accents not really surprising). Given that voice search is the most important part of the update I wouldn't be at all surprised if T-Mobile UK don't bother with a rebuild.

Although maybe if they could add the Amazon MP3 app now that the UK store is open... (wishful thinking)

US G1 Users Get a New Android Build

2009-02-04T09:55:00.003Z

Firmware RC33 has started making its way out to G1 owners in the US (full build: kila-user 1.1 PLAT-RC33 126986).

A changes list has been published on the official forums in this thread. I haven't come across any of the bugs listed as fixed and the new features are largley "meh". Not too fussed about the inevitable delay before T-mobile start rolling out in the UK.

Would be really nice to see some Cupcake features rolled into the official G1 release. Even word on when and if would be an advance on where we are now.

Edit: I just finished the thread and it seems the market app has also been updated to show when updates are available. That one is actually worth installing for.

The biggest hurdle for open source

2008-12-23T07:24:00.003Z

I have heard several stories recently that have reinforced an opinion that I have held for a while. Open source's biggest hurdle is not reliability or usability, it is the fact that most people don't understand the principles behind it.

While listening to CS TechCast this morning I was listening to the interview with Curt Finch thinking "Is this really still new to some people"? The answer is of course yes.

There are a lot of people that see something that is free (beer) and immediately think "What's the catch?" There are certainly cases where this is a sensible approach (e.g. cash transfers from Nigeria) and this makes it a hard problem to n address. A great example is the school teacher who layed into a student for sharing a linux CD with a holier than thou "nothing is free" tirade.

I feel that maybe I have been spoiled. I have spent my entire working life in organisations where open source has been in active use. I have generally been in positions to promote further use of such software as well. While I have a windows PC on my desk to comply with corporate policy I am also free to run a virtual Ubuntu desktop which is where I do the bulk of my work. I have been involved in the installation and administration of more Linux servers than I care to think about. I regularly expand a set of internal tools for IP Address management written using the Mason perl web framework (if I were to start again I would probably use Catalyst but I don't have the time to refactor at the moment). The back end database for this runs on Mysql and was originally created using the Northstar open source IPAM tool. I have also been working with open source management utilities Cricket, Torrus, NFSen and OpenNMS.

I guess my point is that I get open source because I have a lot of exposure to it. The general public don't really have a way to get this exposure. If your company is a microsoft shop you will never get to use a Linux desktop and if you have an opinion at all on the subject it is probably based on the usual FUD spread by microsoft evangelists.

I don't know how the average normon can get enough exposure to get over the "no such thing as a free lunch" effect. Certainly the fact that a lot of consumer electronic devices now have a copy of the GPL in the box can't hurt. It would also help if more companies that use open source would give something back. The telecomms industry is very bad at this, almost all companies in the sector use open source but while many of the staff get open source and would be happy to give back to the community the lawyers handling the regulatory affairs most certainly don't get it and generally make sure that employment contracts contain an "all of your code is belong to us" clause that includes spare time coding.

I'm not saying that everone needs to go as far as Google's 1 day a week on an open source project of your choice but being able to code in the evening for my own enjoyment would be nice...

Android updates on the way

2008-12-19T07:44:00.002Z

Google recently added a new branch to the Android source repository called Cupcake. Android community have just posted on some of the Cupcake changes.

So far these are just additions to the open source codebase and there is no news on when we'll be getting an OTA from T-mobile including any of these changes/fixes. Some interesting stuff none-the-less.

AC pick up on the "Basic x86 support". This could mean confirmation of atom based netbook rumours if its x86 arch support, could indicate a start of "native client" support if its x86 compiled code support or it could mean nothing at all.

My personal favourites: Inline search and cut and paste for the browser (currently only link URLs can be copied) and stereo bluetooth support. Hopefully either the webkit update or the beginnings of IME support will include fixing unicode in the browser.

Comment Spam Etiquette

2008-12-18T17:46:00.004Z

So I just got my first comment spam since resurrecting Random Meanderings on blogger and now I have a question... What is the etiquette involved when dealing with comment spam on Intense Debate?

On my previous Self hosted MT instance I would simply delete then but that doesn't feel like the right thing to do on a social platform like Intense Debate. I have given the comment a thumbs down but the account is probably a throw away anyway... I have seen mention of ID using akismet for spam tracking but I can't find any controls for this on my dashboard. Need to read up on akismet and see if I can figure out if there is a way to flag a comment as spam to help improve the filter...

Any comments welcome (except spam).

[UPDATE]: I must be blind. Enabling akismet on ID is a checkbox on the settings page for the blog. Nothing complex or arcane about setting it up at all. Enabled for this site and meanderings so I guess I'll see how well it works... (one spam usually indicates more on the way so I shouldn't have to wait long).

Anarchy is Good

2008-12-17T19:44:00.005Z

I seem to be in a bit of a Reg bashing mood today...

On Monday El Reg posted an editorial on google's native code. I gave this a bit of a knock myself; however, unlike El Reg, I like that Google puts out largely pointless projects like this.

I have always liked the idea of anarchism in theory but as a political system it doesn't fit with the real world - most people want to be told what to do and don't want to have to think. The Internet has allowed micro anarchism to flourish amongst open source projects and the whole social networking / blogging scene.

I find the idea that a multi-billion dollar company like Google can be largely anarchistic in its internals oddly reassuring and not at all worrying.

Voice needs a reliable network. This is news?

2008-12-17T12:54:00.003Z

I don't usually post press releases from Viatel because this blog isn't run to hype my employer; however the latest release seems to have been picked up by El Reg, misquoted and has generated quite a few comments.

A colleague of mine gave an interview a few days ago and the press release ended up on lightreading.com. Nothing that surprising in it really. We've known all of this for years, but nice to know that our customers understand the problems with running Voice over most of the ADSL networks currently deployed in the UK.

There are currently pretty much 3 options. BT IPStream, BT Datastream and LLU. IPStream is provided over a shared network, has no QoS capabilities and no reliability guarantees. Pricing models mean that you have to run the central hand-offs pretty close to congestion to actually make it work. Many ISPs will deliberately congest these lines as a matter of course and this is exactly why the Net Neutrality debate is so thorny. In my opinion ISPs should not market "unlimited" services when they are knowingly congesting the infrastructure. This sentiment was echoed by another colleague of mine when he said "If a business asks for a cast iron guarantee on VoIP availability [on a contended pipe] the honest reseller has to turn round and say he can’t guarantee it."

LLU allows the provider to control their contention because everything is on their infrastructure. Most players will contend to keep costs down but not all are doing so.

Datastream is a bit "best of both worlds". It allows controlling your own contention without having to put a DSLAM in every exchange where you have customers. This relies on BTs ATM network which they want to phase out when they complete the 21CN rollout.

Going back to the El Reg Story. The story states "Viatel's point, of course, is that those issues can be resolved though the use of uncontested ADSL or something as simple as reconfiguring ADSL routers to prioritise traffic.". That's not quite the point in the original press release. Prioritising traffic at the edge of the DSL line can give some improvements; however the fact that the BT network is contended and non QoS aware means that the effect is limited. That's why edge configuration wasn't mentioned in the original story. The whole point of the story is that to deliver VoIP properly over DSL at the moment in the UK you need to use an uncontended service. A contended service with proper QoS support would also do the job but it is not currently a supported option anywhere in the UK AFAIK.

As I've mentioned previously end-to-end QoS is against the tenets of net neutrality which means that most net neutrality fanatics are indirectly opposed to the concept of reliable VoIP over the Internet...

T-Mobile Mobile Broadband with Sharedock

2008-12-16T07:11:00.006Z

Tail end of last week the shitty service on our Virgin Media cable modem came to a head. The original plan was to go for ADSL but we have been running on mobiles only with no fixed line for several years now and the DSL providers want over £100 as an up-front connection charge. So we've decided to go mobile broadband at home.

Being a T-Mobile customer already I know that they have decent 3G coverage in my area so off we go to the T-Mobile shop for the £20pm Mobile Broadband Plus with Sharedock offer.

Plenty of people have reviewed the T-Mobile Mobile Broadband service so I'm just going to focus on the Sharedock hardware.

The sharedock is a T-Mobile branded Huawei D100T Wi-Fi router with USB 3G dongle support (presumably only Huawei dongles, the mobile broadband package currently includes a Huawei E170). Prior Huawei experience prepared me for poor documentation and I wasn't disappointed; however when I logged into the box I ws pleasantly surprised by the functionality.

First tip: Don't connect the wired ethernet to your network until you have logged in and configured the box. It defaults to IP address 192.168.1.1 (same as my WRT54GS) and defaults to having DHCP turned on. Both useful defaults if you are going to be lugging it around for mobile access. Less so if you want to plug into a wired network. Changing the IP and disabling DHCP was just a matter of a few clicks though, so no real drama.

Wireless security has all the usual suspects and setting up WPA2-PSK was again only a few clicks.

The box itself will act as a caching DNS server so no need to dig out T-Mobile IP addresses if you turn the DHCP off.

All in all I'm very happy with the box so far, just a couple of minor niggles.

Activation of the online account management and hotspot access both depend on recieving SMS messages. The USB stick is capable of SMS and the included software allows sending and recieving of SMS but the sharedock doesn't. I ended up swapping the sim into my G1 because my shitty PC takes forever to recognise new USB devices.

Second tip: The installation instructions assume that you have autorun enabled for removable devices. I don't for two reasons. There's the obvious security issue but most importantly for me is because autorun pisses me off. If I want to run it I'll double click on it. On a fresh install you need to run the autorun twice - once to install the software and a second time to switch the device into 3G mode (it defaults to mass-storage for driver installation).

The (not so) transparent web proxy is also annoying. Images in web pages are automatically compressed giving a noticable reduction in quality. There is an app that can be downloaded from T-Mobile to turn this off but it can be tricky to find. The second thing with the proxy is that it seems to break access to t-mobile.co.uk. I assume this is temporary and it hasn't annoyed me enough to actually call 150 yet.

Last niggle, an oldie but still isn't fixed. If you are going to put a fair use cap on downloads then give users access to the stats! This isn't rocket science. If you are tracking it for billing purposes and don't collate it in real time then print it on the bill. Just give us a clue.

Do Google's Devs Speak to Each Other?

2008-12-10T14:27:00.002Z

OK, I'm confused again. Google have announced Native Client for embedding applications in web pages.

Immediate thoughts: What if you're not running on an x86 architecture? Given that most of my recreational browsing is done from my Android handset these days this looks like yet another technology that I will avoid while browsing.

My suspicion here is that Google are just rocking the boat again. ActiveX was a train wreck. Silverlight... ::shrugs:: Never really come across anything that used it. That leaves Flash and Java. Flash is a bloated CPU hog in most cases but I've seen it used very well. Java can be a bit bloated in the libraries, but well coded Java can run pretty damn quick. Both of them are a lot more portable than NaCl (I have to admit to liking the acronym).

Something that would excite me is if they were to announce that they'll support NaCl targeted for Dalvik rather than webpages. This would give Android developers additional languages to play with as well as allowing embedded content in Android's mobile safari browser... I'm not holding my breath though.

Eedroid?

2008-12-10T13:36:00.004Z

Something I neglected to mention last night when I posted about new Open Handset Alliance members was the addition of ASUS. Some people are reporting that this may be confirmation of the rumours about an Android powered Eee PC/Netbook.

I'm trying not to read too much into it though, as ASUS already make quite an extensive range of mobile handsets, so joining the OHA makes sense anyway.

Fairwell AS2.0

2008-12-10T07:16:00.004Z

The IETF have published RFC5396 recommending that the plain decimal representation should be used in all cases. I have to admit to being disappointed. I kinda liked the asdot presentation format and it certainly made 32-bit ASNs easier for humans to interpret. I can see that asplain or asdot+ are easier from a machine interpretation point of view though.

Can't help but wonder why this RFC is needed when the aaaa:bbbb format for communities and the a.b.c.d:nn format for route distinguishers are allowed. I suspect it is likely due to the use of a dot as the separator: AS paths are often processed by regular expressions and junior operators used to commmand line globs often don't realise dot is a wildcard and needs to be escaped. Colon wouldn't work because it makes it hard to distinguish between ASNs and communities.

Sony to go Android

2008-12-09T18:50:00.006Z

Sony (and others) join the Open Handset Alliance. This could be very nice in theory but I have my doubts about a company like Sony fully embracing an open platform.

If Sony were to properly embrace the Android concept and contribute to the community it could be great. If Sony put an Android handset towards the top of their range they are likely to want to put the Walkman logo on there, which will mean them writing a new media player for Android. There is no chance they will allow that app to find its way onto anyone else' hardware... Vendor specific apps like this are inevitable but I don't have to like it though :(

With mozilla saying that they won't port firefox mobile if they have to code in the Android Java environment a port of. Songbird is also unlikely.

Maybe I'm overly pessimistic. Time will tell.

Will the big telcos ever get a clue?

2008-12-09T07:48:00.003Z

What a joke. It seems that the big US telcos are still sponsoring Scott Cleland to talk crap. His latest.report claims Google pay 1:21 of what they should for bandwidth.

Predictably he seems to be basing his argument on the traditional PSTN settlement model. The Internet is not the PSTN and the sooner the big players realise this and stop trying to meddle the better.

In the traditional PSTN model each call is billed individually, usually to the call originator but it can be billed to the terminating party in a toll-free scenario. The only way the premise of this report stands up is if you assume google to some form of toll-free directory service. This argument is flawed in many ways, including the fact that PSTN directory services are rarely, if ever, free to the calling party these days.

Users pay for all bandwidth on their lines. Google pay for all bandwidth on their lines. The difference is one of scale. Pushing multiple Gb/s on peering is a lot cheaper per bit than consumer DSL? This is breaking news?

The connections that large content providers like Google have is actually much better for the telco than the consumer DSL. At least you can bill based on usage there without some normon complaining about getting a bill for the bandwidth he consumed distributing illegal content.

Flawed reports like this just damage the appearance of the whole industry and he should have his sponsorship pulled for peddling this tripe. The content of this report should be laughable but I find myself unable to laugh because yet again spin from the big boys is painting the whole industry in a bad light.

My First T-Mobile Customer Services Experience.

2008-12-04T22:08:00.007Z

The fact that T-Mobile have dropped the minimum contract requirement for the G1 from the £40pm Flext40 plan to the £30pm Flext30 plan as well as bundling an 8GB SD card instead of the 2GB I got on release day is annoying, epecially because they did this less than a month after launch. I entered a contract though so I have to live with it I thought.

I was a bit surprised yesterday when I read this Android Community post. Wow. T-Mobile sound like they actually realise their customers are important and want to be fair.

I should know better by now.

I am not sure where the info about allowing price plan changes came from but it is certainly wrong. The email I received from customer services points out in no uncertain terms that I was unlucky and am stuck with my current plan.

The G1 may not be challenging the iphone, but shipping a million handsets in the first month during a global recession is no mean feat Yet still T-Mobile don't seem to care about the early adopters and online evangelists that helped them achieve this success.

Thanks for confirming my disillusionment T-Mobile. None of the other carriers care about their customers, so why should you?

[UPDATE] Pocket lint (the original source of the story above) are reporting that their T-Mobile contacts are still confirming that early adopters will be getting a refund. More later...

[UPDATE] The figure of 1 million in the first month was due to misreading on my part. The actual story is that HTC expect the handset sales to hit a million by year end. I still think 1 million in two months is impressive.

Imports of old posts

2008-12-02T17:17:00.002Z

I'm going to be posting a few old articles from my Moveable Type archives. I'll be posting them with the original date, so in theory they shouldn't show up in the feed. Apologies if they do...

Are Google missing out on the Hype?

2008-12-02T11:04:00.003Z

I have seen several stories today referencing the latest Net Applications Market Share report most of which have been pushing the fact that Windows has fallen to below 90% share of web hits and that Safari is up.

When I had a quick view through the reports I started to wonder "Where are the stats for Android based devices?" Are there really more people browsing the web from SCO unix than there are on G1s?

The user agent reported on the G1 is:

"Mozilla/5.0 (Linux; U; Android 1.0; en-gb; dream) AppleWebKit/525.10+ (KHTML, like Gecko) Version/3.0.4 Mobile Safari/523.12.2"

Two things here: The OS is reported as Linux with Android 1.0 as the distribution, so that'll be why its not showing up in the OS reports. Secondly the closest thing to an interpretable browser version string is "Version/3.0.4 Mobile Safari/523.12.2".

I had to do a quick experiment. I hit a site which I have Google Analytics running on with my G1, and then had a look at the visitor stats. Lo and behold in my analytics stats one of my visitors was running Safari/523.12.2 on Android (Google recognise Android as a seperate OS which isn't surprising but most others will bundle it up as a Linux distribution).

Not giving the Android browser a meaningful user-agent probably ups the compatibility with existing sites without webmasters needing to recode but its a bit of a shot in the foot from a PR point of view. I am left wondering whether any of the Safari growth reported in the Net Applications article is actually due to the launch of Android...

Opera on the G1 First Impression

2008-11-24T19:56:00.001Z

Opera Mini 4.2 beta is now in the Android market. I've been using it for a couple of hours and am ready to post some first impressions.

First the good. Opera mini has two features I have wished for from the beginning on the built in browser: Unicode text input support and a pointer when using the trackball. These two features really make me want to love this browser but unfortunately there are other factors.

The scrolling is slow and jerky. The flick gesture for quick scrolling doesn't appear to work which makes navigating large pages painful. Text input has proper character encoding support but it is very clumsy. Selecting any text input field brings up a full screen zoom which is good for multiline fields but makes quick entry of login details quite painful. passwords are fully visible when the field is zoomed, not good when you are in a cramped environment and could have people looking over your shoulder. Exiting a text field is inconsistent. There seem to be at least four ways to exit a text field (touch the screen, press the trackball, press enter or press menu and select ok). Most of these methods do not work all of the time and there doesn't seem to be predict which ones will work. The only one that always works is the trickiest (menu->ok).

As far as I can tell there is no tabbed/windowed browsing which is a showstopper for me.

A good first attempt but a long way still to go.

That Old Chestnut: Net Neutrality.

2008-11-18T21:45:00.002Z

Listening to my podcasts on the way into work today and it seems that with Obama measuring up for the Whitehouse people have dredged up the Net Neutrality debate again.

A while back some Australian operators caused some controversy when they said that Net Neutrality is an exclusively American problem. I'd probably take this a bit further: Net Neutrality is an issue because people give marketeers too much slack.

Declaration of interest: Please note that I work in the telecommunications industry and so am immediately the bad guy from many people's point of view. If you disagree then feel free to comment or trackback; don't expect to change my mind though.

The entire reason for the much maligned "Fair Use Policies" is to satisfy sulking marketing types threatening to throw a tantrum because "Next door give unlimited access. We need to give unlimited access or I'm throwing my crayons on the floor". Providing network access is a lot more expensive than most people outside the industry realise. From a technical/Engineering standpoint unlimited has always been shaky ground (note: my employer does not advertise unlimited for contended services). Unlimited/uncontended services are expensive.

Overbooking using statistical multiplexing in the core is standard practice and perfectly acceptable. If the ISP were to size their core to handle every end-user line running at full rate they would probably peak to around 1-2% utilisation on the core which just isn't economically feasible. If this works in the core, then what's the problem with extending it to the access layer?

Statistical multiplexing only works because of the individual connections averaging out gives a predictable aggregate. As you move closer to the edge you reduce the population of the aggregate and lose the predictability. Overbooking this edge capacity with no prioritisation allows a small number of heavy users to screw over all the normal users. The only way to give users the prices they demand is to overbook this capacity. With net neutrality restrictions in place there are two options: let everyone's service be degraded, or raise everyone's subscription to cover a capacity upgrade. Both of these solutions equate to the normal users being screwed over by the heavy users. If you scrap net neutrality you can charge the heavy users extra, and use this revenue to upgrade the capacity giving everyone a better service.

Even in the core the situation isn't cut and dried. I am certainly in favour of neutrality across customers in the core but I do not believe in neutrality of services in the core. Some services (such as voice) need to be prioritised or they don't work to an acceptable level. And where do you draw the line? If all traffic must be treated equally does that mean that you can't filter DDoS traffic because the malicious user deserves equal treatment?

With all this in mind I will also say that a lot of ISPs (especially the big players) have done a really bad job when it comes to fairness. If services were sold with their limits documented and if transparent metrics are used this would never have escalated to this level.

Also, if you are going to implement a bandwidth restriction, show the user what they are using. Preferably also show what the average bandwidth per user is alongside it. Most normons have no idea what a Gigabyte is - telling them they have 20Gb per month is pointless. It would be like signposts with distances in furlongs: sure its a measure of distance, but how many people have a clear picture of how long a furlong is in their mind?

In conclusion, I do not believe that the popular understanding of the term Net Neutrality would be good for the Internet and I do not believe that differentiation of services stifles innovation. I believe that enforcing net neutrality would stifle innovation because it would raise prices and hamper take-up of services.

I love iKnow!

2008-11-18T18:58:00.004Z

Just in case you hadn't noticed; I am currently studying Japanese on a site called iKnow!. The general premise is not new. Academics researching memory have known for decades about principals such as forgetting curves and spaced repetition (which actually stems from physical flash card system but has become much simpler with the proliferation of the personal computer) and Cerego, the company behind iKnow!, have been actively involved in this research for a long time.

Recently a few sites have realised how the web2.0 social networking thing fits learning like a glove. Sitess I have used along these lines include Reviewing the Kanji and to a lesser extent Anki. IKnow! Takes this to the next level and then some.

They started testing a while back with English lessons for Japanese speakers. This expanded in September to include Japanese for English speakers (which is where I jumped on board). Their presentation when they launched at Demo clearly demonstrated they weren't happy to stop there and there are signs that they are approaching the launch of further subjects. A public example of this is the announcement that there is a Mandarin course being sponsored by the US government. A less public example is that a list recently appeared containing a proof of concept molecular biology course for NYU.

Getting back to the present: Why do I love this system so much? They are not approaching this half cocked like many web2.0 startups do. The Japanese courses are now up to around 6000 vocabulary items all of which have professional voice acting for both the item and example sentences and the English content seemsa to be of a similar high standard. The officially produced content is just the start though. Every user can create their own lists around themes, textbook chapters, blog posts, youtube videos, etc. When you add friends you get notification when they pass milestones so you can congratulate them. These congratulations as well as prompts to write journal entries do a good job of drawing you into the system and getting you involved. This makes learning fun which helps both retention and motivation.

iKnow! Is one of those select few sites that manages to be simultaneously useful and well presented. I'm proud to be an early adopter and wish them all the best because I'm planning to be using their services for a long time to come.

Win or Fail? You Decide...

2008-11-17T10:40:00.003Z

Apparently Visa have demoed a new credit card with integrated one time key generation. The card features a small eInk display and 12 key keypad (0-9, OK, Cancel).

Looks pretty cool. One time passwords FTW. As is eInk (I really hope some large scale eInk production runs like this will bring the prices of readers down). One immediate problem that springs to mind though... These cards are usually valid for 2-3 years. In that time period there will be a lot of pressing of the keys; which means that a moderately used card will probably give away the digits of your pin, meaning that someone who has stolen your card only has to try around two dozen combinations to get a four digit pin rather than ten thousand...

Although if you change your pin this risk is reduced. Hopefully this thing will allow for more than 4 digits in the pin. The limit to 4 digit pins in the UK is seriously braindead.

Will Classful Addressing Ever Die

2008-11-14T12:34:00.003Z

A perennial favourite just came up again in the office.

This customer wants a class C.

No. They don't. They want a /24 assignment from one of our CIDR allocations.

The Internet moved away from using Classful IPv4 addressing over a decade ago (RFC1519 was published in 1993. The current BCP is RFC4632). Yet still people perpetuate the terminology.

I frequently get requests about customer class C networks and most of these requests are about networks that wouldn't have been considered class C before the publication of RFC1519, let alone afterwards.

The original definitions of the classes were based on the most significant bits of the network address. This meant that a network with a 0 in the MSB is class A. 10 would indicate a class B, 110 Class C, etc (although D, E, F were obsolete before they were ever used). Most active allocation pools for LIRs currently are using parts of the old class A space (last allocation I received from RIPE was from what would have been the 74/8 Class A way back when). 10.1.1.0/24 is not a class C. When classful addressing was common this would have been a Class A subnetwork. Nowadays it is simply a /24 network. Possibly a "/24 network from the former Class A space" if you are feeling verbose.

The only networks that could be validly called class C networks are those with a /24 mask between 192.0.0.0/24 and 223.255.255.0/24, although personally I would still call these /24 networks in the former Class C space or probably just "/24 assignments".

The Internet is Classless these days people! Get a clue.

G1 Firmware Update

2008-11-14T09:25:00.005Z

OK, so I finally have the update so here's an update to my earlier post.

It seems that the guys at Google realised the impact of this problem pretty quickly and I have to say they did a good job not only getting the patch out quickly (I believe it was <24 hrs from bug report to the new android builds) but also for pushing T-Mobile into getting it out to the users.

I recieved a message saying that the T-Mobile UK guys were testing the security update on twitter Nov 4th at 10:22 GMT. Interestingly this is around 14 hours before the released TC5-RC8 was built, so I'm not sure what they were testing at that time...

It appears that the latest release kila_uk-user 1.0 TC5-RC8 116470 brings the G1 in the UK up to speed with the US firmware version TC4-RC30 116143. RC30 was the rebuild specifically to fix the issue with a root shell being attached to the console at boot and so is not that interesting; however up until now the UK has not had the various feature fixes released in the US as RC28 and the security fixes in RC29.

I have only had the update installed for around 4 hours now so it is too early to tell how it affects the performance (some of the fixes reported in RC28 include the elimination of the annoying pause when returning to the homescreen when the phone has been up for a while as well as mixed reports on battery life improvements). I can confirm that the SD card is not automagically mounted as soon as you plug the device in; you now get a notification in the status bar that you need to select. The most common reason for me to plug my phone into the computer is to charge it, file transfers are much less common, so this behaviour makes a lot more sense for me.

To sum up: Kudos for Google for the fix, but to T-Mobile UK - 10 days to evaluate a security fix? Could have been a lot worse, but it could have been a lot better too...

Googles endless beta mindset

2008-11-10T08:36:00.003Z

A lot more is understood about the jailbreak now. Instructions even go so far as showing How to install debian on the G1

One thing jumps out. In my last post I was wondering how an unpriviledged shell could start up a telnet daemon. The answer is: it can't. The reason the jailbreak works is that everything you type into any application is sent to a root shell for interpretation. This can easily be seen by typing ¶reboot¶

What the hell were they thinking? I can see that this could possibly be useful during development; but putting normons in a root shell without even telling them? That's a recipe for a bricked phone! Dislike someone with a G1 and some basic sysadmin knowledge? Send them a text asking how to erase all files on the system. In the time it takes them to type rm -rf /¶ you have just screwed their phone.

I second the suggestion made by Ed Burnette that all android users type ¶cat¶ to swallow all input sent to the root shell and protect themselves until the next reboot.

Another write up of the G1 jailbreak

2008-11-07T17:56:00.004Z

OK so the G1 jailbreak is pretty well documented already but so far all the guides I've seen require you to be on a wi-fi hotspot. I have a variation that works without wifi so I thought I would share it.

First of all you will need to have both pterminal and connectbot installed. You will also need access to a unix shell account via ssh and permission to set up tunneling on the connection.

First start up pterminal and enter the following command: /system/bin/telnetd

Next fire up connectbot and make your remote connection.

Once connected go to the menu and hit the tunnel button.

Select remote. Choose a random high port (e.g. 65456) for the source and set the destination to localhost:23

From the remote commandline issue the command: telnet localhost 65456 (or whichever port you chose)

Welcome to the Android root shell.

This will be fixed in RC30 in the states. No word on when it will be fixed in the UK.

I have to admit to being a bit confused over how it works. It always used to be the case that unpriviledged accounts couldn't bind to low numbered ports and the telnetd file isn't SUID. How a non SUID process can bind port 23 and exec sh as uid 0 I have no idea...

Google have probably been playing with capabilities and not fully understood what they were doing...

2008-11-05T19:01:00.001Z

lex poses for the new phone

Posted via Pixelpipe.

G1 Love/Hate

2008-11-05T06:41:00.001Z

At the risk of feeding a meme...

So I've had the G1 for nearly a week now and have been using it heavily. In the spirit of the posts by Ewan and Mostlythis here are some of my thoughts on Android and the G1.

Internal memory There's just not enough. Not sure whether to blame HTC for only installing 256M or to blame google for mandating that Market apps cannot be installed on the SD card.

No release notes I guess I'm spoiled by working with Cisco/Juniper. I'm used to masssive publically available docs with lists of fixes. Some way to see which public source version a release branch is rooted from would be very helpful here. Come on guys, this is supposed to be open!

Direct address entry Going directly into address entry when you type in the browser is very annoying when you are on a site that provides keyboard shortcuts for quick navigation.

Lack of task manager I've basically started to turn the phone off once a day to clear up any apps which are hanging around in the background. pTerminal and ps/kill give a back door to some of the required funcionality but I would prefer to be able to use the GUI

Lack of documentation the market doesn't link to the project site and most apps don't include docs to keep package size down. Some of the apps I have tried are probably very good but have been binned because I can't figure out the interface.

no flash this needs to work well when it arrives. Letting the community develop it probably isn't going to deliver the goods

This has turned into a bit of a bash session... Not really sure how that happened. I guess it's just easier to put the annoyances into words.

connectbot as a network engineer and unix geek a good ssh client is essential. And this is a good SSH client.

No app censorship we all know about the app fascism over at the apple app store. Hearing news about apple blocking Opera for the iPhone made me glad I opted for Android.

Could probably go on for hours... Time to sign off.

G1 Firmware Rundown

2008-11-04T11:27:00.000Z

OK, so I signed myself up for a G1

Being a techie I find myself fascinated by the software versions and OTA updates. There is quite a bit of info out there about the versions of Android that have been seen in the wild but I can't find an aggregated source of info so I thought I'd post what I've found just in case anyone else is interested...

Device was first launched in the US market on Oct 23 2008. Out of the box phones have android 1.0 build TC4-RC19 (109652). The first OTA update in the US was soon after launch TC4-RC28 (114235). This was quickly superceded (before the roll out of RC28 had finished) by TC4-RC29 (115247) which fixes the well publicised browser flaw.

In the UK the phone launched on Oct 30 2008. I recieved mine in the post on Oct 31st and promptly went into the settings menu to find the release. TC5-RC7 (112931).

There is no official documentation that I can find for what is or isn't included in each release. I am currently working on the assumption that the six digit number is some sort of commit ID in the google code repository (not the open source repository, Google's internal repository) which would place the current UK release somewhere between the US shipping release and first update. It puts it well before the security patch, which is confirmed by this tweet from the T-Mobile Twitter Team. The yet to be officially announced UK security update is due to go out over the air in November 2008.

So far I have uncovered the following information about the updates:

TC4-RC28 Fixes problem with Amazon MP3 store. Unspecified "enhancements".

TC4-RC29 Security updates.

TC5-RC?? Incorporate security updates in TC4-RC29

As yet unconfirmed... Do those of us in the UK get the rest of the fixes in TC4-RC29 when the UK release gets pushed out...

The US release of RC29 started around Oct 28th and is planned to be complete by Nov 12th. The UK release hasn't even started yet...

2008-09-18T19:17:00.003+01:00

Are orange really bad at accounting, or is my package much better than the docs imply? I'm on dolphin payg which includes social network access but only facebook/bebo/myspace. I've been on vox Shozu ping.fm and twitter without incurring charge...

2008-09-18T18:19:00.001+01:00

Currently enjoying http://iknow.co.jp/ looks cool. Don't seem to be able to do much from my mobile though.

2008-09-18T07:06:00.001+01:00

Bloody knackered. And coming down with a cold. Is it the weekend yet?

2008-09-17T17:57:00.001+01:00

Playing with http://ping.fm and am pretty impressed so far

Post Aggregation

2007-09-18T11:33:00.001+01:00

So I'm moving from personal installs to hosted installs. I want a single RSS feed I can pull into every site. Ahah! I think, I've already got one of those in Rojo... or at least I set one up ages ago and never bothered updating it. So off I go and add in all the RSS feeds I can think of and tag them as "me". I leave it an hour and check back - great, the new posts are all there. Unfortunately for some reason rojo is not reading the post time from the rss feed and has marked them all with the download time screwing any hope of ordered history. Oh well, at least it should be ok going forward...

Moving away from self hosting

2007-09-18T10:07:00.000+01:00

I've decided to move away from self hosting all of my blogs to putting each strand into a different hosted blog platform. Mostly because I never seem to find time to make sure my MT install is up to date, but also partly so that each site can benefit from being part of a social network. I have some ideas in the back of my mind for making some sort of aggregator website using Catalyst... Maybe I'll even get around to them ;) Here on blogger I will focus on technical posts (perl, networks, etc). Family/photo/hobbies will be over on my Vox. Meanderings will move to LiveJournal. Music at last.fm.

Chasing Ubuntu

2006-08-26T20:07:00.001+01:00

I've been hearing a lot of good things about Ubuntu Linux for a while now.

I'm a debian boy and have been for years (I can't remember whether it was bo, rexx or buzz that I first installed… whichever was earliest). I am plenty happy with my etch box, and have no problem keeping up with the various packages, etc directly using apt-cache and apt-get; however I share the PCs at home with my wife and son and have recently started wondering whether the Debian based Ubuntu distro was worth a shot.

I've done two installs of Dapper (Ubuntu 6.06) now, both under vmware (workstation and server). The workstation install I did first was an absolute breeze; I was thoroughly impressed both with the ease of install and the default selection of desktop packages. Blimey, I thought, the hype's true. And mostly it is. I found way they have tuned GNOME gives a similar feel to MacOS X.

I was running the first test on my work PC (left it installing in the background while working in my etch vm). Once it finished the install I could more or less switch everything straight over. I found that a few packages I use were a bugger to find (pretty old git-core and no cogito, found it with some tweaking of the active repositories). Found that the evolution and nautilus installs were a breeze for working with the corporate windows network.

Installing on my home etch box under a vmware server vm was trickier, didn't realise the memory requirements of the desktop live cd. The 128M VM I was running under just wasn't up to the job. Eventually after some googling I found that I needed the alternative CD, and the text based install went fine.

I now need to do some serious thinking. I've been a debian evangelist for years; I think it was in 97 when I suggested to the boss that the slackware distro wasn't up to much and we should look for a change. He couldn't get his head around dselect, but I immediately saw the power of the strong dependency tracking in the deb package format and was hooked. Now I'm seriously considering moving my primary home pc away from Debian. Although at least it's still a distro based around deb and apt suite at its core.

Going to be away from the computers for the next week at my grandad's place in Wales. I'll use the time to consider the options very carefully.