BGP Traffic Engineering

This document is intended to give an introduction to communities for people who would like to take advantage of their upstream ISP's BGP communities to engineer their traffic.

Please note that this isn't a complete BGP configuration HOWTO. This document won't teach you how to filter announcements in a multi-homed environment, you should already have a functioning BGP setup before you attempt this. Follow the Simple or Transit AS Howtos first! Be Warned!

BGP communities are as means of 'tagging' or 'colouring' BGP routes. These 'tags' can then be used to filter and manipulate routes at other points in the network, even in upstream and downstream networks! Communities are about information. Information is power.

We're going to use egress and ingress engineering communities to split our traffic so that all domestic traffic goes via Onyx and all non-domestic traffic goes via Demon. This is a very simplistic example, but should lay the groundwork for you to understand more complicated BGP traffic engineering applications.

For the purpose of this guide, we're AS45000 with two upstream providers, Onyx Internet and Demon Internet. Onyx has two upstream providers too, Level 3 and KPN. Onyx also has a connection to the LINX for peering. It's not overly important what Demon has, as we'll be focusing on engineering our announcements through Onyx.

The diagram above shows an example of how routes flow between the ASes involved. For the purposes of this document we're going to ignore any communities being announced from Level3, KPN or Demon. Demon will be recieving similar routes to Onyx from similar locations.

Onyx is recieving a full global routing table from both Level 3 and KPN, and a domestic routing table from LINX peers. BGP will prefer the LINX routes because they'll have a lower AS-PATH than the routes via Level 3, for example, to reach a Mistral ADSL customer (10.2.32.45) from our AS we might have the following AS-PATH via Level 3 :

But via the LINX where we peer with Mistral the AS-PATH would be much shorter :

Onyx will therefore favour the LINX route and send it on to us. If Demon don't get the Mistral routes via LINX, we'll favour Onyx's shorter route over their Level 3 route.

As you can see from the diagram, our local routing table is built from a mix of routes from several different locations. We've preferred Demon to reach 12.0.0.0/8, presumably because they peer with that network, whereas Onyx doesn't.

Now that we've established where our routes are coming from, you can see that the split isn't very equal, most of our routes are coming in via Onyx! This means that we'll send most of our traffic out via Onyx and probably that most of our traffic will come in over Onyx too. We're going to solve this by adding engineering communities to our announcements to Onyx and selectively chosing which routes we want to use based on the BGP communities Onyx are sending us.

Firstly we'll tell our router to apply route-maps out our incoming and outgoing announcements :

Next we'll setup ingress route-maps to prefer Onyx routes over Demon routes for domestic traffic, but Demon routes over Onyx routes for everything else:

As you can see from the configuration above, we first try to match all incoming routes from Onyx with the community 6067:3000, which identifies Onyx's peering routes and set their preference to be 200, if they don't match the community we set their preference to 50. We then set the preference for all Demon routes to be 100. It's worth noting at this point that local-preference will override AS-PATH calculations. The flowchart below depicts how routes will be handled by the route-maps :

Now we will announce a community to Onyx to tell them to prepend our announcements 3 times with their own AS number when they announce them to Level 3 and KPN. This will cause the AS-PATH for those routes to look like this :

This will make the announcements for our networks coming from Onyx look very unappealing compared to the announcements coming from Demon.

At Onyx's side they'll have a route-map which filters ougoing routes on transit BGP sessions and anything matching 6067:31000 will have AS6067 prepended to its AS-PATH 3 times before it's announced.

You could now monitor how your traffic flow has changed and see if you need to perform any more complicated engineering by announcing different blocks with different preferences, for example.

Anyone wanting to implement engineering communities within their AS should read the Transit AS HOWTO.