Often when turning up new BGP sessions in your network you want to be careful not to upset the balance you have. You want to bring up your new peering BGP sessions in a controlled manor.
Sometimes after turning up a BGP session there might be some user complaints. These may or may not be related to the BGP session you just turned up. It more often than not just a coincidence but we panic and say what did we change?
The first reaction is to shut the BGP session down and hope that the problem goes away.
Rather than taking such a drastic approach here are some ideas on how to take a more measured approach.
Typically there is a Route Collector and two Route Servers at an IXP. This is no different at JINX, CINX, DINX and NMBINX.
The Route Collector (RC) is a great tool to use as a "mirror". The RC collects routes but does NOT advertise them to any peers.
Therefore the most logical first step is to setup a BGP session with the RC at the IX you have just connected to.
Use the same outbound BGP filter that you would want to use on the Route Servers (RS) with your BGP session with the RC.
Once the BGP session is established you can then use the Looking Glass (LG) to see what your advertisements really look like.
You can visit the INX Looking Glass here : https://lg.inx.net.za/
Select the route collector for the IX you are connected to and search for your network.
You can now see what prefixes you are advertising across the BGP session with the RC. In this modern age of BGP communities you can also see the BGP communities you applied (or think you applied :) )
You can now play with your outbound filter and check after you have made changes what they look like. Remember that this will not affect any routing as the RC doesn't advertise these out.
It is always a good idea to create an inbound filter on your side for the BGP-RC session that denies all prefixes. Even if you trust a peer it is always a good idea to have filters in place.
Every BGP session should always have at minimum
- outbound filter
- inbound filter
- max prefix limit set
Right now you have your outbound filter ready.
Spin up the BGP sessions with the Route Servers, but lets take it one step at a time. Have a DENY-ALL filter set on both your inbound and outbound filters for the RS-BGP sessions.
Now you have all three BGP sessions up at the IXP.
This is a great place to rest and breathe. Your NMS should be happy as there are no down BGP session alerts on the dashboard. The IXP operator is happy because there are no alarms down for BGP sessions.
When you have caught your breathe you can then set your outbound filter on one of the Route Server BGP sessions to the same as that of your outbound filter for the Route Collector.
Wait and see how traffic starts to shift. As your prefixes are accepted by other networks they will send you traffic and traffic will come inbound into your network.
Yes, you will have one way traffic. Asymetrical routing does not break things. There is a lot of asymetrical routing going on out there on the big wide internet :)
If things break for some reason remember to do traceroutes in both directions from the source and destination IPs. This will help to debug.
Change the RS outbound filter back to DENY-ALL. Don't shut the BGP session, it will make your NMS unhappy and is not solving any problems by shutting them down.
Have a look at the traceroutes and work out why things may not be working the way they should be.
Generally speaking if you have standard filters on all your BGP sessions across your network there should not be any issues once you start advertising your prefixes to the Route Servers.
Once you are happy with the prefixes you send to the one RS, amend the outgoing filter to the other RS so you advertising the exact same prefixes to both route servers
There are two Route Servers for redundancy, you want to have both these sessions up and configured the same.
When you are happy with your outbound routing you can now apply an inbound filter.
Remember that you are receiving around 100 000 prefixes from each of the route servers and this may make your router work a bit harder than usual until it has imported all the routes and recaluated which is the best route.
Apply your inbound filter on the one route servers and keep an eye on things and you should now see some traffic leaving your network via your peering interface.
Once happy with that apply your inbound filter to the other RS.
In summary
Generally there will be no issues in turning up Route Server BGP sessions, however you may hit a snag and need to debug this.
Keep your BGP sessions up all the time but change the incoming and outgoing filters until you have worked out what the issue is.
As always if you are having issues get in touch with the INX Ops team.
