building a rest API for ExBGP

The last couple of years there is a trend to extend layer three to the top of rack switch (TOR). This gives a more stable and scalable design compared to the classic layer two network design. On major disadvantage of the layer 3 to the TOR switch is IP mobility. In the classic L2 design it was a simple live migration of a vm to a  different compute host in a different rack. When L3 is extended to the TOR IP mobility isn’t that simple anymore. A solution for this might be to let the VM Host advertise a unique service IP for a particular VM when it becomes active on that VM host. A great tool for this use case is ExaBGP.

ExaBGP does not modify the route table on the host itself it only announces routes to its neighbours. After ExaBGP starts the routes it advertises can be influenced by sending messages to STDIN
Below is the config used by the ExaBGP daemon

group ebgp {
neighbor {
local-as 65001;
peer-as 65000;
process add-routes {
run /etc/exabgp/;

Most of this is pretty self explanatory the important stuff happens on line 9-11. These lines start a script and all output of this script is parsed by ExaBGP.

The script provides a rest API which outputs on STDOUT the announce and withdraw commands for ExaBGP.

For testing purposes I created a simple setup within KVM and two hosts, docker1 which runs ExaBGP and firewall-1 which runs the birdc bgp daemon. There is a L2 segment between those clients over which BGP peering is created

The python script is only 75 lines long.

#!/usr/bin/env python
import web
from sys import stdout
from netaddr import *
from pprint import pprint
urls = (
    '/announce/(.*)', 'announce',
    '/withdraw/(.*)', 'withdraw',
class MyOutputStream(object):
    def write(self, data):
        pass   # Ignore output
web.httpserver.sys.stderr = MyOutputStream()
class bgpPrefix:
    def __init__(self,prefix,action="announce",next_hop="self",attributes={}):
        print self.attributes
    def get_exabgp_message(self):
        if (self.action=='withdraw'):
            exabgp_message="{0} route {1} next-hop {2}".format(self.action,self.prefix,self.next_hop)
            for attribute in self.attributes:
                 if attribute == "local-preference":
                     attribute_string+=" local-preference {0}".format(self.attributes[attribute])
                 elif attribute == "med":
                     attribute_string+=" med {0}".format(self.attributes[attribute])
                 elif attribute == "community":
                     print self.attributes[attribute]
                     if len(self.attributes[attribute])>0:
			 attribute_string+=" community [ "
			 for comm in self.attributes[attribute]:
			     attribute_string+=" {0} ".format(comm)
			 attribute_string+=" ]"

            exabgp_message="{0} route {1} next-hop {2}{3}".format(self.action,self.prefix,self.next_hop,attribute_string)
	return exabgp_message
def verifyIp(ip):
    if not '/' in ip:
        raise web.badrequest("invalid IP")

class announce:
    def GET(self, prefix):
       # bgp_prefix=bgpPrefix(str(ip_object),action="announce",attributes={'local-preference': 300})
        stdout.write( bgp_prefix.get_exabgp_message() + '\n')
        return "OK"

class withdraw:
    def GET(self, prefix):
        stdout.write( bgp_prefix.get_exabgp_message() + '\n')
        return "OK"

app = web.application(urls, globals())

if __name__ == "__main__":

The heavy lifting of the web service is handled by this is a powerfull library to create a webserver in Python. I am a network engineer with very limitted experience with Python but creating the script only took me a couple of hours.

The script in action

We start with starting the ExaBGP Daemo

Mon, 29 Aug 2016 21:01:14 | INFO     | 15213  | reactor       | New peer setup: neighbor local-ip local-as 65001 peer-as 65000 router-id family-allowed in-open
Mon, 29 Aug 2016 21:01:14 | WARNING  | 15213  | configuration | Loaded new configuration successfully
Mon, 29 Aug 2016 21:01:14 | INFO     | 15213  | processes     | Forked process add-routes

Mon, 29 Aug 2016 21:01:16 | INFO     | 15213  | network       | Connected to peer neighbor local-ip local-as 65001 peer-as 65000 router-id family-allowed in-open (out)

By default the service is started at port 8080

root@docker-1:/home/eelcon# netstat -anp | grep 8080
tcp        0      0  *               LISTEN      15183/python

The BGP neighbor is also shown as established by bird

bird> show protocols all bgp3
name     proto    table    state  since       info
bgp3     BGP      master   up     15:01:33    Established
  Preference:     100
  Input filter:   ACCEPT
  Output filter:  REJECT
  Routes:         0 imported, 0 exported, 0 preferred
  Route change stats:     received   rejected   filtered    ignored   accepted
    Import updates:              0          0          0          0          0
    Import withdraws:            0          0        ---          0          0
    Export updates:              0          0          0        ---          0
    Export withdraws:            0        ---        ---        ---          0
  BGP state:          Established
    Neighbor address:
    Neighbor AS:      65001
    Neighbor ID:
    Neighbor caps:    AS4
    Session:          external AS4
    Source address:
    Hold timer:       155/180
    Keepalive timer:  51/60


adding a route is as simple as doing a simple curl on the host on which the ExaBGP is running

nettinkerer@docker-1:~$ curl

ExaBGP gets the announce message

Mon, 29 Aug 2016 21:08:18 | INFO     | 15231  | processes     | Command from process add-routes : announce route next-hop self
Mon, 29 Aug 2016 21:08:18 | INFO     | 15231  | reactor       | Route added to neighbor local-ip local-as 65001 peer-as 65000 router-id family-allowed in-open : next-hop
Mon, 29 Aug 2016 21:08:18 | INFO     | 15231  | reactor       | Performing dynamic route update
Mon, 29 Aug 2016 21:08:19 | INFO     | 15231  | reactor       | Updated peers dynamic routes successfully

the bgp daemon on the firewall also knows the route

bird> show route all         via on ens9 [bgp3 15:08:36] * (100) [AS65001i]
        Type: BGP unicast univ
        BGP.origin: IGP
        BGP.as_path: 65001
        BGP.local_pref: 100

the REST API also accepts communities and meds

curl ""

which is shown by the bird daemon as well

bird> show route all         via on ens9 [bgp3 15:14:01] * (100) [AS65001i]
        Type: BGP unicast univ
        BGP.origin: IGP
        BGP.as_path: 65001
        BGP.next_hop: 200
        BGP.local_pref: 100 (100,400) (300,600)

Withdrawing routes can also be done easily with a curl statement

 curl ""

And the route is gone

bird> show route all
Network not in table

At the moment there is only limitted input validation. The REST API does check if the ip address entered is valid but no other checks are implemented at this moment. I might add this if need arises.

The script and configs used in this blog can be found on my Github


Using the Python UCS library

Recently some VCE vBlocks have been taken into production at my current job. Although VCE installs everything for you they didn’t configure all the required production Vlans. The vlans need to be added to various components in the vBlock

  • Nexus 9000
  • Nexus 1000V
  • UCS-FI

configuring them on the Nexus devices is pretty straight forward but configuring them on the FI as a chore for the operations team. First add the Vlan to the system and them add the VLAN to every vNIC template

As I am still trying to improve my Python skills I just wrote a script to add a vlan from the cli to do this for me.

It starts with downloading the the Python SDK from Cisco and install them on you management system. After installation you are good to go an you can start wrting your own scripts. The documentation provided is not very elaborate but sufficient for a script like this.

First some modules are to be loaded. Besides the ones required for the UCS related stuff I add a few to make the script “nice” argparse is a library to support command line options and getpass allows entering passwords without showing them on screen

from UcsSdk.MoMeta.FabricLanCloud import FabricLanCloud
from UcsSdk.MoMeta.FabricVlan import FabricVlan
from UcsSdk.MoMeta.VnicLanConnTempl import VnicLanConnTempl
from UcsSdk.MoMeta.VnicEtherIf import VnicEtherIf
from UcsSdk import *

import argparse
import os
import getpass

The argument parser is created.

parser=argparse.ArgumentParser(description="Command adds or removes a vlan to a FI and all VniC profiles present")
parser.add_argument("--fi", type=str, required=True, help="IP/hostname of FI")
group.add_argument("--add",action='store_true', help="vlan will be added")
group.add_argument("--del",action='store_true', help="vlan will be removed")
parser.add_argument("--id", type=int, required=True, help="vlan ID")
parser.add_argument("--name", type=str,  required=True, help="vlan Name")
userName=raw_input("Username: ")
print "Modify vlan %s with name %s on %s with user %s and pw ***" % (,,,userName)

This arguments parser adds a number of command line options

  • –fi the ip or hostname of the fabric interconnect
  • –add to add a vlan
  • –del to remove a vlan
  • –id the vlan id (the number)
  • –name the vlan name

When one of the options is missing an error is raised and some help tekst is provided. Argparser also prevents you from providing both add an del together.

Line 9 and 10 prompts for the username andpassword. Getpass prevents the password to be echoed on screen.

#login to the UCS FI 
  handle = UcsHandle()
  #get the MO for every vnic
    #get the MO for the LANCLOUD
    LanCloud= handle.GetManagedObject(None, FabricLanCloud.ClassId())
    vlanExist=handle.GetManagedObject(LanCloud,FabricVlan.ClassId(), {FabricVlan.NAME:vlanName})

line 1 and 2 store the entered values for the vlan ID and vlan name in a more recognizable variable name.  A try expect structure is started and an handle to the UCS is created. All actions on the UCS will be done via this handle. The first thing to do now is do a login with the supplied credentials and ip address or hostname of the FI.

line 9 retrieves every vnic template on the system. This is simply done by retrieving all objects of the the class “vnicLanConnTempl” this string is the ouput of VnicLanConnTempl.ClassId(). The hardest part of writing scripts for UCS is determining the required ClassId. The easiest way to do this in my opinion is to dump the XML from the UCSM gui and find the required classes. Open the UCS GUI and select the object you want some info about. Press the right button and select Copy XML

Copy XML

The XML for this object is placed on the clipboard.

<vnicLanConnTempl childAction="deleteNonPresent" descr="" dn="org-root/lan-conn-templ-ESX_001_Prod2" identPoolName="" intId="118443" mtu="1500" name="ESX_001_Prod2" nwCtrlPolicyName="" operIdentPoolName="" operNwCtrlPolicyName="org-root/nwctrl-default" operQosPolicyName="" operStatsPolicyName="org-root/thr-policy-default" pinToGroupName="" policyLevel="0" policyOwner="local" qosPolicyName="" statsPolicyName="default" switchId="A" target="adaptor" templType="updating-template"> 
<vnicEtherIf addr="derived" childAction="deleteNonPresent" configQualifier="" defaultNet="no" fltAggr="0" name="Vlan1246" operState="indeterminate" operVnetDn="" operVnetName="" owner="logical" rn="if-Vlan1246" switchId="A" type="ether" vnet="1"/>
<vnicEtherIf addr="derived" childAction="deleteNonPresent" configQualifier="" defaultNet="no" fltAggr="0" name="Vlan3002" operState="indeterminate" operVnetDn="" operVnetName="" owner="logical" rn="if-Vlan3002" switchId="A" type="ether" vnet="1"/> 
<vnicEtherIf addr="derived" childAction="deleteNonPresent" configQualifier="" defaultNet="no" fltAggr="0" name="Vlan1300" operState="indeterminate" operVnetDn="" operVnetName="" owner="logical" rn="if-Vlan1300" switchId="A" type="ether" vnet="1"/>
<vnicEtherIf addr="derived" childAction="deleteNonPresent" configQualifier="" defaultNet="no" fltAggr="0" name="Vlan3000" operState="indeterminate" operVnetDn="" operVnetName="" owner="logical" rn="if-Vlan3000" switchId="A" type="ether" vnet="1"/> 
<vnicEtherIf addr="derived" childAction="deleteNonPresent" configQualifier="" defaultNet="no" fltAggr="0" name="Vlan124" operState="indeterminate" operVnetDn="" operVnetName="" owner="logical" rn="if-Vlan124" switchId="A" type="ether" vnet="1"/>
 <vnicEtherIf addr="derived" childAction="deleteNonPresent" configQualifier="" defaultNet="no" fltAggr="0" name="Vlan123" operState="indeterminate" operVnetDn="" operVnetName="" owner="logical" rn="if-Vlan123" switchId="A" type="ether" vnet="1"/> 

This is a lot of informartion but the most important part is vnicLanConTempl the ClassId of this object. It is also obvious that the children of vnicLanConnTempl are the vlans which are allowed on this. So we already know that objects of ClassId vnicEtherI needs to be added if we want to modify the allowed vlans.

Line 11 retrieves the LanCloud. Under the LanCloud all objects related to L2 are stored. In line 12 the Lancloud is used as a starting point for a search for the vlan with the name which needs to be added. If it is present it should not be added or deleted later on in the script.

if (args.add):
    #add vlan
      print "add vlan %s to lanCloud and vNics" % (vlanName)
      if vlanExist:
        print vlanName + ": Already defined"
        #add the vlan to the LANCLOUD
          handle.AddManagedObject(LanCloud,FabricVlan.ClassId(), {FabricVlan.NAME:vlanName,FabricVlan.ID:vlanId})
          #add the vlan to all nics
            for vnic in vnics:
              vlanDn="%s/if-%s" % (vnic.Dn,vlanName)
              handle.AddManagedObject(vnic,VnicEtherIf.ClassId(), {VnicEtherIf.DN:vlanDn,VnicEtherIf.NAME:vlanName,VnicEtherIf.DEFAULT_NET:"no"},True,YesOrNo.FALSE)
          except Exception, err:
              print "Exception:", str (err)
        except Exception, err:
          print "Exception:", str (err)

This part of the script handles the adding of a vlan to the UCS. Line 4 and 5 check if the vlan already exists. When this is true the scripts logs a messages and continues with a logout. If the vlan is not found another try except structure is created. On line 9 the second UCS API command in the script, AddManagedObject, is used. This command is adds an object below another object. In this case we are adding a vlan below the LanCloud. The parameters used to create the vlan are the name and the id.

When the addition of the vlan  is successful another try expect is started. This one is to add the vlan to the vNics obtained earlier. For some reason the Dn of the new VnicEtherIf needs to be supplied as one of the parameters. I have not been able to find a list of required parameters of the various ClassIds.

The format of the Dn was again obtained by using the XML retrieved from the GUI. One important thing to notice is the True value in the AddManagedObject. This prevents the API to raise an error if the vlan is already part of the allowed vlans on the vNic.

The last line close the various try statements.

      print "del vlan %s from lanCloud and vNics" % (vlanName)
      #remove vlan from vnics
      if vnicEtherIfMOS:
      #remove vlan from LanCloud
      if vlanExist:
   #delete vlan
  #logout from the UCS FI
  except Exception, err:
    print "Exception:", str (err)

except Exception, err:
  print "Exception:", str (err)

The final section of the script handles the removal of the Vlan from the vNics and the vlan from the LanCloud. Line 4 searches for all VnicEtherIf with the name of the Vlan which needs to be removed. The base for this search is are the vnics obtained earlier. Line 5-7 removes all these VnicEtherIfs in one operation, but only if there is at least one Vnic. Line 9 and 10 do the same for the vlan.

The last lines closes the try, except and does a logout from the script.

Seeing the script in action

root@python-dev:~/ucs# python --fi --add --id 123 --name test123
Username: admin
Modify vlan 123 with name test123 on with user admin and pw ***
add vlan test123 to lanCloud and vNics

Best way is to keep the UCS GUI open while executing the script so you can see the vlans appear magically when executing this simple script.

Finding smallest subnet for two host

@netmanchris asked for a method to determine the smallest common subnet for two hosts. Below is my solution based on the python netaddr library

from netaddr import * 
for net in superNets: 
  if highIP in net: 

On line 2 and 3 the ip are used to create to an IPNetwork. On line 4 a list is created containing al supernets of the IPNetwork. As the list is from large to small it needs te be reserved. By looping over eacht of the supernets and checking of the second ip is part of the subnet the common subnet is determined.

The Python netaddr is very versatile and can help you with various tedious ip operations