ServiceNetworkManager maintains virtual service network elements and states in a local store. It also exposes REST APIs for creating, updating, getting, and removing service network elements, which can be found in VTN API. Virtual networks and ports consumed by OpenStack Nova are created via Neutron, which is set to use ML2 plugin and networking-onos mechanism driver. When VTN receives Neuton network state update, VTN coverts it to VTN service network to store them. XOS VTN synchronizer, on the other hand, directly calls VTN APIs to make changes on service network states. ServiceNetworkManager is responsible for notifying network state updates as well so that InstanceHandler and DependencyHandler can take actions to data plane.
NodeManager manages compute nodes and bootstraps them to be ready to program. It leverages OVSDB southbound and SSH exec to configure the OVS and the interfaces on compute nodes. The node bootstrap procedure includes the following steps.
- Connect to OVSDB server on the compute node
- Create an integration bridge, typically “br-int”, and set its controller
- Add VXLAN port and interface to br-int bridge
- Add data network interface to br-int
- Set local management network IP and data network IP address to “br-int” bridge
Addition or configuration of the compute node is done via network configuration.
DependencyHandler creates a network connectivity between two service networks, subscriber and provider. Basically, subscriber network instances have an indirect access to provider network. That is, a subscriber network instance is not allowed to access individual provider network instances directly with their IP but automatically selected instance with provider network’s service IP (subnet gateway IP). To enable direct access to a particular provider network instance, “bidirectional” flag should be set true for the provider network. DependencyHandler leverages OpenFlow select group to achieve load-balanced indirect access. Refer to this page(https://wiki.opencord.org/display/CORD/Service+Composition+and+the+Role+of+VTN) for more about interconnecting networks. Creating or removing a dependency is done by updating “providers” on a subscriber service network via service network API.
InstanceHandler creates a network connectivity for an instance. What kind of network connectivity is decided by what type of service network is attached to the instance. Here are the list of supported service network types.
PRIVATE: Isolated tenant network
PUBLIC: Provider network that offers connectivity to external network via L3. This network relies on the underlay network infrastructure or vRouter for gateway and first hop routing service.
MANAGEMENT_LOCAL: Virtual machine management network that offers limited connectivity from the virtual instance's host machine. This network does not span compute nodes, and cannot be part of service chain.
MANAGEMENT_HOST: Virtual instance management network that offers connectivity to head node. This network runs over the physical management network, and cannot be part of service chain.
- ACCESS_AGENT: special network for R-CORD access agent instance
(deprecated) VSG: special network for a vSG instances
When InstanceHandler detects a new port on the OVS switch, it searches a service port mapped to the newly added port with the port name on OVS, and then installs a set of flow rules based on the service network and port information. It also adds a host to ONOS for the port with the MAC and IP address of the service port to easily handle ARP and DHCP requests from the instance.
VTN proxies ARP requests from instances. Basically, it responses or forwards the request to appropriate port only if the request is for a known virtual instances or a service network gateway IP. Otherwise, it drops the request. Note that as PUBLIC network relies on vRouter or underlay network infrastructure for the gateway and first hop routing, VTN needs to reply with real MAC for the gateway ARP request so that VM sets correct destination MAC for outbound packets, or underlay infrastructure discards the packets. We put this information via publicGateways field of network configuration.
VTN proxies DHCP requests as well. It responses only if the request comes from a host in ONOS system and replies with the host's IP address.
Ingress Table (id = 0)
Additional rules for MANAGEMENT_LOCAL network
Additional rules for VXLAN
Input Port Table (id = 1)Checks if the packet is received from local ports or the tunnel port.
if in_port is tunnel port, take the packet through VNI table
if in_port is data port, take the packet though dstIP table
if in_port is local port and src_ip is a service instance, take the packet through access table
if in_port is local port and src_ip is not a service instance, take the packet through service chaining table
Access Table (id = 2)Matches source and destination IP range and see if the packet is allowed or not. Allowed packets are sent to dstIP table or group table according to the destination. By default, instances in the same network are allowed to talk to each other. For instances in different service networks, service dependency is required for one service instance to send a packet to another service.
(higher priority) if dst_ip is a service IP (normally gateway IP of the service network) and src and dst networks are in a dependency relationship, take the packet to the group of the dst network service
(medium priority) if src and dst networks are in a dependency relationship, take the packet through dstIP table
(lower priority) if dst_ip is not in any of service network IP range (not external access), drop
(table miss) take the packet through data interface connecting to the underlay fabric
Service Chaining Table (id = 3)
DstIP Table(id = 4)Matches destination IP and forward the packet to local instance port or vxlan port based on the destination instance location.
if dst_ip is the local instance IP, set dst_mac to the destination instance MAC and send to the port where the destination instance connected
if dst_ip is the remote instance IP, set dst_mac to the destination instance MAC, tun_id to the destination service network VNI, tun_dst to the remove compute node data network IP, and send the packet to the vxlan port
VNI Table (id = 5)Matches VNI and destination MAC, and take the packet though the correct local instance port.
VLAN Table (id = 6)Matches input port and VLAN ID, and take the packet through the local instance port or the underlay fabric based on the input port.
if in_port is the data interface, take the packet to the correct local instance port by matching vlan_id with in_port
if in_port is the local instance port and the vlan_id is something allowed to the instance, take the packet through the data plane interface
Note that VLAN ID 500 is reserved for vSG WAN.
Example flow rules
In the example, 10.0.5.0/24 is a subscriber service of the service network 10.0.6.0/24, and bidirectional access is enabled. Service instance of the 10.0.5.0/24 network with IP address 10.0.5.2 is in the local port 11, and the other service network instance with IP address 10.0.6.2/24 is in the remote node.