Firewall Ports for Viptela Deployments
This article describes which ports Viptela devices use. If your network has firewall devices, you must open these ports on the firewalls so that devices in the Viptela overlay network can exchange traffic.
Viptela-Specific Port Terminology
By default, all Viptela devices use base port 12346 for establishing the connections that handle control and traffic in the overlay network. Each device uses this port when establishing connections with other Viptela devices.
Port Offset
When multiple Viptela devices are installed behind a single NAT device, you can configure different port numbers for each device so that the NAT can properly identify each individual device. You do this by configuring a port offset from the base port 12346. For example, if you configure a device with a port offset of 1, that device uses port 12347. The port offset can be a value from 0 through 19. The default port offset is 0.
For NAT devices that can differentiate among the devices behind the NAT, you do not need to configure the port offset.
Port Hopping
In the context of a Viptela overlay network, port hopping is the process by which devices try different ports when attempting to establish connections with each other, in the event that a connection attempt on the first port fails. After such a failure, the port value is incremented and the connection attempt is retried. The software rotates though a total of five base ports, waiting longer and longer between each connection attempt.
If you have not configured a port offset, the default base port is 12346, and port hopping is done sequentially among ports 12346, 12366, 12386, 12406, and 12426, and then returning to port 12346.
If you have configured a port offset, that initial port value is used and the next port is incremented by 20. For example, for a port configured with an offset of 2, port hopping is done sequentially among ports 12348, 12368, 12388, 12408, and 12428, and then returning to port 12348.
Incrementing the ports by 20 ensures that there is never any overlap among the possible base port numbers.
vEdge routers use port hopping when attempting to establish connections to vManage NMS, vBond orchestrators, and vSmart controllers. You can also manually request a vEdge router to port-hop.
vSmart controllers and vManage NMSs are normally installed behind a properly behaving NAT device, so port hopping is generally not needed and generally does not occur on these devices.
vBond orchestrators always connect to other Viptela devices using port 12346. They never use port hopping.
To describe how port hopping works, we use as an example a vEdge router with the default base port of 12346. When a router has attempted to connect to another Viptela device but the connection does not succeed within a certain time, the router hops to the next base port and tries establishing the connection on that port.
If the first connection attempt on the initial base port does not succeed after about 1 minute, the router hops to port 12366. After about 2 minutes, it hops to port 12386; after about 5 minutes, it hops to port 12406; and after about 6 minutes, it hops to port 12426. Then the cycle returns to initial port, 12346.
With a full-cone NAT device, the source ports for all connections initiated by a given vEdge router remain consistent across all sessions initiated by the vEdge router. For example, if the router initiates a session with public source port 12346, this is the port used for all communication.
Effects of Port Hopping
Viptela devices use port hopping to make every attempt to keep the control plane of the overlay network up and operational. If a controller device—a vBond orchestrator, vManage NMS, or vSmart controller—goes down for any reason and the vEdge routers remain up, when the controller device comes back up, the connection between it and the vEdge router might shut down and restart, and in some cases the BFD sessions on the vEdge router might shut down and restart. This behavior occurs because of port hopping: When one device loses its control connection to another device, it port hops to another port in an attempt to reestablish the connection.
Two examples illustrate when this might occur:
- When a vBond orchestrator crashes, the vManage NMS might take down all connections to the vEdge routers. The sequence of events that occurs is as follows: When the vBond orchestrator crashes, the vManage NMS might lose or close all its control connections. The vManage NMS then port hops, to try to establish connections to the vSmart controllers on a different port. This port hopping on the vManage NMS shuts down and then restarts all its control connections, including those to the vEdge routers.
- All control sessions on all vSmart controllers go down, and BFD sessions on the vEdge routers remain up. When any one of the vSmart controllers comes back up, the BFD sessions on the routers go down and then come back up because the vEdge routers have already port hopped to a different port in an attempt to reconnect to the vSmart controllers.
Ports Used by vEdge Routers
When a vEdge router joins the overlay network, it establishes DTLS control plane connections with the controller devices—the vBond orchestrator, the vManage NMS, and the vSmart controller. The router uses these control connections to learn the location of the vSmart controller from the vBond orchestrator, to receive its configuration from the vManage NMS, and to receive its policy and any policy updates from the vSmart controller. When initially establishing these DTLS connections, the vEdge router uses the base port 12346. If it is unable to establish a connection using this base port, it port-hops through ports 12366, 12386, 12406, and 12426, returning, if necessary, to 12346, until it successfully establishes the DTLS connections with the three controller devices. This same port number is used to establish the IPsec connections and BFD sessions to the other vEdge routers in the overlay network. Note that if the vEdge configuration includes a port offset, the base port number and the four sequential port numbers are incremented by the configured offset.
To see which port DTLS and BFD are using for the control and data connections, look at the Private Port column in the output of the show control local-properties command. The command output also shows the public port number that the interface is using. If the vEdge router's WAN port is not connected to a NAT device, the private and public port numbers are the same. If a NAT device is present, the port number listed in the Public Port column is the one being used by the NAT device, and it is the port that BFD is using. This public port number is the one remote vEdge routers use to send traffic to the local site.
In a network with firewall devices, you must open the Viptela base ports on the firewall devices to allow traffic to flow across the overlay network. You open all the base ports that the vEdge routers in the network might use, which are the default base ports and the four base ports that the router can port-hop among.
(Note that port hopping is generally not needed on vSmart controllers and vManage NMSs.)
For vEdge routers configured to use TLS tunnels, which use TCP, the routers select a random TCP port, so you must configure proper NAT entries for vManage NMS and vSmart controllers to be able to communicate with vEdge routers.
For vEdge routers configured to use DTLS tunnels, which use UDP, at a minimum you must open the five base ports that are used by a vEdge router with a default port offset of 0. Specifically, you open:
- Port 12346
- Port 12366
- Port 12386
- Port 12406
- Port 12426
If you have configured a port offset value on any of the Viptela devices, you also need to open the ports configured with the port offset value:
- Port (12346 + port offset value)
- Port (12366 + port offset value)
- Port (12386 + port offset value)
- Port (12406 + port offset value)
- Port (12426 + port offset value)
Ports Used by Viptela Devices Running Multiple vCPUs
The vManage NMSs and vSmart controllers can run on a virtual machine (VM) with up to eight virtual CPUs (vCPUs). The vCPUs are designated as Core0 through Core7.
Each core is allocated separate base ports for control connections. The base ports differ, depending on whether the connection is over a DTLS tunnel (which uses UDP) or a TLS tunnel (which uses TCP).
Note: vBond orchestrators do not support multiple cores. vBond orchestrators always use DTLS tunnels to establish control connections with other Viptela devices, so they always use UDP. The UDP port is 12346.
The following table lists the port used by each vCPU core for the vManage NMS. Each port is incremented by the configured port offset, if offset is configured.
| Core Number | Ports for DTLS (UDP) | Ports for TLS (TCP) |
|---|---|---|
| Core0 | 12346 | 23456 |
| Core1 | 12446 | 23556 |
| Core2 | 12546 | 23656 |
| Core3 | 12646 | 23756 |
| Core4 | 12746 | 23856 |
| Core5 | 12846 | 23956 |
| Core6 | 12946 | 24056 |
| Core7 | 13046 | 24156 |
Note that you can configure the TLS connection port number with the security control tls-port configuration command. If you do configure the TLS port number, Core0 uses the configured port number and each subsequent core increments the port number by 100.
Administrative Ports Used by vManage NMS
vManage NMS uses the following administrative ports for protocol-specific communication:
| Purpose | Traffic Direction | Protocol | Port Number |
|---|---|---|---|
| Netconf | Bidirectional | TCP | 830 |
| HTTPS | Incoming | TCP | 443 |
| SNMP query | Incoming | UDP | 161 |
| SSH | Incoming | TCP | 22 |
| RADIUS | Outgoing | UDP | 1812 |
| SNMP trap | Outgoing | UDP | 162 |
| Syslog | Outgoing | UDP | 514 |
| TACACS | Outgoing | TCP | 49 |
vManage clusters use the following ports for communication among the NMSs that comprise the cluster:
| vManage Service | Traffic Direction | Protocol | Port Numbers |
|---|---|---|---|
| Application server | Bidirectional | TCP | 80, 443, 7600, 8080, 8443, 57600 |
| Configuration database | Bidirectional | TCP | 2424, 2434 |
| Coordination server | Bidirectional | TCP | 2181, 3888 |
| Message bus | Bidirectional | TCP | 9092 |
| Statistics database | Bidirectional | TCP | 9200, 9300 |
| Tracking of device configurations (NCS and Netconf) | Bidirectional | TCP | 830 |
Configure the Port Offset
When two or more Viptela devices are behind the same full-cone NAT device, one device can use the default port offset, and you should configure a port offset on the remaining devices:
Viptela(config)# system port-offset number
The port offset can be a value from 0 through 19. The default port offset is 0.
In the following example, vEdge-1 uses the default port offset of 0, and on vEdge-2 the port offset is set to 1.
In this example:
- vEdge-1 attempts to connect first using base port 12346. If that attempt is not successful, the router attempts port 12366, 12386, 12406 and 12426.
- vEdge-2 has a port offset of 1, so the first port it attempts to connect on is 12347 (12346 plus offset of 1). If it fails to connect using port 12347, the router hops by increments of 20 and attempts to connect on ports 12367, 12387, 12407, and 12427.
Perform Port Hopping Manually
You can manually request a vEdge router to port-hop:
vEdge# request port-hop
One reason to use this command is if the router's control connections are up, but BFD is not starting. The request port-hop command restarts the control connections on the next port number, and BFD should then also start.
Additional Information
port-hop
port-offset
request port-hop
show control local-properties