In UCS B environment, many components are described as many different type of virtual component such as vnic, vnmic,vif, vethernet, virtual circuit, border port, uplink port, server port, virtual cable, physical cable. It is not surprising that there can be confusion about what path packets are actually taking through the UCS infrastructure.
However, knowing the full data path through the UCS infrastructure is very critical to understand where to troubleshoot.
UCS B Infrastructure VIF Path Physical and Logical Architecture
Green command output
At FI connect to nxos
- Border interface will be connecting to uplink switch and we can see that Po2 is the Border Interface and connecting to uplink switch.
- Run "show port-channel summary" cmd to see the member port. We can see the member port is Eth1/1 and Eth1/2.
- Run "show cdp neighbour interface eth1/1" and "show cdp neighbour interface eth1/2" and we can understand the Eth1/1 is connecting to uplink switch Nexus5K-1 and the Eth1/2 connecting to Nexus5k-2.
Orange command output
Return to FI console from NXOS and run " show service-profile circuit server 1/1". This cmd will give you "virtual circuit" information. Next key concept to understand is that whenever vNIC on a Cisco CNA like the Virtual Interface Card (VIC), this automatically creates the corresponding virtual ethernet port on the fabric interconnects (On both FI's if fabric failover is enabled) and connects the vethernet to the vNIC with a virtual cable as shown above, this creates a Virtual Network Link (VN-Link).
Here is the cmd output
Go to UCSM > Servers > select blade "Service Profiles" > VIF Paths
+ So "Virtual Circuit 1740" is created between "vnic0" and "vif 1740". This VIF 1740 shows as interface vethernet 1740 in FI NXOS.
Return to NXOS and run "show run interface vethernet 1740". Now we see that Vethernet 1740 is referenced to server 1/1 and configured as trunk port. What vlan is allowed through. More importantly this Vethernet 1740 is bound to Ethernet1/1/1. The first 1 refer to chassis number. Ignore second 1 and the last 1 is server number.
Explorer more about the Ethernet 1/1/1 by "show run interface ethernet1/1/1"
- This output tells that if traffic leaves this interface E1/1/1, we will do "vntag" and will send out to fabric-interface Eth1/17. This Eth1/17 is the "Server port" you defines in UCSM FI.
The "Server Port" must show "switchport mode" as "fex-fabric". It also tells this server port is associated with FEX (IOM).
Also run "show pinning server-interface". This output may confuse you but it shows virtual interface as well as physical interface that faces the server.
Yellow command output
Next cmd you will use is "show fed detail".
- Fex Port is basically server. This Fex port is connected to Fabric port in Fabric Interconnect.
- Next cmd you need to understand is "show interface fex-fabric" cmd.
+ This cmd tells you FI server port (fabric port) is physically connected to IOM port. So Eth1/17 is connected to Fex (IOM) port 1, Eth1/18 is port 2, Eth1/19 is 3 and Eth1/20 is 4 accordingly.
Blue command output
- Go to UCSM and browse to DCE interface and check the mac address of DCE 1 interface. We can confirm DCE interface is Ethernet1/1/1.
- Returm to FI NXOS and check mac address with "show interface e1/1/1" cmd and "show mac address | inc 9a" cmd
At Esxi Server
ssh into ESXi host running in blade 1/1 and run "esxcfg-vnic -l" cmd. You can see 6 vNICs. In ESXi hypervisor, vNIC is called as "VMNIC".
return to NXOS and you can confirm that mac address of vmnic1 belongs to the Veth1740.
