# Installation Notes: # - This RCF utilizes QoS and requires TCAM re-configuration, requiring RCF # to be loaded twice with the Cluster Switch rebooted in between. # # - Perform the following 4 steps to ensure proper RCF installation: # # (1) Apply RCF first time, expect following messages: # - Please save config and reload the system... # - ERROR: Failed to write VSH commands # - TCAM region is not configured... # # (2) Save running-configuration and reboot Cluster Switch # # (3) After reboot, apply same RCF second time and expect following messages: # - % Invalid command at '^' marker # # (4) Save running-configuration again # # - If running NX-OS versions 9.3(5) 9.3(6), 9.3(7), or 9.3(8) # - Downgrade the NX-OS firmware to version 9.3(5) or earlier if # NX-OS using a version later than 9.3(5). # - Do not upgrade NX-OS prior to applying v1.9 RCF file. # - After the RCF is applied and switch rebooted, then proceed to upgrade # NX-OS to version 9.3(5) or later. # # - If running 9.3(9) 10.2(2) or later the RCF can be applied to the switch # after the upgrade. banner motd # ****************************************************************************** * NetApp Reference Configuration File (RCF) * * Switch : Nexus N9K-3132Q-V * Filename : Nexus-3132QV-RCF-v1.8a-Cluster-Breakout.txt * Date : 05-18-2022 * Version : v1.8a * * Port Usage: * Ports 1- 6: Breakout mode (4x10G) Intra-Cluster/HA Ports, int e1/1/1-4, * e1/2/1-4, e1/3/1-4,int e1/4/1-4, e1/5/1-4, e1/6/1-4 * Ports 7-30: 40GbE Intra-Cluster/HA Ports, int e1/7-30 * Ports 31-32: Intra-Cluster ISL Ports, int e1/31-32 * * Note: * - This RCF supports Clustering, HA, RDMA, and DCTCP using a single port profile. * ****************************************************************************** # interface breakout module 1 port 1-6 map 10g-4x feature lacp feature ssh feature sftp-server feature scp-server vlan 17,18 exit cdp enable cdp advertise v1 cdp timer 5 system default switchport no system default switchport shutdown snmp-server community cshm1! group network-operator errdisable recovery interval 30 port-channel load-balance src-dst ip-l4port-vlan ip domain-lookup logging console 1 hardware access-list tcam region vpc-convergence 256 hardware access-list tcam region racl 256 hardware access-list tcam region e-racl 256 hardware access-list tcam region qos 256 #********** PFC/QoS/ECN ********** class-map type qos match-any HA match dscp 40 match cos 5 class-map type qos match-any RDMA match dscp 16 match cos 2 class-map type qos match-any CT match dscp 0 match cos 0 policy-map type qos ROCE_QOS class type qos HA set qos-group 5 class type qos RDMA set qos-group 2 class type qos class-default set qos-group 0 class-map type network-qos HA_NETQOS match qos-group 5 class-map type network-qos RDMA_NETQOS match qos-group 2 class-map type network-qos CT_NETQOS match qos-group 0 policy-map type queuing ROCE_QUEUING class type queuing c-out-8q-q7 priority level 1 class type queuing c-out-8q-q6 bandwidth remaining percent 0 class type queuing c-out-8q-q5 bandwidth remaining percent 43 random-detect threshold burst-optimized ecn class type queuing c-out-8q-q4 bandwidth remaining percent 0 class type queuing c-out-8q-q3 bandwidth remaining percent 0 class type queuing c-out-8q-q2 random-detect threshold burst-optimized ecn bandwidth remaining percent 28 class type queuing c-out-8q-q1 bandwidth remaining percent 0 class type queuing c-out-8q-q-default bandwidth remaining percent 28 random-detect threshold burst-optimized ecn policy-map type network-qos NETQOS class type network-qos HA_NETQOS pause pfc-cos 5 mtu 9216 class type network-qos RDMA_NETQOS pause pfc-cos 2 mtu 9216 class type network-qos CT_NETQOS mtu 9216 system qos service-policy type network-qos NETQOS service-policy type queuing output ROCE_QUEUING copp profile strict #********** Port Profiles ********** port-profile type ethernet CLUSTER description 10/40GbE Intra-Cluster/HA Nodes switchport mode trunk switchport trunk allowed vlan 1,17-18 spanning-tree port type edge trunk spanning-tree bpduguard enable mtu 9216 state enabled exit #********** Interfaces ********** interface e1/1/1-4, e1/2/1-4, e1/3/1-4, e1/4/1-4, e1/5/1-4, e1/6/1-4, e1/7-30 inherit port-profile CLUSTER priority-flow-control mode auto service-policy input ROCE_QOS exit #********** Intra-Cluster ISL ports ********** interface Ethernet1/31 description Intra-Cluster Switch ISL Port 1/31 (port channel) priority-flow-control mode auto switchport mode trunk switchport trunk allowed vlan 1 mtu 9216 channel-group 1 mode active no shutdown interface Ethernet1/32 description Intra-Cluster Switch ISL Port 1/32 (port channel) priority-flow-control mode auto switchport mode trunk switchport trunk allowed vlan 1 mtu 9216 channel-group 1 mode active no shutdown interface port-channel1 service-policy input ROCE_QOS priority-flow-control mode auto switchport mode trunk switchport trunk allowed vlan 1 mtu 9216 no shutdown end