1.背景: 双活存储VMAX200k,承载主备中心数据库 光纤交换机F96(fabric 扩展license) 华为波分DWDM OptiX OSN 8800,两对波分和SAN交换机由三家运营商的两条裸光纤组成,一条为移动,另一条为电信联通冗余;
2.在SAN交换机中做了virtual fabric,虚拟机交换机中只有级联和存储SRDF端口,速率为8Gbps
3.级联端口配置:
portshow输出
portIndex: 94
portName: port94
portHealth: No Fabric Watch License
Authentication: None
portDisableReason: None
portCFlags: 0x1
portFlags: 0x10004907 PRESENT ACTIVE E_PORT G_PORT U_PORT LOGICAL_ONLINE LOGIN LED
LocalSwcFlags: 0x0
portType: 24.0
POD Port: Port is licensed
portState: 1 Online
Protocol: FC
portPhys: 6 In_Sync portScn: 16 E_Port
port generation number: 12
state transition count: 9
portId: 420200
portIfId: 43021034
portWwn: 20:5e:c4:f5:7c:d4:df:5d
portWwn of device(s) connected:
Distance: static (desired distance = 40 Km)
portSpeed: N8Gbps
FEC: Inactive
Credit Recovery: Active
LE domain: 0
FC Fastwrite: OFF
Interrupts: 0 Link_failure: 2 Frjt: 0
Unknown: 0 Loss_of_sync: 0 Fbsy: 0
Lli: 2551 Loss_of_sig: 2
Proc_rqrd: 7881 Protocol_err: 0
Timed_out: 0 Invalid_word: 360335
Rx_flushed: 0 Invalid_crc: 1860
Tx_unavail: 0 Delim_err: 1
Free_buffer: 0 Address_err: 0
Overrun: 0 Lr_in: 2
Suspended: 0 Lr_out: 423
Parity_err: 0 Ols_in: 2
2_parity_err: 0 Ols_out: 2
CMI_bus_err: 0
--------------------portcfgshow------------
Area Number: 2
Octet Speed Combo: 1(16G|8G|4G|2G)
Speed Level: AUTO(SW)
AL_PA Offset 13: OFF
Trunk Port ON
Long Distance LS
VC Link Init ON
Desired Distance 40 Km
Reserved Buffers 326
Locked L_Port OFF
Locked G_Port OFF
Disabled E_Port OFF
Locked E_Port OFF
ISL R_RDY Mode OFF
RSCN Suppressed OFF
Persistent Disable OFF
LOS TOV enable OFF
NPIV capability ON
QOS Port OFF
Port Auto Disable: OFF
Rate Limit OFF
EX Port OFF
Mirror Port OFF
SIM Port OFF
Credit Recovery ON
F_Port Buffers OFF
E_Port Credits OFF
Fault Delay: 0(R_A_TOV)
NPIV PP Limit: 126
CSCTL mode: OFF
D-Port mode: OFF
D-Port over DWDM: OFF
Compression: OFF
Encryption: OFF
FEC: ON
4.其他配置是如何设置的?
1)LOS TOV enable: 建议设置为ON,FC交换适当开启告警防抖功能。
2)2、Fillword:对于FC8G及以上速率,Fillword建议设置为ARB(FF),并使用mode3( ARB/ARB then IDLE/ARB );对于FC4G/2G/1G,fillword设置为Idle即可,使用Mode 0(IDLE/IDLE)
3)Trunk Port:建议设置为OFF。否则可能因为时延差问题导致TRUNK失效
4)ISL R_RDY Mode
5)Credit Recovery
6)Port Fencing设置和策略
5.现有SAN交换机经常出现告警:
2020/08/04-03:00:19, [C3-1023], 146776, CHASSIS, WARNING, IBM_2498_F96, Single RDY/Frame Loss detected and recovered on Slot 0,Port 94(180) rdy(0x0)/frame(0x8).
已经寻求厂商给出解决方案,建议找波分厂商给建议,波分厂商建议更换光模块,但对底层的运营商抖动无法察觉。在应急处置上,请问有没有什么好的办法?