存储链路和TCP链路闪断测试

GPFS复制类型	磁盘超时属性	交易影响时间
存储波分链路闪断	rw_timeout=40	44秒
存储波分链路闪断	rw_timeout=30	30秒
私网网卡断4秒	N/A	5秒
网络波分链路断	N/A	4秒

NSD网络写可能的缺点

• 网络TCP发包在网络链路里面经过HASH算法固定走一条链路，没有负载均衡.
• 固定链路闪断，网络如果没有断链路的策略，会导致TCP不停切换回原链路，影响比较久
• 延时稍微比走存储链路高一点点，没有明显影响
• 网络断开会导致DISK下线，不像走存储链路，有双重保护（存储断，走网络）
• 私网没有设置etherchannel检测网关，影响未知

修改传输类型

AGDPCMB2:
mmchdisk chgm_db2data stop -d "gpfs3nsd;gpfs4nsd;gpfs5nsd;gpfs10nsd;gpfs12nsd"
AGDPCMB1:AGDPCMB2:
rmdev -l hdisk11;rmdev -l hdisk12;rmdev -l hdisk13;rmdev -l hdisk14;rmdev -l hdisk15
AGDPCMB2:
mmchdisk chgm_db2data start -a

文件系统512GB， TPS影响时间不到3分钟

AGDPCMB2:
mmchdisk chgm_db2data stop -d "gpfs3nsd;gpfs4nsd;gpfs5nsd;gpfs10nsd;gpfs12nsd"
AGDPCMB1:AGDPCMB2:
rmdev -l hdisk11;rmdev -l hdisk12;rmdev -l hdisk13;rmdev -l hdisk14;rmdev -l hdisk15
AGDPCMB2:
mmchdisk chgm_db2data start -a

---

BGDPCMB1:
mmchdisk chgm_db2data stop -d "gpfs6nsd;gpfs7nsd;gpfs8nsd;gpfs9nsd;gpfs11nsd“
BGDPCMB1:BGDPCMB2:
rmdev -l hdisk3;rmdev -l hdisk4;rmdev -l hdisk5;rmdev -l hdisk6;rmdev -l hdisk7

AGDPCMB2:
mmchdisk chgm_db2data start -a

AGDPCMB1:/#mmchdisk chgm_db2data start -a
mmnsddiscover: Attempting to rediscover the disks. This may take a while ...
pure2: failure - examine the system log on the remotehost for additional information
AGDPCMB1: failure - examine the system log on the remotehost for additional information
BGDPCMB1: failure - examine the system log on the remotehost for additional information
AGDPCMB2: failure - examine the system log on the remotehost for additional information
AGDPCCF1: failure - examine the system log on the remotehost for additional information
BGDPCMB2: failure - examine the system log on the remotehost for additional information
BGDPCCF1: failure - examine the system log on the remotehost for additional information
mmnsddiscover: Finished.
pure2: Rediscovered nsd server access to gpfs1002nsd.
AGDPCMB1: Rediscovered nsd server access to gpfs6nsd.
AGDPCMB1: Rediscovered nsd server access to gpfs7nsd.
AGDPCMB1: Rediscovered nsd server access to gpfs8nsd.
AGDPCMB1: Rediscovered nsd server access to gpfs9nsd.
AGDPCMB1: Rediscovered nsd server access to gpfs11nsd.
BGDPCMB1: Rediscovered nsd server access to gpfs3nsd.
BGDPCMB1: Rediscovered nsd server access to gpfs4nsd.
BGDPCMB1: Rediscovered nsd server access to gpfs5nsd.
BGDPCMB1: Rediscovered nsd server access to gpfs10nsd.
BGDPCMB1: Rediscovered nsd server access to gpfs12nsd.
AGDPCMB2: Rediscovered nsd server access to gpfs6nsd.
AGDPCCF1: Rediscovered nsd server access to gpfs6nsd.
AGDPCMB2: Rediscovered nsd server access to gpfs7nsd.
AGDPCMB2: Rediscovered nsd server access to gpfs8nsd.
AGDPCMB2: Rediscovered nsd server access to gpfs9nsd.
AGDPCMB2: Rediscovered nsd server access to gpfs11nsd.
BGDPCMB2: Rediscovered nsd server access to gpfs3nsd.
BGDPCCF1: Rediscovered nsd server access to gpfs3nsd.
BGDPCMB2: Rediscovered nsd server access to gpfs4nsd.
BGDPCMB2: Rediscovered nsd server access to gpfs5nsd.
BGDPCMB2: Rediscovered nsd server access to gpfs10nsd.
BGDPCMB2: Rediscovered nsd server access to gpfs12nsd.
AGDPCCF1: Rediscovered nsd server access to gpfs7nsd.
AGDPCCF1: Rediscovered nsd server access to gpfs8nsd.
AGDPCCF1: Rediscovered nsd server access to gpfs9nsd.
AGDPCCF1: Rediscovered nsd server access to gpfs11nsd.
BGDPCCF1: Rediscovered nsd server access to gpfs4nsd.
BGDPCCF1: Rediscovered nsd server access to gpfs5nsd.
BGDPCCF1: Rediscovered nsd server access to gpfs10nsd.
BGDPCCF1: Rediscovered nsd server access to gpfs12nsd.
GPFS: 6027-589 Scanning file system metadata, phase 1 ...
15 % complete on Wed Jan 4 15:52:33 2017
31 % complete on Wed Jan 4 15:52:43 2017
100 % complete on Wed Jan 4 15:52:45 2017
GPFS: 6027-552 Scan completed successfully.
GPFS: 6027-589 Scanning file system metadata, phase 2 ...
GPFS: 6027-552 Scan completed successfully.
GPFS: 6027-589 Scanning file system metadata, phase 3 ...
GPFS: 6027-552 Scan completed successfully.
GPFS: 6027-589 Scanning file system metadata, phase 4 ...
GPFS: 6027-552 Scan completed successfully.
GPFS: 6027-565 Scanning user file metadata ...
46.99 % complete on Wed Jan 4 15:54:07 2017 ( 789797 inodes with total 489266 MB data processed)
50.71 % complete on Wed Jan 4 15:54:47 2017 ( 835584 inodes with total 528044 MB data processed)
100.00 % complete on Wed Jan 4 15:54:48 2017
GPFS: 6027-552 Scan completed successfully.

AGDPCMB1:/#ssh AGDPCCF1 /usr/lpp/mmfs/bin/mmlsdisk chgm_db2data -M

Disk name IO performed on node Device Availability

---

gpfs3nsd localhost /dev/hdisk11 up
gpfs4nsd localhost /dev/hdisk12 up
gpfs5nsd localhost /dev/hdisk13 up
gpfs6nsd localhost /dev/hdisk3 up
gpfs7nsd localhost /dev/hdisk4 up
gpfs8nsd localhost /dev/hdisk5 up
gpfs9nsd localhost /dev/hdisk6 up
gpfs10nsd localhost /dev/hdisk14 up
gpfs11nsd localhost /dev/hdisk7 up
gpfs12nsd localhost /dev/hdisk15 up

gpfs1002nsd pure2 /dev/hdisk2 up
AGDPCMB1:/#ssh BGDPCMB1 /usr/lpp/mmfs/bin/mmlsdisk chgm_db2data -M

Disk name IO performed on node Device Availability

---

gpfs3nsd localhost /dev/hdisk11 up
gpfs4nsd localhost /dev/hdisk12 up
gpfs5nsd localhost /dev/hdisk13 up
gpfs6nsd AGDPCCF1 /dev/hdisk3 up
gpfs7nsd AGDPCCF1 /dev/hdisk4 up
gpfs8nsd AGDPCCF1 /dev/hdisk5 up
gpfs9nsd AGDPCCF1 /dev/hdisk6 up
gpfs10nsd localhost /dev/hdisk14 up
gpfs11nsd AGDPCCF1 /dev/hdisk7 up
gpfs12nsd localhost /dev/hdisk15 up
gpfs1002nsd pure2 /dev/hdisk2 up
AGDPCMB1:/#mmlsdisk chgm_db2data -M

Disk name IO performed on node Device Availability

---

gpfs3nsd BGDPCCF1 /dev/hdisk11 up
gpfs4nsd BGDPCCF1 /dev/hdisk12 up
gpfs5nsd BGDPCCF1 /dev/hdisk13 up
gpfs6nsd localhost /dev/hdisk3 up
gpfs7nsd localhost /dev/hdisk4 up
gpfs8nsd localhost /dev/hdisk5 up
gpfs9nsd localhost /dev/hdisk6 up
gpfs10nsd BGDPCCF1 /dev/hdisk14 up
gpfs11nsd localhost /dev/hdisk7 up
gpfs12nsd BGDPCCF1 /dev/hdisk15 up
gpfs1002nsd pure2 /dev/hdisk2 up