硬件层面
a、 硬件规划
b、SSD选择
c、BIOS设置
a、Linux OS
b、Ceph Configurations
c、PG Number调整
d、CRUSH Map
e、其他因素
性能优化从底层硬件、网络、操作系统、软件参数、缓存等几方面
一、硬件:选用ceph合适的硬件。尽量增加SSD,合理分配到pool和index
二、网络:1.增加收发包ethtool -G eth4 rx 8192 tx 8192
2.增加网络带宽,区分集群内外网络
三、操作系统:1.调整包 echo "8192”> /sys/block/sda/queue/read_ahead_kb
2.减少swap使用vm.swappiness = 5
四、软件:
1.RGW(对象存储)
rgw_cache_enabled = true # 开启RGW cache
rgw_thread_pool_size = 2000
rgw_cache_lru_size = 20000
rgw_num_rados_handles = 128
2.RBD(块存储)
rbd_cache_enabled = true # 开启RBD cache
rbd_cache_size = 268435456
rbd_cache_max_dirty = 134217728
rbd_cache_max_dirty_age = 5
3.优化OSD scrub周期及相关参数
osd scrub begin hour = 4
osd scrub end hour = 12
osd scrub chunk min = 5
osd scrub chunk max = 5
osd scrub sleep = 1(不建议增大太多,会延长scrub完成时间)
osd scrub min interval = 2592000
osd scrub max interval = 2592000
osd deep scrub interval = 2419200
osd scrub load threshold = 0.30
osd scrub during recovery = false
osd scrub priority = 5
osd scrub interval randomize ratio = 0.35
五、缓存: 增加磁盘缓存到SSD上(有版本要求) ceph-volume lvmcache add