虎年除夕(2022-01-31),Redis 7.0 RC1发布,这是新的Redis团队开发的第2个版本( 6.2和7.0),该版本提供了很多新的特性、做了很多改进(尤其是在性能和内存优化上,关键字Significant),当然也修复了很多小bug,总结下重点如下:
一、新的特性、演进、变化
- Function:相比于lua(since 2.8)、module(since 4.0),Redis Gears(one module)它是一种新的服务端脚本编程方式,解决了以往的一些痛点。(#8693)
- 新的AOF模式:将AOF进行切片(譬如按照大小),主要解决AOF重写对可用性干扰,同时还在日志中添加了元信息(时间戳)可以进行定点恢复(point-in-time recovery )。(#9788)(#9326)
这个功能是阿里云团队提供的,在之前开源的ApsaraCache已经引入该特性(此处有误,是百度团队贡献的)
maxmemory-client配置: 用来限制全部客户端的总内存(防止溢出),此前版本中只能针对单个客户端的输入输出缓冲区进行限制。(#8687)
# A memory value can be used for the client eviction threshold,
for example:
maxmemory-clients 1g
A percentage value (between 1% and 100%) means the client eviction threshold
is based on a percentage of the maxmemory setting. For example to set client
eviction at 5% of maxmemory:
maxmemory-clients 5%
- Redis Cluster: 支持hostname配置(之前版本只支持IP)、较好支持pubsub(之前会有消息扩散)、在cluster bus链接管理做了优化防止内存泄露。(#9530) (#8621) (#9774)
- ACL:更细粒度的基于key的权限控制、基于selectors的命令权限控制。(#9974)
- 使用新的数据结构listpack完全代替ziplist:包含encoding返回类型、hash/set/zset内部实现方式和配置等(#8887, #9366, #9740)
- repl-diskless-sync:无盘复制默认为yes,这个需要关注。(#10092)
- config set maxmemory执行后立即返回,不等evict完成,这个对于运维同学需要关注,可能是个坑。(#10019)
- jemalloc升级到5.2.1 (#9623)
二、新的命令
用户命令:
- zmpop、bzmpop:按排序批量弹出zset的元素
- lmpop、blmpop: 从左侧批量从list弹出元素
- sintercard、zintercard: 计算交集的集合元素个数
- expiretime、pexpiretime: 返回过期的秒级或者毫秒级时间戳
- spublish、ssubscribe、sunsbuscribe、pubsub shardchannels/shardnumsub:集群相关的pubsub命令
- expire:支持nx/xx/gt/lt选项
- set: 支持nx和get组合
- bitops、bitcount:支持bit/byte选项
- shutdown:添加now、force、abort选项
- config get/set: 支持多个配置
- function相关命令:FUNCTION *, FCALL, FCALL_RO - https://redis.io/commands/function-load
- docs、command-tips: 命令文档和命令tips命令
- quit添加到正式命令中
- eval_ro、evalsha_ro:只读版本的eval、evalsha,在只读从节点执行
- sort_ro:只读版本sort,在只读从节点执行
- xadd: 通过"-*"实现自动序号
三、性能和内存优化
- 集群模式性能和内存优化:使用新的数据结构代替radix tree保存key和slot映射(#9356) (Significant)
- 优化dict内部结构(privdata)实现内存优化:对整体key-value、hash key、zset key效果明显 (Significant)
- 设计共享复制缓冲区:减少多个slave复制造成的内存开销 (#9166)
来源:百度的Redis内核同学,Redis 7.0 共享复制缓冲区的设计与实现
- 优化copy-on-write内存开销(#8974) (Significant)
- 使用listpack替代ziplist (#8887, #9366, #9740)
- 集群模式:优化send buffer中无用的内存开销 (#9255)
- 支持list大小超过4GB(#9357)
- 更充分的使用cliengt reply buffers空间(#8968)
- 去掉multi-bulk limit的hard code,使用INT_MAX代替 (#9528)
- 优化fsync相关避免对磁盘产生大量写入。(#9545)
四、新的配置
- appenddirname:定义分片AOF的目录名 (#9872)
- shutdown-timeout: 定义shutdown的超时时间,默认10秒 (#9872)
- maxmemory-clients:上文已经介绍
- cluster-port:自定义cluster通信端口,默认是+10000 (#9389)
- list-max-listpack-_, hash-max-listpack-_, zset-max-listpack-* :代替ziplist (#8887, #9366, #9740)
- busy-reply-threshold:lua-time-limit的别名 (#9963)
- bind-source-addr:针对outgoing connections(outgoing connections (from replica to master, from Sentinel to instances, cluster bus, etc)的bind (#9142)
- repl-diskless-sync-max-replicas: 控制在full repl-diskless-sync-delay前是否开始复制 (#10092)
- latency-tracking:是否开启latency追踪,默认是yes,如开启会展示latencystats section (#9462)
- cluster-announce-hostname and cluster-preferred-endpoint-type:redis cluster支持hostname,默认是ip(#9530)
- cluster-allow-pubsublocal-when-down:redis cluster是down state时候,是否允许使用pubsub,默认是yes (#8621)
- cluster-link-sendbuf-limit:限制cluster bus link's send buffer,默认是0,即不限制,可以通过cluster link查询相关状态 (#9774)
五、新的Info统计
- latencystats section (#9462):展示latency信息
- total_active_defrag_time and current_active_defrag_time (#9377):展示碎片整理相关耗时
- total_eviction_exceeded_time and current_eviction_exceeded_time (#9031):展示逐出数据相关
- INFO: evicted_clients:被maxmemory-clients干掉的客户端连接数 (#8687)
- INFO: mem_cluster_links, total_cluster_links_buffer_limit_exceeded:cluster link's send buffer相关内存 (#9774)
- current_cow_peak:展示copy-on-write消耗峰值 (#8974)
- 移除aof_rewrite_buffer_length选项 (#9788)
- MEMORY STATS: 展示集群模式下key slots映射关系数据结构用到的内存量 (#10017)
- INFO MEMORY: 拆分Functions and EVAL内存统计 (#9780)
- INFO MEMORY: 添加mem_total_replication_buffers, 改变mem_clients_slaves含义 (#9166)
- CLIENT LIST: tot-mem, multi-mem,展示总内存 (#8687)
- CLIENT LIST, INFO: 展示RESP版本 (#9508)
- SENTINEL INFO: tilt_mode_since:sentinel在tilt的持续时间 (#9000)
- LATENCY: 展示module-acquire-GIL latency (#9608)
六、一些changes和bug fix
一些changes和bug fix,可能会对当前客户端、周边生态(平台、相关工具)、线上生产环境产生一些问题,由于条目众多只对其中比较重要的进行描述:
比较重要的changes:
- repl-diskless-sync:无盘复制默认为yes,这个需要关注。
- INFO commandstats:支持子命令
- LPOP/RPOP count在空list执行返回空
- CONFIG SET maxmemory执行后立即返回,不等evict完成
比较重要的bug修复:
- memory usage准确度有进一步提升 (#9095)
- 修复了list-compress-depth的相关bug (#9849, #9779)
- redis-cli: Fix SCAN sleep interval for --bigkeys, --memkeys, --hotkeys
其他changes:
- Modifying the bind parameter to a non-default value will no longer implicitly disable protected-mode (#9034)
- Remove EVAL script verbatim replication, propagation, and deterministic execution logic (#9812) This has been deprecated and off by default since Redis 6 and is no longer supported.
- ACL: pub/sub channels are blocked by default (acl-pubsub-default=resetchannels) (#10181)
- SCRIPT LOAD and SCRIPT FLUSH are no longer propagated to replicas / AOF (#9812)
- ACL: Declarations of duplicate ACL users in startup files and command line arguments will result in an error, whereas previously the last declaration would overwrite the others. (#9330)
- Replication: TTLs are always replicated as absolute (not relative) millisecond timestamps (#8474)
- Fixes in handling multi-key commands with expired keys on writable replicas (#9572)
- CONFIG SET maxmemory returns before starting eviction (#10019)
- AOF: The new Multi-Part mechanism stores data as a set of multiple files in a designated folder (#9788)
- Remove STRALGO command, preserve LCS a standalone command which only works on keys (#9799)
- Remove gopher protocol support (#9057)
- MODULE and DEBUG commands disabled (protected) by default, for better security (#9920)
- Snapshot-creating and other admin commands in MULTI/EXEC transactions are now rejected (#10015)
- PING is now rejected with -MASTERDOWN when replica-serve-stale-data=no (#9757)
- ACL GETUSER reply now uses ACL syntax for keys and channels (#9974)
- COMMAND reply drops random and sort-for-scripts flags, which are now part of command tips (#10104)
- LPOP/RPOP with count against non-existing list return null array (#10095)
- INFO commandstats now shows the stats per sub-command (#9504)
- ZPOPMIN/ZPOPMAX used to produce wrong replies when count is 0 with non-zset (#9711)
- LPOP/RPOP used to produce wrong replies when count is 0 (#9692)
- CONFIG GET bind now returns the current value in effect, even if the implicit default is in use (#9034)
- CONFIG REWRITE now rewrites the list of modules to load (#4848)
- Config: repl-diskless-sync is now set to yes by default (#10092)
- When shutting down, Redis can optionally wait for replicas to catch up on the replication link (#9872)
- Most CONFIG SET, REWRITE, RESETSTAT commands are now allowed during loading (#9878)
- READONLY and READWRITE commands are now allowed when loading and on stale replicas (#7425)
- Fix ACL category for SELECT, WAIT, ROLE, LASTSAVE, READONLY, READWRITE, ASKING (#9208)
- RESET is now allowed even when on unauthenticated connections (#9798)
- SCRIPT LOAD is now allowed on stale replicas (#10126)
除此之外还有一些bug修复,就不逐一说明了:
- Fix COMMAND GETKEYS on EVAL without keys (#9733)
- Improve MEMORY USAGE with allocator overheads (#9095)
- Unpause clients after manual failover ends instead of waiting for timed (#9676)
- Lua: fix crash on a script call with many arguments, a regression in v6.2.6 (#9809)
- Lua: Use all characters to calculate string hash to prevent hash collisions (#9449)
- Prevent LCS from allocating temp memory over proto-max-bulk-len (#9817)
- Tracking: Make invalidation messages always after command's reply (#9422)
- Cluster: Hide empty replicas from CLUSTER SLOTS responses (#9287)
- CLIENT KILL killed all clients when used with ID of 0 (#9853)
- Fix bugs around lists with list-compress-depth (#9849, #9779)
- Fix one in a blue moon LRU bug in RESTORE, RDB loading, and module API (#9279)
- Reset lazyfreed_objects info field with RESETSTAT, test for stream lazyfree (#8934)
- Fix RDB and list node compression for handling values larger than 4GB (#9776)
- Fix a crash when adding elements larger than 2GB to a Set or Hash (#9916)
- Diskless replication could not count as a change and skip next database SAVE (#9323)
- Fix excessive stream trimming due to an overflow (#10068)
- Safe and organized exit when receiving SIGTERM while loading (#10003)
- Improve EXPIRE TTL overflow detection (#9839)
- Add missed error counting for INFO errorstats (#9646)
- DECRBY LLONG_MIN caused negation overflow (#9577)
- Delay discarding cached master when full synchronization (#9398)
- Fix Stream keyspace notification and persistence triggers in consumer creation and deletion (#9263)
- Fix rank overflow in zset with more than 2B entries (#9249)
- Avoid starting in check-aof / check-rdb / sentinel modes if only the folder name contains that name (#9215, #9176)
- create the log file only after done parsing the entire config file (#6741)
- redis-cli: Fix SCAN sleep interval for --bigkeys, --memkeys, --hotkeys (#9624)
- redis-cli: Fix prompt to show the right DB num and transaction state after RESET (#9096)
- Module API: fix possible propagation bugs in case a module calls CONFIG SET maxmemory outside a command (#10019, #9890)
- Module API: carry through client RESP version to module blocked clients (#9634)
- Module API: release clients blocked on module commands in cluster resharding and down state (#9483)
- Sentinel: Fix availability after master reboot (#9438)
- Sentinel: Fix memory leak with TLS (#9753)
- Sentinel: Fix possible failover due to duplicate zero-port (#9240)
- Sentinel: Fix issues with hostname support (#10146)
- Sentinel: Fix election failures on certain container environments (#10197)
七、结束
整体看,对用户侧比较重要的是一些新的命令,对于Redis管理侧,需要关注的是若干个 (Significant) 优化、一些不兼容(譬如RDB)、新的AOF兼容性、一些新的配置的最佳实践。总之,个人觉得Redis 7值得期待,按照套路至少等上几个RC以及几个小版本后,使用更加靠谱。
后面针对Redis 7会做更详细的介绍:
- 内存优化详解
- 新的命令
- Function
- 新的AOF
- ziplist的替代者listpack
- 新的管理命令和统计信息
- 新的配置最佳实践
添加新评论0 条评论