过期策略以及内存淘汰机制
1、过期策略
常用的过期策略：
- 定时删除：在设置键的过期时间的同时，创建一个定时器，让定时器在键的过期时间来临时，立即执行对键的删除操作。该方法对内存友好，对 CPU 不友好
- 惰性删除：放任键过期不管，但是每次从键空间中获取键时，都检查取得的键是否过期，如果过期的话，就删除该键；如果没有过期，就返回该键。该方法对 CPU 友好，对内存不友好
- 定期删除：每隔一段时间，程序就对数据进行一次检查，删除里面的过期键。该方法是上面两种方法的折中
Redis 采用的过期策略：
定期删除[ 随机抽取部分键，并不是全部 ] + 惰性删除
不采用定时删除的原因：
由于 CPU 时间用在删除和当前任务无关的过期键上，对服务器的响应时间和吞吐量造成影响
存在的问题：
当定期删除没有删除键，也没有请求过键，那么 redis 的内存就会越来越大，这时引入了内存淘汰机制
2、内存淘汰机制
# MAXMEMORY POLICY: how Redis will select what to remove when maxmemory
# is reached. You can select among five behaviors:
#
# volatile-lru -> Evict using approximated LRU among the keys with an expire set.
# allkeys-lru -> Evict any key using approximated LRU.
# volatile-lfu -> Evict using approximated LFU among the keys with an expire set.
# allkeys-lfu -> Evict any key using approximated LFU.
# volatile-random -> Remove a random key among the ones with an expire set.
# allkeys-random -> Remove a random key, any key.
# volatile-ttl -> Remove the key with the nearest expire time (minor TTL)
# noeviction -> Don't evict anything, just return an error on write operations.
#
# LRU means Least Recently Used
# LFU means Least Frequently Used
#
# Both LRU, LFU and volatile-ttl are implemented using approximated
# randomized algorithms.
#
# Note: with any of the above policies, Redis will return an error on write
# operations, when there are no suitable keys for eviction.
#
# At the date of writing these commands are: set setnx setex append
# incr decr rpush lpush rpushx lpushx linsert lset rpoplpush sadd
# sinter sinterstore sunion sunionstore sdiff sdiffstore zadd zincrby
# zunionstore zinterstore hset hsetnx hmset hincrby incrby decrby
# getset mset msetnx exec sort
#
# The default is:
#
# maxmemory-policy noeviction
# noeviction 当内存不足以容纳新写入数据时，新写入操作会报错
# volatile-lru 当内存不足以容纳新写入数据时，在设置了过期时间的键空间中，移除最近最少使用的key
# allkeys-lru 当内存不足以容纳新写入数据时，在键空间中，移除最近最少使用的key
# volatile-lfu 当内存不足以容纳新写入数据时，在设置了过期时间的键空间中，移除最不经常使用的key
# allkeys-lfu 当内存不足以容纳新写入数据时，在键空间中，移除最不经常使用的key
# volatile-random 当内存不足以容纳新写入数据时，在设置了过期时间的键空间中，移除最近最少使用的key
# allkeys-random 当内存不足以容纳新写入数据时，在设置了过期时间的键空间中，随机移除某个key
# volatile-ttl 当内存不足以容纳新写入数据时，在设置了过期时间的键空间中，有更早过期时间的key优先移除
其中 LFU 是在 redis 4.0 新增的一类内存淘汰机制，在 redis 4.0.3 版本支持基于 LFU 的热点 key 发现机制，可参考这篇文章：
数据持久化
Redis 提供了两种不同的持久化方法来将数据存储到硬盘里面：一种方法叫快照，即 RDB，它可以将存在于某一时刻的所有数据都写入硬盘里面；另一种方法叫只追加文件，即 AOF，它会在执行写命令时，将被执行的写命令复制到硬盘里面
1、RDB [ Redis 默认采用 ]
RDB 持久化能够快速地存储和恢复数据，但是在服务器停机时会丢失大量数据
在 Redis 中的配置文件：
################################ SNAPSHOTTING ################################
#
# Save the DB on disk:
#
# save <seconds> <changes>
#
# Will save the DB if both the given number of seconds and the given
# number of write operations against the DB occurred.
#
# In the example below the behaviour will be to save:
# after 900 sec (15 min) if at least 1 key changed
# after 300 sec (5 min) if at least 10 keys changed
# after 60 sec if at least 10000 keys changed
#
# Note: you can disable saving completely by commenting out all "save" lines.
#
# It is also possible to remove all the previously configured save
# points by adding a save directive with a single empty string argument
# like in the following example:
#
# save ""
# 触发持久化的条件
save 900 1
save 300 10
save 60 10000
# By default Redis will stop accepting writes if RDB snapshots are enabled
# (at least one save point) and the latest background save failed.
# This will make the user aware (in a hard way) that data is not persisting
# on disk properly, otherwise chances are that no one will notice and some
# disaster will happen.
#
# If the background saving process will start working again Redis will
# automatically allow writes again.
#
# However if you have setup your proper monitoring of the Redis server
# and persistence, you may want to disable this feature so that Redis will
# continue to work as usual even if there are problems with disk,
# permissions, and so forth.
# 如果持久化操作失败，Redis则会停止接受更新操作
stop-writes-on-bgsave-error yes
# Compress string objects using LZF when dump .rdb databases?
# For default that's set to 'yes' as it's almost always a win.
# If you want to save some CPU in the saving child set it to 'no' but
# the dataset will likely be bigger if you have compressible values or keys.
# 是否压缩文件
rdbcompression yes
# Since version 5 of RDB a CRC64 checksum is placed at the end of the file.
# This makes the format more resistant to corruption but there is a performance
# hit to pay (around 10%) when saving and loading RDB files, so you can disable it
# for maximum performances.
#
# RDB files created with checksum disabled have a checksum of zero that will
# tell the loading code to skip the check.
# 添加校验码
rdbchecksum yes
# The filename where to dump the DB
# 文件名称
dbfilename dump.rdb
# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# The Append Only File will also be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
# 存储路径
dir /usr/local/var/db/redis/
客户端可以使用命令自行触发持久化：
BGSAVE：redis 会调用 fork 来创建一个子进程，然后子进程负责将快照写入硬盘，而父进程则继续处理命令请求SAVE：redis 在快照创建完毕之前将不再响应其他任何命令
RDB 对过期键的处理：
- 在生成
RDB文件时，程序会对数据库中的键进行检查，已过期的键不会被保存到新创建的RDB文件中 - 当 Redis 服务器启动时，载入
RDB文件时：如果服务器是主服务器，程序会对键进行检查，过期键会被忽略；如果服务器是从服务器，不进行键检查，全部载入
2、AOF
AOF 持久化能够有效地提高数据的安全性， 但是在储存和恢复数据方面却要耗费大量的时间
在 Redis 中的配置文件：
############################## APPEND ONLY MODE ###############################
# By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check http://redis.io/topics/persistence for more information.
# 是否启用 AOF
appendonly no
# The name of the append only file (default: "appendonly.aof")
# AOF 文件名称
appendfilename "appendonly.aof"
# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".
# 写入策略
# appendfsync always
appendfsync everysec
# appendfsync no
# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync none". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.
# 是否在后台写时同步单写
no-appendfsync-on-rewrite no
# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.
# 自动重写 AOF 文件
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
# An AOF file may be found to be truncated at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
# 在 redis 恢复时，会忽略最后一条可能存在问题的指令
aof-load-truncated yes
# When rewriting the AOF file, Redis is able to use an RDB preamble in the
# AOF file for faster rewrites and recoveries. When this option is turned
# on the rewritten AOF file is composed of two different stanzas:
#
# [RDB file][AOF tail]
#
# When loading Redis recognizes that the AOF file starts with the "REDIS"
# string and loads the prefixed RDB file, and continues loading the AOF
# tail.
#
# This is currently turned off by default in order to avoid the surprise
# of a format change, but will at some point be used as the default.
# 是否启用 RDB-AOF 混合持久化
aof-use-rdb-preamble no
AOF 执行的过程：
写命令 ——> aof_buf —————> 内存缓冲区 ————————> AOF 文件
| 写入 同步 |
|------------->------------------|
时间循环中的 flushAppendOnlyFile 函数执行该过程，行为由配置文件中的 appendfsync 决定
AOF 对过期键的处理：
- 在生成 AOF 文件时，程序会对 AOF 文件显示追加命令
- 当执行 AOF 文件重写的过程中，程序会对数据库中的键进行检查，已过期的键不会被保存到新创建的 AOF 文件中
3、RDB-AOF 混合持久化
Redis 4.0 推出了 RDB-AOF 混合持久化：这种持久化能够通过 AOF 重写操作创建出一个同时包含 RDB 数据和 AOF 数据的 AOF 文件， 其中 RDB 数据位于 AOF 文件的开头，它们储存了服务器开始执行重写操作时的数据库状态；至于那些在重写操作执行之后执行的 Redis 命令， 则会继续以 AOF 格式追加到 AOF 文件的末尾，也即是 RDB 数据之后。
开启混合持久化的条件：
127.0.0.1:6379> config get 'appendonly'
1) "appendonly"
2) "no"
127.0.0.1:6379> config get 'aof-use-rdb-preamble'
1) "aof-use-rdb-preamble"
2) "no"
