Redis arrives in most architectures as a cache and quietly becomes a database. The session store grows counters, the queue grows a lock service, the rate limiter grows an idempotency key store, and one morning the service depends on data that everyone assumed was disposable. That drift is the actual subject of Redis operations: not whether the in-memory engine is fast, it is, but what the persistence settings promise about the data nobody admitted to caring about. This article walks the persistence machinery from the ground up, the two snapshot and log mechanisms, the replication trap that looks like durability, the eviction policies that decide what disappears first, and the working rules for deciding which data may live in Redis at all.
What Redis is at the core and why the data model matters
Redis holds the entire working set in memory and serves commands from a single threaded execution loop, with I/O threads added in recent versions to multiplex network work while command execution itself stays serialized. Every operation is therefore atomic at the command level, and the latency is the memory access plus the network round trip, which is the number that makes Redis feel instant. Data structures are the product: plain strings for counters and blobs, hashes for object fields, lists and streams for queues and append logs, sorted sets for leaderboards and rate windows, and sets for tags and memberships. The right structure is the one that lets one command do what an application loop would otherwise do, the sorted set that answers the top ten in one call:
ZADD leaderboard 940 "player:17"
ZREVRANGE leaderboard 0 9 WITHSCORESand that shape of thinking, ask the structure for the answer, is the Redis idiom. The memory residency is the budget line: the working set is bounded by RAM, keys carry per key overhead on top of their value, and the eviction policy is the policy statement about what the instance considers expendable.
RDB snapshots and what point in time copies promise
The first persistence mechanism is the RDB, a compact binary snapshot of the dataset taken periodically. The snapshot is produced by a fork of the server process, which benefits from the operating system's copy on write so the parent keeps serving while the child serializes the fork's view to disk. Configuration is two directives in redis.conf:
save 3600 1
save 300 100meaning a snapshot if at least one key changed in the last hour, or at least one hundred in the last five minutes, and BGSAVE triggers one on demand for scripts and backups. The snapshot file is small, it loads fast at boot, and it survives replication between environments because it is a complete image.
The honest description of what it promises is the recovery point objective: everything up to the last successful snapshot, everything after that is gone. A crash between snapshots loses up to the whole snapshot interval, so the configuration line above expresses an RPO of up to an hour for the busy instance and up to five minutes for the busier one. For a cache that is perfect. For the session store it means users re-login after a restart, and for the idempotency store it means duplicates after a crash, which is why the second mechanism exists.
AOF and the one second that everysec chooses to lose
The append only file, AOF, records every write command as it executes, and recovery replays the log. The durability knob is the fsync policy, the moment the operating system is forced the log to physical disk:
appendonly yes
appendfsync everysecThe three settings trade latency for safety explicitly. always fsyncs per write and loses nothing, at the cost of a disk flush in every write path, which is a throughput ceiling and a latency floor few workloads accept. no hands fsync to the kernel and loses whatever the last seconds buffered, which is durability by hope. everysec, the default and the sane middle, fsyncs once per second and the documentation is refreshingly plain: in a worst case crash the instance loses one second of writes.
The log grows with the write traffic, and the rewrite mechanism keeps it bounded: BGREWRITEAOF rewrites the log as the minimal command sequence that reproduces the current dataset, compacting a hundred thousand increments into one set, with the same copy on write fork as snapshots. Auto rewrite triggers on growth percentage:
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mbThe operational habit is checking the persistence section of INFO after every deployment that touches Redis, because the pair of settings, RDB and AOF together, gives both fast boot and a one second floor, and either one alone gives only half of the promise.
Replication is not persistence and the promotion that loses data
The classic architecture mistake is reading the replication feature as a durability feature. Redis replication is asynchronous: the primary acknowledges a write when the primary has it, not when a replica has applied it, so at any instant the replica lags by the network and the apply queue. A primary that crashes and a replica that is promoted lose the tail of that queue, the writes that were acknowledged and never reached the new primary. The window is usually milliseconds, but the phrase usually milliseconds is not a recovery promise, it is a hope with good latency.
Redis acknowledges this with min-replicas-to-write and min-replicas-max-lag, which make the primary refuse writes when fewer than the given count of replicas are within the given lag, converting silent loss into visible unavailability. That is the correct trade for the data that matters, and it is also the moment the team realizes the cache has become a system of record and deserves the treatment one gives those: persistence enabled on every node, an fsync policy chosen on purpose, and backups of the snapshot taken on schedule, because the snapshot file is a complete image that restores anywhere.
The patterns teams actually build on it
The drift from cache to store happens through specific patterns, and each one carries a durability question worth asking out loud. The rate limiter is the first, built on atomic increment with expiry:
INCR ratelimit:client:42
EXPIRE ratelimit:client:42 60The counter lives for its window and then vanishes, and if the instance restarts mid window the limit resets early, which is the acceptable failure for abuse prevention and worth writing down as such. The distributed lock is the second, held with a unique token so it can be released safely:
SET lock:report job-4821 NX EX 30The NX makes the acquire atomic, the token makes the release conditional on still owning the lock, and the EX bounds the lock's life so a crashed holder cannot fence the resource forever. The known sharp edge, that a lock expiring while its holder still runs permits two holders briefly, is a fundamental property of expiry based locking, and the standard answers are fencing tokens checked in the resource itself or accepting the small window for jobs where a duplicate is harmless. The idempotency key store is the third pattern, SET NX again with the request identifier, and it is the one where durability matters most, because a lost key after a crash turns exactly once processing into a duplicate delivery, which is the moment this section's persistence settings stop being theory.
Lua scripts are the fourth pattern and the general form of the others. The server executes a script atomically against the data it touches, which turns read decide write sequences into single commands:
if redis.call("SETNX", KEYS[1], ARGV[1]) == 1 then
redis.call("EXPIRE", KEYS[1], ARGV[2])
return 1
end
return 0The atomicity is a promise about interleaving, not about durability: no other command runs between the script's steps, which removes the race, while the script's write still reaches the disk on the fsync schedule like any other write. Teams that internalize that split, atomic in memory, durable on the log's terms, use Lua confidently for exactly the small decision sequences it suits.
maxmemory eviction and the decision between cache and store
When the working set exceeds maxmemory, Redis must choose what to keep, and the eviction policy is the choice, written as configuration:
maxmemory 4gb
maxmemory-policy allkeys-lruThe policies split by what they assume the instance is for. allkeys-lru and allkeys-lfu evict from all keys by recency or frequency, the cache policies, built on the assumption that old or cold data may vanish and clients will survive. volatile-lru and volatile-ttl evict only among keys carrying expiration, the mixed policy, where persistent keys are protected and expiring keys form the sacrificial pool. noeviction refuses writes when memory is full, the store policy: nothing disappears, and the application receives errors instead, because for a store an error is recoverable and silent eviction is not.
The policy is therefore an architectural statement about every key in the instance, and mixed workloads on one instance are the common configuration error: the cache keys and the session keys and the queue keys sharing maxmemory means the cache can evict the sessions at the worst moment, since to allkeys-lru a session is just a cold key. The production answer is separation by purpose, cache instances and store instances with different policies, different persistence settings and different alerting, so that the statement each instance makes about its data is coherent.
The working rules and the recovery drill
The operational core of Redis is honest about its memory residency, and the working rules follow from it. The checklist that keeps the drift from cache to store from becoming an incident:
- Declare each instance's purpose, cache or store, and let the eviction and persistence settings express that declaration;
- Set every key that can expire, with TTL, because keys without expiry are the memory leak nobody scheduled;
- Choose everysec with RDB snapshots for stores, and treat always as the rare answer that needs latency measurements;
- Check INFO persistence after deployment, aof_last_write_status, rdb_last_save_time, and the rewrite sizes;
- Take scheduled BGSAVE copies of store instances and test the restore, because an untested snapshot is a hope;
- For data that must survive the crash of a single node, use min-replicas-to-write or move that data out of Redis.
The engine that reads like a cache and holds like a database deserves the compliment of being taken seriously. The team that knows which of its Redis instances may lose a second and which must lose nothing has already done the hard part, and the settings above are just the paperwork.
Every pattern above is a small lesson in the same trade: the engine gives atomic decisions at memory speed, and the operator decides what those decisions owe the disk. The settings are the contract, and the contract is only as good as the restore that proved it.
Streams as the queue that asks the right durability questions
The list patterns, push a job, pop a worker, are the first thing teams build and the first thing they lose when a worker crashes between the pop and the processing. The stream data structure exists for exactly that lesson, and it deserves its place in the article because it sharpens the persistence question this whole piece is about. A producer appends and the server assigns a monotonic identifier:
XADD jobs * task "resize" image "4312"A consumer group reads with acknowledgement semantics: the job is delivered, it stays in the stream, and the consumer must explicitly confirm it before it leaves the pending list:
XREADGROUP GROUP workers worker-1 COUNT 10 STREAMS jobs >
XACK jobs workers 1740000000001-0A crashed worker leaves its deliveries in the pending entries list, visible and claimable, XAUTOCLAIM reassigns them after a timeout, and the queue becomes crash tolerant without the consumer holding locks. The durability question arrives with the semantics: an acknowledged job that the crash loses is a duplicated side effect in the downstream system, which is the AOF and fsync policy from the earlier sections deciding the behavior of the application layer above it. Streams are also the honest bounded queue in Redis, because trimming policy exists, XTRIM and MAXLEN cap the stream's memory, and the retention of a queue is a property the team sets on purpose instead of discovering at the memory ceiling.