BUGLYST.
Practice

Redis · beginner

Redis MISCONF: diagnose persistence failure before changing eviction

Redis errors that begin with "MISCONF" are not memory pressure signals; they are the persistence subsystem telling the server that its durability guarantees can no longer be honored. Before any operator changes `maxmemory-policy` or raises memory limits, the persistence failure itself must be diagnosed, because disabling eviction pressure only masks the underlying safety policy violation that triggered the write refusal. This guide frames the diagnostic as a sequence: confirm the exact MISCONF error, prove that the disk and filesystem layer is the boundary that failed, then decide whether persistence should be repaired, temporarily relaxed, or reconfigured.

The symptoms

  • •Redis client receives an error beginning with "MISCONF" and containing "persistence" or "Background save failed" on any write command such as SET, INCR, or LPUSH.
  • •Server logs at WARNING or NOTICE level show "Background saving failed with error" followed by a filesystem-related reason (e.g., "Permission denied", "No space left on device", "Read-only file system").
  • •The `rdb_last_bgsave_status` and `aof_last_bgrewrite_status` fields returned by `INFO persistence` report `err` rather than `ok` for an extended window.
  • •Application metrics show write rejection or latency spikes aligned with the timestamp of the first MISCONF error in the Redis log.
  • •Replication-linked instances report disconnect or stream errors that coincide with the master's persistence failure, since replicaofs depend on RDB transfer.

Likely causes

  • •The volume hosting the `dir` configured for Redis (commonly `/var/lib/redis` or `/data`) is full, leaving no room for the RDB snapshot or append-only file segment.
  • •File ownership or permission on the configured `dir` or `dbfilename`/`appendfilename` path was changed by a deployment, container image, or filesystem mount, so the Redis process can no longer write its dump file.
  • •The underlying disk became read-only because of a hardware fault, an I/O error escalation by the storage layer, or a node-level remount that turned the data directory read-only.
  • •A snapshot or AOF rewrite was triggered while a previous save was still running, and the policy of `stop-writes-on-bgsave-error` is `yes` (the default), so Redis blocks writes to protect durability.
  • •Filesystem-level quotas, container overlay constraints, or ephemeral disk limits silently cap the data directory before the OS-level "ENOSPC" surfaces to Redis.

First ten minutes

  1. 01Capture the literal error string from the client and from the Redis log; confirm it contains "MISCONF" and references persistence, and record the timestamp to anchor later correlation.
  2. 02Read `INFO persistence` and record `rdb_last_bgsave_status`, `aof_last_bgrewrite_status`, `rdb_last_bgsave_time_sec`, and `loading` so the failure boundary is fixed before any config change.
  3. 03Inspect the data directory boundary by listing the `dir` path from the Redis configuration and confirming that the Redis process user can write to it; record the filesystem and mount options without writing to it yet.
  4. 04Check available space and inode count on the volume that backs the Redis `dir`; treat less than 1 GiB free or any inode exhaustion as a confirmed boundary failure rather than a memory issue.
  5. 05Decide whether to repair the persistence path first (free disk, fix ownership, remount writable) or only then revisit `maxmemory-policy`; do not flip eviction settings until the persistence boundary is proven.

Evidence to collect

  • •The exact MISCONF error text from both the client response and the server log entry, with surrounding log lines for context.
  • •`INFO persistence` output, with focus on `rdb_last_bgsave_status`, `aof_last_bgrewrite_status`, `rdb_last_save_time`, `rdb_current_bgsave_time_sec`, and any non-zero `rdb_last_cow_size`.
  • •Filesystem evidence on the Redis `dir`: total and free bytes, inode usage, mount options (`rw` vs `ro`), and ownership of the directory and any partial `dump.rdb` or `appendonly.aof` files.
  • •Recent configuration changes captured by version control or config audit, especially to `dir`, `dbfilename`, `appenddirname`, `appendfilename`, and `stop-writes-on-bgsave-error`.
  • •OS-level event markers (storage daemon alerts, kernel I/O errors, container OOM-kill of sidecar processes) that map to the same timestamp as the first MISCONF entry.

Where to look

  • •The Redis server log file, filtering for the literal substring "MISCONF" and "Background saving failed" to find the first occurrence and the propagation path.
  • •The persistence section of `INFO persistence`, which surfaces the last-known RDB and AOF background save status without requiring disk access.
  • •The filesystem that hosts the configured `dir`, inspected via standard OS utilities; this is the actual write boundary for the dump file.
  • •The Redis configuration file or runtime `CONFIG GET` output for `dir`, `dbfilename`, `appenddirname`, `appendfilename`, `stop-writes-on-bgsave-error`, and `maxmemory-policy`.
  • •Container or orchestration manifests that bind-mount the data directory, since mount propagation and read-only mounts are a common silent cause.

Diagnostic steps

  1. 01Confirm the failure category by parsing the error: if the message contains "Background save failed" or "Can't save in background," the cause is at the persistence boundary, not at the memory boundary, and eviction tuning will not resolve it.
  2. 02Read `INFO persistence` and treat `rdb_last_bgsave_status: err` as evidence that the last background RDB attempt failed; cross-check `aof_last_bgrewrite_status` to see whether AOF rewrite is also failing.
  3. 03Inspect the disk boundary: report free space, inode usage, mount options, and ownership of the `dir` path; map any non-`rw` mount or zero free space directly to the MISCONF error before considering Redis configuration.
  4. 04Reproduce the save attempt read-only by triggering `BGSAVE` via `CONFIG SET` if available and observing whether the new status returns `ok` once the boundary is healthy; this isolates whether the issue was transient or persistent.
  5. 05Only after the persistence boundary is verified healthy, evaluate whether `maxmemory-policy` should remain `noeviction` (current safety posture) or be relaxed; the choice must be justified by an explicit application requirement, not by the MISCONF error itself.

Common mistakes

  • •Reading "MISCONF" as a memory exhaustion signal and immediately raising `maxmemory` or switching `maxmemory-policy` away from `noeviction`, which leaves the real persistence fault unaddressed.
  • •Setting `stop-writes-on-bgsave-error no` to unblock writes without verifying that the durability expectation of the deployment can tolerate a silent loss of the last successful snapshot.
  • •Trusting only "free memory" metrics on the host; Redis persistence fails on filesystem space, inodes, and permissions, none of which appear in the Redis memory counters.
  • •Restarting Redis to "clear" the error, which can re-trigger the same failure on the next save attempt if the filesystem boundary is still broken, and which can also drop the unsaved working set depending on persistence mode.
  • •Confusing client-reported "OOM command not allowed when used memory > 'maxmemory'" with MISCONF; the former is a memory boundary error and has a different remediation path.

Safe fixes

  • •Repair the persistence boundary first: free disk space, expand the volume, fix directory ownership and permissions, or remount the filesystem read-write, then verify that `INFO persistence` reports `ok` after a manual `BGSAVE` cycle.
  • •For a documented cache-aside deployment where data loss is acceptable, set `stop-writes-on-bgsave-error no` only after the application's recovery contract is recorded and the deployment is known to be a cache rather than a system of record; treat this as a deliberate, version-controlled change.
  • •If AOF is the active persistence mode and the failure is specifically in rewrite, consider a controlled switch to RDB-only by disabling AOF and performing a fresh `BGSAVE`, accepting the trade-off that recent append-only history will be discarded at restart.
  • •After the persistence boundary is proven healthy, revisit `maxmemory-policy` decisions only against an explicit workload argument (cache vs. durable store) and not as a response to the MISCONF error.

Prove the fix

  1. 01`INFO persistence` returns `rdb_last_bgsave_status: ok` and `aof_last_bgrewrite_status: ok` after a deliberate `BGSAVE` cycle, with a non-zero `rdb_last_save_time` newer than the fix timestamp.
  2. 02A representative write command (e.g., `SET`, `INCR`) succeeds without any MISCONF-prefixed error, both immediately after the fix and across a sustained sample window that covers at least one full save interval.
  3. 03No new entries containing "Background saving failed" appear in the Redis log during the verification window; if `stop-writes-on-bgsave-error` was changed, the change is recorded in configuration management with a justification comment.
  4. 04The filesystem holding the Redis `dir` retains healthy free space and a writable mount for the full verification window, demonstrating that the boundary that originally failed is no longer the constraint.

Prevention and next steps

  • •Monitor the same boundary that Redis itself depends on: disk usage, inode usage, and mount writability on the volume hosting the `dir`, with alerts that fire before Redis would refuse writes.
  • •Track `rdb_last_bgsave_status` and `aof_last_bgrewrite_status` from `INFO persistence` as first-class metrics, so a transition to `err` is visible before applications see a MISCONF error.
  • •Keep `stop-writes-on-bgsave-error` at its safe default for any deployment whose data is treated as durable, and document the explicit, narrow exception where it is disabled.
  • •Use configuration management for `dir`, `dbfilename`, and `appenddirname` so deployment changes cannot silently alter the persistence path or its ownership, which is a common precursor to MISCONF errors.

Safe commands and checks

redis-cli -p <port> INFO persistence
redis-cli -p <port> CONFIG GET dir
redis-cli -p <port> CONFIG GET dbfilename
redis-cli -p <port> CONFIG GET appenddirname
redis-cli -p <port> CONFIG GET stop-writes-on-bgsave-error
redis-cli -p <port> CONFIG GET maxmemory-policy
df -h <redis-dir-path-from-config>
grep -E 'MISCONF|Background saving failed' <redis-log-path>