When I came back to work on Monday, I encountered an error message when using codis to perform data operations:
MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.
Redis is configured to save database snapshots, but it currently cannot be persisted to the hard disk. The command used to modify the collection data cannot be used. Please check the detailed error information in the Redis log.
There are three reasons for this:
BGSAVE, Redis spawns a child process to save data on disk. This error can be checked from the log for the exact reason why BGSAVE failed (usually
/var/log/redis/redis-server.logon a linux computer), but many times BGAVE fails because fork cannot allocate memory. It is also possible that due to a conflict in the optimization of the OS, the fork cannot allocate memory (although the computer has enough available RAM). (In most cases, this is the error).
If you are using Redis in a very write-heavy application, while saving an RDB file on disk or rewriting the AOF log Redis may use up to 2 times the memory normally used. The additional memory used is proportional to the number of memory pages modified by writes during the saving process, so it is often proportional to the number of keys (or aggregate types items) touched during this time. Make sure to size your memory accordingly.
For example, the server currently has 8g of memory, other processes occupy 1g of memory, redis occupies 5g of memory, and the remaining available memory is 2g.
When redis triggers bgsave, it will trigger Can't save in background: fork: Cannot allocate memory. Because redis will fork a process in bgsave, because the remaining memory of the system is less than 5g, an error is triggered.
cd /var/lib/redis_7021 tail -n 10 redis_7021.log
After running the
config set stop-writes-on-bgsave-error no command, close the configuration item
stop-writes-on-bgsave-error to solve the problem.
The effect of this is: stop saving the snapshot, without actually solving the problem.
root@ubuntu# ./redis-cli 127.0.0.1:6379> config set stop-writes-on-bgsave-error no OK 127.0.0.1:6379> lpush myColour "red" (integer) 1