Wondershare EdrawMind
MindMap Gallery Redis
- 14
Redis
This is a mind map about Redis, with a detailed introduction and comprehensive description. I hope it will be helpful to interested friends!
Edited at 2023-12-21 17:36:42
- One Hundred Years of Solitude Character Relationship Chart
One Hundred Years of Solitude is the masterpiece of Gabriel Garcia Marquez. Reading this book begins with making sense of the characters' relationships, which are centered on the Buendía family and tells the story of the family's prosperity and decline, internal relationships and political struggles, self-mixing and rebirth over the course of a hundred years.
- One Hundred Years of Solitude Character Relationship Chart
One Hundred Years of Solitude is the masterpiece of Gabriel Garcia Marquez. Reading this book begins with making sense of the characters' relationships, which are centered on the Buendía family and tells the story of the family's prosperity and decline, internal relationships and political struggles, self-mixing and rebirth over the course of a hundred years.
- Project Management Process Template
Project management is the process of applying specialized knowledge, skills, tools, and methods to project activities so that the project can achieve or exceed the set needs and expectations within the constraints of limited resources. This diagram provides a comprehensive overview of the 8 components of the project management process and can be used as a generic template for direct application.
Redis
- One Hundred Years of Solitude Character Relationship Chart
One Hundred Years of Solitude is the masterpiece of Gabriel Garcia Marquez. Reading this book begins with making sense of the characters' relationships, which are centered on the Buendía family and tells the story of the family's prosperity and decline, internal relationships and political struggles, self-mixing and rebirth over the course of a hundred years.
- One Hundred Years of Solitude Character Relationship Chart
One Hundred Years of Solitude is the masterpiece of Gabriel Garcia Marquez. Reading this book begins with making sense of the characters' relationships, which are centered on the Buendía family and tells the story of the family's prosperity and decline, internal relationships and political struggles, self-mixing and rebirth over the course of a hundred years.
- Project Management Process Template
Project management is the process of applying specialized knowledge, skills, tools, and methods to project activities so that the project can achieve or exceed the set needs and expectations within the constraints of limited resources. This diagram provides a comprehensive overview of the 8 components of the project management process and can be used as a generic template for direct application.
- Recommended to you
- Outline
Redis
affairs
definition
A transaction is a separate isolation operation. All operations in the transaction will be serialized and executed in order. The transaction execution will not be interrupted by commands sent by other customer service terminals.
effect
Concatenate multiple commands to prevent other commands from jumping in the queue
Order
WATCH key [key ...]
Before executing multi, execute watch key1 [key2] first, which can monitor one (or more) keys. If this (or these) keys are changed by other commands before the transaction is executed, the transaction will be interrupted.
Multi
The entered commands will enter the command queue, but will not be executed. They will wait for the Exec command to be executed, and then execute them in sequence.
Exec
Execute command queue
discard
Drop command queue
Transaction error handling
The team formation is successful and the execution is successful. All commands will be executed in an orderly manner.
If the team is successfully formed, select discard and all commands will not be executed.
If an error occurs in a command in the queue, the entire queue will be canceled during execution.
The team is formed successfully, but a certain command goes wrong during execution. Except for the wrong command, all other commands will be executed.
Redis transactions are not atomic
Three major characteristics of Redis transactions
separate quarantine operations
All commands in a transaction are serialized and executed sequentially. During the execution of the transaction, it will not be interrupted by command requests sent by other clients.
No concept of isolation level
The commands in the queue will not be actually executed until they are submitted, because no instructions will be actually executed before the transaction is submitted.
No guarantee of atomicity
If a command in the transaction fails to execute, subsequent commands will still be executed without rollback.
The difference between Redis transactions and Mysql transactions (around ACID)
Persistence
RDB
Core configuration
dbfilename configuration file name
save
How many operations were performed in how many seconds?
Trigger RDB
Automatically trigger save
Will block the main process until RDB persistence is completed
Manual save: bgsave
Redis will perform snapshot operations asynchronously in the background, and the snapshot can also respond to client requests.
You can get the time of the last snapshot execution through lastsave.
Will not block the main process
Execute flushall command
Executing the flushall command will also generate the dump.rdb file, but it is empty and meaningless.
flushdb command does not generate dump.rdb file
Exit Redis normally
More configurations in redis.conf
stop-writes-on-bgsave-error
When Redis cannot write to the disk, directly turn off the writing operation of Redis. Recommend yes
rdbcompression compressed files
If you don't want to consume CPU for compression, you can set this feature to off. Recommend yes.
rdbchecksum checks integrity
After storing the snapshot, you can also ask redis to use the CRC64 algorithm for data verification.
Persistence process (memory snapshot)
Redis will create (fork) a child process separately for persistence. It will first write the data to a temporary file. After the persistence process is completed, this temporary file will be used to replace the last persisted file. During the entire process, the main process does not perform any IO operations, which ensures extremely high performance. If large-scale data recovery is required and the integrity of data recovery is not very sensitive, the RDB method is more efficient than the AOF method. of high efficiency. The disadvantage of RDB is that data after the last persistence may be lost
The function of Fork is to copy a process that is the same as the current process. All the data (variables, environment variables, program counter, etc.) of the new process have the same values as the original process, but it is a completely new process and serves as a child process of the original process.
Advantage
Suitable for large-scale data recovery
More suitable for data integrity and consistency requirements that are not high
Save disk space
Fast recovery
Disadvantages
All modifications from the last snapshot may be lost
AOF
introduce
Record each write operation in the form of a log. Files can only be appended but not rewritten.
Not enabled by default
appendonly no
When AOF and RDB are enabled at the same time, AOF will be given priority.
persistence process
The client's requested write command will be appended to the AOF buffer.
The AOF buffer synchronizes operations to the AOF file on disk according to the AOF persistence policy.
When the AOF file exceeds the rewrite policy or is manually rewritten, the AOF file will be rewritten to compress the AOF file capacity.
When the Redis server restarts, it will load the write operations in the AOF file.
Sync frequency setting
appendfsync always
Always synchronized, every Redis write will be recorded in the log immediately; performance is poor but data integrity is better
appendfsync everysec
Synchronize every second, first write the log to the memory buffer of the AOF file, and then write the contents of the buffer to the disk every second. If there is a downtime, the data of this second may be lost.
appendfsync no
Redis does not actively synchronize and leaves the synchronization timing to the operating system.
Advantage
The backup mechanism is more robust and data is not easily lost
Readable log files to handle error operations
Disadvantages
Takes up more disk space than RDB
Slow recovery
If reading and writing are synchronized every time, performance will be under pressure.
There are some bugs that make recovery impossible.
Redis expired key deletion strategy
Redis uses a combination of regular deletion and lazy deletion strategies.
Periodic deletion: Every once in a while, keys with expiration time are randomly selected, checked to see if they have expired, and deleted after expiration.
Repeat 10 times every 1 second, randomly select 20 keys to check whether they have expired. If 25% of the keys have expired, repeat this behavior
A lot of expired keys will be missed
Lazy deletion: When the data expires, it will not be processed first. When the expired key is accessed, it will be deleted.
Trade space for time
If a lot of expired keys are missed and not checked, lazy deletion will not be performed. A large number of expired keys will accumulate in the memory, causing Redis memory to be exhausted. At this time, the memory elimination mechanism will be used.
master-slave replication
what is
After the host data is updated, the data is automatically synchronized to the master/slave mechanism according to the policy. The master is responsible for writing and the slave is responsible for reading.
What can you do
Read and write separation, performance expansion
Disaster recovery and rapid recovery to improve availability
How to play
Start multiple redis services
Configure slave library
Execute slaveof IP port
Query master-slave information
info replication
Common situations
One master and two servants
Can the slave machine write? Is it possible to set?
It cannot be written, nor can it be set.
What happens after the host is shut down? Is the slave in the upper position or on standby?
The slave machine is on standby
After the master comes back and the master adds a new record, can the slave still copy it smoothly?
Can
What happens after one of the slave machines goes down? Can it keep up with the large army as it is?
Can
Passing the torch
Can machines with dual identities of master and slave write data? ?
No, as long as you have the identity of the slave, you cannot write data.
Mainly anti-customer
If the master machine is down, you can manually turn the slave machine into the master machine.
Copy principle
After the slave starts and successfully connects to the master, it will send a sync command.
Master receives the command to start the background save process and collects all received commands for modifying the data set. After the background process is completed, the master will transfer the entire RDB data file to the slave to complete a complete synchronization.
Full copy: After receiving the database file data, the slave service saves it and loads it into memory.
Incremental replication: Master continues to pass all new collected modification commands to slave in order to complete synchronization
But as long as the master is reconnected, a full synchronization (full replication) will be automatically performed
Sentry mode
Sentinel is a Redi process running in feature mode and has three main tasks:
monitor
Send PING commands to all master-slave libraries to check whether they are alive. If they do not respond within the specified time, they will be judged to be offline.
Subjective offline
If the sentinel finds that the response of the master or slave library to the PING command times out, then the current sentinel will first mark it For "subjective offline
Objective offline
If N/2 1 sentinels mark the main database as subjectively offline, then the main database will be marked as "objectively offline"
Only when the main database is marked as objectively offline, Sentinel will elect a new main database.
If there is only one sentinel, and due to network reasons or excessive pressure on the main database, the main database does not respond to the sentinel in time and is marked as subjective offline by the sentinel. At this time, there is only one sentinel, and a new main database will be directly elected. If there is a sentinel, the new main database will be directly elected. In a cluster, more than N/2 1 sentinels must mark the main database as subjective offline, then the main database will be marked as objectively offline, and then the sentinels will elect a new main database to reduce misjudgments.
Choose the master
If the master library fails, a new master library will be selected from the slave machine.
The process of electing a new master database
filter
From the current online status of the library, it is also necessary to determine its previous network connection status.
Score
Slave library priority
Copy progress from library
Slave library ID number
notify
Send the new master library information to other slave libraries, let other slave libraries establish connections with the new master library, and perform data replication.
what is
The anti-customer-oriented automatic version can monitor whether the host is faulty in the background. If it fails, it will automatically switch from the slave database to the main database based on the number of votes.
How to play
Create a new sentinel.conf file, the name must not be wrong
Configure sentinel and fill in the content
sentinel monitor mymaster 127.0.0.1 6379 1
Among them, mymaster is the server name of the monitoring object, and 1 is the minimum number of sentinels that agree to migrate.
Start Sentinel
/usr/local/bin
Execute redis-sentinel /myredis/sentinel.conf
When the master hangs up, a new master will be selected from the slaves.
election rules
priority
Default in redis.conf: replica-priority 100, the smaller the value, the higher the priority.
Offset
The offset refers to the most complete data obtained from the original host.
runid
After each redis instance is started, a 40-digit runid will be randomly generated (obtain and view through the info server)
cluster
What is a cluster
Horizontal expansion of Redis
If some nodes in the cluster fail or cannot communicate, the cluster can continue to process command requests.
If the master machine is down, the slave machine will be automatically promoted to Master.
Configuration
redis cluster configuration modification
cluster-enabled yes turns on cluster mode
cluster-config-file nodes-6379.conf sets the node configuration file name
cluster-node-timeout 15000 sets the node disconnection time. After this time (milliseconds), the cluster automatically switches between master and slave.
start up
redis-cli --cluster create --cluster-replicas 1 192.168.11.101:6379 192.168.11.101:6380 192.168.11.101:6381 192.168.11.101:6389 192.168.11.101:6390 192 .168.11.101:6391
Do not use 127.0.0.1, please use a real IP address
What are slots
A Redis cluster contains 16384 hash slots, and each key in the database belongs to one of these 16384 slots.
For key values that are not in the same slot, multi-key operations such as mget and mset cannot be used.
Disadvantages of clustering
Multi-key operation is not supported
Multi-key Redis transactions are not supported. Lua scripts are not supported
More complex
Why a cluster requires at least three nodes
If a node is down, and more than half of the nodes think it is down through the ping-pong voting mechanism, then the cluster will be down. If there are only two nodes and one node is down, the other node will vote. Only 50%, But half of them, this cluster will not fail, so at least three nodes are needed
Why a cluster requires at least six servers
Because a cluster requires at least three nodes, and each node requires a backup machine, a cluster requires at least six servers.
JAVA connection development Jedis
Dependencies: <artifactId>jedis</artifactId>
JedisPool connection pool
setBlockWhenExhausted
/Whether to block when the connection is exhausted, false will report an exception, true will block until timeout, default is true
setTestOnBorrow
Check whether the connection is deactivated when fetching it from the connection pool
Autowiring JedisPool objects
Five major data types
application
Basic use
installation manual
redis-benchmark: a performance testing tool that can be run in your own notebook to see how your notebook performs.
redis-check-aof: Repair problematic AOF files, rdb and aof will be discussed later
redis-check-dump: fix problematic dump.rdb file
redis-sentinel: used by Redis cluster
redis-server: Redis server startup command
redis-cli: client, operation entrance
Configuration file redis.conf
Start in background
daemonize no to yes
Start the server redis-server /myredis/redis.conf
Use the client to access redis-cli
-p specifies port
-c cluster mode
Network related configuration
bind
No writing means unlimited access to all IP addresses.
protected-mode protected mode
Enabled by default. Even if access from all IP addresses is accepted, access cannot be successful.
Port port number
Default 6379
tcp-backlog connection queue
timeout timeout time
How long does it take for an idle client to close? 0 turns off this function, that is, the connection will never be disconnected.
tcp-keepalive heartbeat detection of clients
If set to 0, keepalive detection will not be performed.
Common configuration
daemonize daemon thread
The default no setting is yes, which is a daemon process and can be started in the background.
pidfile The location where the pid file is stored
loglevel log level
debug: will print and generate a large amount of information, suitable for the development/testing phase
verbose: contains a lot of less useful information, but is not as confusing as debug level
notice: Moderately verbose, suitable for production environments
warning: Only record very important and critical warning messages
logfile log file name
The default is logfile stdout
databases sets the number of databases
The default number of databases is 16 and the default selected database is 0
limit
maxclients maximum number of server connections
The default is 1W connections. When the upper limit is reached, the connection will be refused.
maxmemory
It is recommended that it must be set, otherwise, the memory will be full and cause the server to crash.
maxmemory-policy memory elimination policy
volatile-lru: Use LRU algorithm to remove keys, only keys with expiration time set; (least recently used)
volatile-ttl: Remove those keys with the smallest TTL value, that is, those keys that will expire recently
volatile-random: Remove random keys from the expired set, only keys with an expiration time set
allkeys-lru: Among all set keys, use the LRU algorithm to remove the key
allkeys-random: remove random keys from all set keys
noeviction: No removal. For write operations, only error information is returned
IO limit
io-threads-do-reads Whether to enable IO multi-threading
The default is no. If enabled, change it to yes.
io-threads Number of IO threads
It is recommended to set 2 or 3 threads for a 4-core machine, and 6 threads for an 8-core machine. The number of threads must be less than the number of machine cores, and try not to exceed 8.
Redis is closed
Single instance shutdown
redis-cli shutdown
Multiple instance shutdown
redis-cli -p 6379 shutdown
Single thread multiple IO multiplexing
Redis6.0 introduces multi-threading
NoSQL database
What is a NoSQL database
Does not follow SQL standards
Does not support ACID
atomicity
consistency
Isolation
persistence
Far exceeds the performance of SQL
Applicable scene
Highly concurrent reading and writing of data
Reading and writing massive data
Highly scalable for data
Not used scenario
Need transaction support
Requires structured query storage, processing complex relationships, and ad hoc queries
Ad hoc query: Users can flexibly select query conditions according to their own needs, and the system generates corresponding statistical reports based on the user's selections.
Redis five major data types
String type
increase
set <key><value>Add key-value pair
setex <key><expiration time><value> When setting the key value, set the expiration time in seconds.
setnx <key><value>Set the value of key only if key does not exist
mset <key1><value1><key2><value2>Set one or more key-value pairs at the same time
check
get <key> Query the corresponding key value
strlen <key> gets the length of the value
mget <key1><key2><key3> Get one or more values at the same time
getrange <key><start position><end position> Get the range of values, similar to substring in java, front package, back package
change
incr <key>
Increase the numeric value stored in key by 1
Can only operate on numeric values. If it is empty, the new value is 1.
decr <key>
Decrement the numeric value stored in key by 1
Can only operate on numeric values. If it is empty, the new value is -1
incrby / decrby <key><step>increases or decreases the numeric value stored in key. Custom step size.
delete
del key
special
append <key><value> appends the given <value> to the end of the original value
getset <key><value>
Replace old with new, set new value and get old value at the same time
The product number and order number are generated using the increasing number characteristics of string.
List type
increase
lpush/rpush <key><value1><value2><value3> .... Insert one or more values from the left/right.
check
lrange <key><start><stop>
Get elements according to index subscript (from left to right)
lindex <key><index>Get elements according to index subscript (from left to right)
llen <key> gets the list length
change
lset<key><index><value>replaces the value whose subscript is index in the list key with value
delete
lrem <key><n><value>Delete n elements equal to value from the left (from left to right)
special
lpop/rpop <key>Puts out a value from the left/right. The value is in the key, and the value is in the key.
rpoplpush <key1> <key2> spits out a value from the right side of the <key1> list and inserts it into the left side of the <key2> list.
linsert <key> before <value><newvalue>Insert <newvalue> after <value> to insert the value
Such as friend list, fan list, message queue, latest news ranking, etc. The rpush method is equivalent to putting the message into the queue, and lpop/rpop is equivalent to taking the message from the queue for consumption.
Hash type
increase
hset <key><field><value> assigns <value> to the <field> key in the <key> collection
hmset <key1><field1><value1><field2><value2>... Set hash values in batches
check
hget <key1><field> retrieves value from <key1> collection <field>
hmget <key1><field1> <field2>... Get hash values in batches
hlen <key> gets the number of field-values in the hash
hexists<key1><field>Check whether the given field field exists in the hash table key
change
hincrby <key><field><increment> adds increment to the value of field field in hash table key (negative means subtraction)
hsetnx <key><field><value> sets the value of field field in hash table key to value if and only if field field does not exist
delete
hdel <key><field> deletes the specified <field> in the <key> collection
special
hgetall <key> gets all fields and values in
hkeys <key> lists all fields of the hash set
hvals <key> lists all values of the hash set
Use an object to store user information, product information, order information, etc.
ZSet type
increase
zadd <key><score1><member 1><score2><member 2>…
Add one or more member elements and their score values to the ordered set key
check
zcard<key> returns the number of elements in the ordered set
zcount <key><min><max> counts the set, the number of elements in the score interval
zrange <key><start><stop> [WITHSCORES] Sort from small to large
Returns the elements in the ordered set key whose subscripts are between <start><stop>
With WITHSCORES, the scores and values can be returned to the result set.
zrevrange <key><start><stop> [WITHSCORES] Sort from large to small
zrangebyscore key min max [withscores] [limit offset count]
Returns all members of the sorted set key whose score value is between min and max (including equal to min or max). The members of the ordered set are arranged in order of increasing score value (from small to large).
zrevrangebyscore key max min [withscores] [limit offset count] (note that the parameter is size)
Same as above, but arranged from largest to smallest.
zrank <key><member> returns the rank of member in the ordered set key. The members of the ordered set are arranged in order of increasing score value (from small to large). The ranking is based on 0, that is, the member with the smallest score value is ranked 0.
zrevrank <key><member> Get the ranking of members in descending order of score value (from large to small).
change
zincrby <key><increment><member > adds an increment to the element's score
delete
zrem <key><member>Delete the element with the specified value in the collection
special
Ranking: For example, a video website needs to rank the videos uploaded by users.
Set type
increase
sadd <key><value1><value2> .....
Add one or more member elements to the collection key. Existing member elements will be ignored.
check
smembers <key> takes out all the values of the collection
scard<key> returns the number of elements in the collection
ismember<key> <value> Query whether the value in the collection already exists
change
delete
srem <key><value1><value2> .... Delete an element in the collection
special
spop <key> spits out a random value from the collection
smove <source><destination>value moves a value in the collection from one collection to another
sinter <key1><key2> returns the intersection element of two sets.
sunion <key1><key2> returns the union elements of two sets.
sdiff <key1><key2> returns the difference elements of the two sets (those in key1, excluding those in key2)
voting record Common friends, common interests, classification tags