Tencent interviewer: "Do you understand the database transaction mechanism?"

"Inner monologue: It's trivial, it's not about ACID. After thinking about it, I'm interviewing a technical expert, so it won't be such a simple question, right?"

Cheng Xuyuan: "balabala... spoke with great confidence and calmness."

Tencent interviewer: "Do you understand Redis ' transactions? Can its transaction mechanism achieve ACID properties?"

Cheng Xuyuan: "Scratching my head, this... I know that lua scripts can implement transactions..."

Tencent interviewer: "Okay, go back and wait for the notice."

Brother Code, I followed you to learn the " Redis Series " and won a lot of offers, but I didn't expect to lose in the question " How does Redis implement transactions?"

Let's analyze step by step:

1. What is transactional ACID?

2. How does Redis implement transactions?

3. What properties can Redis transactions implement?

4. Lua script implementation.

What is ACID for a transaction

There is a saying in the "Insect Valley of Yunnan" by the Ghost Blowing Lantern, "Joy will lead to life, and split will lead to death." In order to find Zheng Chenzhu, the three of them have a clear division of labor and work together to advance and retreat to succeed.

A transaction is a unit of concurrency control, consisting of a sequence of operations that are either executed or none of them are executed.

"It is an inseparable unit of work".

When a transaction is executed, it provides special property guarantees:

• Atomicity: Multiple operations of a transaction must be completed, or none of them are completed (ps: What is the atomicity of MySQL? Comments in the message area are welcome);

• Consistency: After the transaction is executed, the integrity constraints of the database are not destroyed, and the order before and after the transaction is executed is a legal data state.

  Database integrity constraints include, but are not limited to:

• Entity integrity (e.g. the row's primary key exists and is unique);

• Column integrity (such as the type, size, and length of the field to meet the requirements)

• foreign key constraints;

• User-defined integrity (for example, the sum of the two account balances should remain unchanged before and after the transfer).

• Isolation: Operations within a transaction are isolated from other transactions, and concurrently executed transactions cannot interfere with each other.

  The emphasis is on the interaction between different transactions, and strict isolation corresponds to the serializable (Serializable) in the isolation level.

• Durability: Once a transaction is committed, all modifications will be permanently saved in the database, even if the system crashes and restarts, the data will not be lost.

Code brother, after understanding the specific requirements of ACID, how does Redis implement the transaction mechanism?

How Redis implements transactions

The MULTI , EXEC , DISCARD and WATCH commands are the basis for Redis to implement transactions.

The execution process of a Redis transaction consists of three steps:

1. open business;

2. command enqueue;

3. execute transaction or discard;

Explicitly start a transaction

The client explicitly starts a transaction through a MULTIcommand , and subsequent commands will be queued and cached and will not actually be executed.

command enqueue

The client sends a series of instructions to be executed in the transaction to the server.

It should be noted that although the command is sent to the server, the Redis instance only temporarily stores this series of commands in a command queue and will not execute it immediately.

execute transaction or discard

The client sends a command to the server to submit or discard the transaction, and let Redis execute the specific command sent in the second step or clear the queue command and give up the execution.

Redis can schedule queued commands to execute simply when EXEC is called.

Commands saved in the queue in the second step can also be discarded via DISCARD .

Redis transaction case

Execute our sample code through the online debugging website: https://try.redis.io

normal execution

Execute a transaction procedure via MULTIand :EXEC

# 开启事务> MULTIOK# 开始定义一些列指令> SET “公众号:码哥字节” "粉丝 100 万"QUEUED> SET "order" "30"QUEUED> SET "文章数" 666QUEUED> GET "文章数"QUEUED# 实际执行事务> EXEC1) OK2) OK3) OK4) "666"

We see that the return result after each read and write instruction is executed is that the QUEUEDthank you operation has been temporarily stored in the command queue and has not been actually executed.

When the EXECcommand is executed, you can see the specific response data of each command.

abandon business

With MULTIand DISCARDdrop queue commands:

# 初始化订单数> SET "order:mobile" 100OK# 开启事务> MULTIOK# 订单 - 1> DECR "order:mobile"QUEUED# 丢弃丢列命令> DISCARDOK# 数据没有被修改> GET "order:mobile""100"

Brother code, can Redis transactions guarantee ACID characteristics?

Good question, let's analyze it together.

Does Redis transaction satisfy ACID?

Redis transactions can execute multiple commands at once, with three important guarantees:

1. Batch instructions will be temporarily stored in the queue before executing the EXEC command;

2. After receiving the EXEC command, the transaction is executed. Any command in the transaction fails to be executed, and the rest of the commands are still executed;

3. During transaction execution, commands submitted by other clients will not be inserted into the current command execution sequence.

atomicity

Code brother, if an error occurs during transaction execution, is atomic performance guaranteed?

During a transaction, two types of command errors may be encountered:

• Before executing the EXECcommand , the sent command itself is wrong. as follows:

• wrong number of parameters;

• The command name is wrong, a non-existing command is used;

• Out of memory (the Redis instance configures the memory limit with the maxmemorydirective ).

• After the EXECcommand , the command may fail. For example, the data types of the command and the operation do not match (a List operation was performed on a value of type String);

• When executing the EXECcommand . A failure of the Redis instance caused the transaction to fail.

Error before execution of EXEC

When the command is enqueued, Redis will report an error and log the error .

At this point, we can continue to submit command operations .

After the EXECcommand , Redis will refuse to execute all submitted command operations and return the result of transaction failure .

In this way, all commands in the transaction will not be executed again, ensuring atomicity.

The following is an example of an error in the enqueue of the instruction, which causes the transaction to fail:

#开启事务> MULTIOK#发送事务中的第一个操作，但是Redis不支持该命令，返回报错信息127.0.0.1:6379> PUT order 6(error) ERR unknown command `PUT`, with args beginning with: `order`, `6`,#发送事务中的第二个操作，这个操作是正确的命令，Redis把该命令入队> DECR b:stockQUEUED#实际执行事务，但是之前命令有错误，所以Redis拒绝执行> EXEC(error) EXECABORT Transaction discarded because of previous errors.

An error is reported after EXEC is executed

When the transaction operation was enqueued, the data type of the command and the operation did not match, but the Redis instance did not check for an error.

However, after executing the EXEC command, when Redis actually executes these commands, an error will be reported.

Hit the blackboard: Although Redis will report an error to the wrong command, the transaction will still execute the correct command. At this time, the atomicity of the transaction cannot be guaranteed!

Code brother, why does Redis not support rollback?

In fact, Redis does not provide a rollback mechanism. Although Redis provides DISCARD command.

However, this command can only be used to actively abandon the transaction execution and clear the temporarily stored command queue, which cannot achieve the effect of rollback.

When EXEC is executed, a failure occurs

If Redis has AOF logging enabled, only part of the transaction operations will be recorded in the AOF log.

We need to use the redis-check-aof tool to check the AOF log file, this tool can remove unfinished transaction operations from the AOF file.

In this way, after we use AOF to restore the instance, the transaction operation will not be executed again, thus ensuring atomicity.

Brief summary :

• An error is reported when the command is queued, and the transaction execution will be abandoned to ensure atomicity;

• No error is reported when the command is queued, and an error is reported when the command is actually executed, and atomicity is not guaranteed;

• The instance fails when the EXEC command is executed. If the AOF log is enabled, atomicity can be guaranteed.

consistency

Consistency is affected by error commands and the timing of instance failures. According to the timing of the two dimensions of command errors and instance failures, three situations can be analyzed.

Before EXEC is executed, an error is reported when entering the queue

Transactions are discarded, so consistency is guaranteed.

After EXEC is executed, the actual execution reports an error

Wrong execution will not be executed, correct instructions can be executed normally, and consistency can be guaranteed.

Instance crashes when EXEC is executed

After the instance fails, it will restart, which is related to the way of data recovery. We will discuss it according to whether the instance has RDB or AOF enabled.

If we do not enable RDB or AOF, then after the instance fails and restarts, the data is gone and the database is consistent.

If we used RDB snapshot, because RDB snapshot is not executed when transaction is executed.

Therefore, the result of the transaction command operation will not be saved in the RDB snapshot, and the data in the database will be consistent when the RDB snapshot is used for recovery.

If we use the AOF log and the instance fails when the transaction operation has not been recorded in the AOF log, then the database data recovered using the AOF log is consistent.

If only some operations are recorded to the AOF log, we can use redis-check-aof to clear the operations that have been completed in the transaction, and the database will be consistent after recovery.

isolation

Transaction execution can be divided into two phases: command enqueue (before EXEC command is executed) and command actual execution (after EXEC command is executed).

Therefore, during concurrent execution, we analyze these two stages in two cases:

1. Concurrent operations are executed before the EXECcommand , and isolation needs to be WATCHguaranteed by the mechanism;

2. Concurrent operations follow EXECcommands , and isolation is guaranteed.

Code brother, what is the WATCH mechanism?

Let's focus on the first case: when the EXEC command of a transaction has not been executed, the command operation of the transaction is temporarily stored in the command queue.

At this time, if there are other concurrent operations, the same key is modified, it needs to see whether the transaction uses the WATCHmechanism .

The role of the WATCH mechanism is to monitor the value changes of one or more keys before the transaction is executed. When the transaction calls the EXEC command to execute, the WATCH mechanism will first check whether the monitored keys have been modified by other clients.

If it is modified, the transaction execution is abandoned to avoid the isolation of the transaction from being destroyed.

At the same time, the client can execute the transaction again. At this time, if there is no concurrent operation of modifying the transaction data, the transaction can be executed normally, and the isolation is also guaranteed.

no watch

If there is no WATCH mechanism, concurrent operations before the execution of the EXEC command read and write data.

When EXEC is executed, the data to be operated within the transaction has changed, and Redis does not achieve isolation between transactions.

Concurrent operations receive execution after EXEC

As for the second case, because Redis executes commands with a single thread, and after the EXEC command is executed, Redis will ensure that all commands in the command queue are executed before executing the following instructions.

So, in this case, concurrent operations do not break transaction isolation.

Persistence

If Redis is not using RDB or AOF, then the persistence properties of transactions are definitely not guaranteed.

If Redis uses the RDB mode, after a transaction is executed, but before the next RDB snapshot is executed, if the instance is down and data is lost, in this case, the data modified by the transaction cannot be guaranteed to be persistent. of.

If Redis adopts AOF mode, there will be data loss because of the three configuration options of AOF mode: no, everysec and always.

Therefore, the durability property of the transaction is still not guaranteed.

Regardless of the persistence mode used by Redis, the durability properties of transactions are not guaranteed.

Summarize

• Redis has a certain atomicity, but does not support rollback.

• Redis does not have the concept of consistency in ACID. (Or rather, Redis was designed to ignore this)

• Redis is isolated.

• Redis cannot guarantee persistence.

The transaction mechanism of Redis can guarantee consistency and isolation, but cannot guarantee durability.

However, because Redis itself is an in-memory database, persistence is not a necessary attribute. We are more concerned with the three attributes of atomicity, consistency and isolation.

The situation of atomicity is more complicated. When the command syntax used in the transaction is incorrect, the atomicity cannot be guaranteed . In other cases, the transaction can be executed atomically.