MySQL Deadlocks Are Our Friends
Why another article on this, Marco?
MySQL deadlocks is a topic covered many times, including here at Percona. I suggest you review the reference section at the end of this post for articles on how to identify deadlocks and from where they are generated.
So why another article?
The answer is that messages we receive like the following are still very common:
User (John): “Marco, our MySQL is having problems”
Marco: “Ok John what problems? Can you be a bit more specific?”
John: “Our log scraper is collecting that MySQL has a lot of errors”
Marco: “Ok can you share the MySQL log so I can review it?”
John: “Errors are in the application log, will share one application log”
Marco reviews the log and in it he finds:
“ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction”
Marco’s reaction is: “Oh my …”
Why? Because deadlocks are not what is expressed in the message, and of course, we have a problem of mindset and terminology. In this very short article, I will try to change your point of view around deadlocks.
What is a MySQL Deadlock?
A deadlock is a situation wherein two or more competing actions are waiting for the other to finish. As a consequence, neither ever does. In computer science, deadlock refers to a specific condition when two or more processes are each waiting for each other to release a resource.
In order for a deadlock to happen, four conditions (Coffman conditions) should exist:
1. Mutual exclusion: At least one resource must be held in a non-shareable mode. Otherwise, the processes would not be prevented from using the resource when necessary. Only one process can use the resource at any given instant of time.
2. Hold and wait or resource holding: A process is currently holding at least one resource and requesting additional resources that are being held by other processes.
3. No preemption: A resource can be released only voluntarily by the process holding it.
4. Circular wait: Each process must be waiting for a resource that is being held by another process, which in turn is waiting for the first process to release the resource.
All the above illustrates conditions that are not bound only to RDBMS but to any system dealing with data transaction processing. In any case, it is a fact that today, in most cases, deadlocks are not avoidable unless to prevent one of the above conditions from happening without compromising the system execution integrity. Breaking or ignoring one of the above rules, especially for RDBMS, could affect data integrity, which goes against the reason for an RDBMS to exist.
Just to help us to better contextualize, let us review a simple case of deadlock.
Say I have MySQL with the World schema loaded, and I have the TWO transactions running, both looking for the same two cities in Tuscany (Firenze and Prato) but in a different order.
mysql> select * from City where CountryCode = 'ITA' and District='Toscana'; +------+---------+-------------+----------+------------+ | ID | Name | CountryCode | District | Population | +------+---------+-------------+----------+------------+ | 1471 | Firenze | ITA | Toscana | 376662 | <--- | 1483 | Prato | ITA | Toscana | 172473 | <--- ... +------+---------+-------------+----------+------------+ And both transactions are updating the population: Connection 1 will have: connection1 > start transaction; Query OK, 0 rows affected (0.01 sec) connection1 > select * from City where ID=1471; +------+---------+-------------+----------+------------+ | ID | Name | CountryCode | District | Population | +------+---------+-------------+----------+------------+ | 1471 | Firenze | ITA | Toscana | 376662 | +------+---------+-------------+----------+------------+ 1 row in set (0.00 sec) connection1 > update City set Population=Population + 1 where ID = 1471; Query OK, 1 row affected (0.05 sec) Rows matched: 1 Changed: 1 Warnings: 0 connection1 > update City set Population=Population + 1 where ID = 1483; Query OK, 1 row affected (2.09 sec) Rows matched: 1 Changed: 1 Warnings: 0 Connection 2 will have: connection 2 >start transaction; Query OK, 0 rows affected (0.01 sec) connection 2 >select * from City where ID=1483; +------+-------+-------------+----------+------------+ | ID | Name | CountryCode | District | Population | +------+-------+-------------+----------+------------+ | 1483 | Prato | ITA | Toscana | 172473 | +------+-------+-------------+----------+------------+ 1 row in set (0.01 sec) connection 2 >update City set Population=Population + 1 where ID = 1483; Query OK, 1 row affected (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0 connection 2 >update City set Population=Population + 1 where ID = 1471; ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction
This is a very simple example of deadlock detection. An image may help:
If we stop for a second and ignore the word “ERROR” in the message, what is really happening is that MySQL is preventing us from modifying the data in the wrong way. If the locks would not be in place, one of the two transactions would modify the population, incrementing a number that is not valid anymore.
So, What’s the Right Thing to Do?
The right thing to do is to abort one of the two transactions and NOTIFY the application that, if you really need to perform the action, and in this case, increase the population, better to redo the execution and be sure it is still the case. Just think, it could happen that the application re-runs transaction two and identifies there is no need to increase the value because it is already what it is supposed to be.
Think about this: you are calculating the financial situation of your company and you and your colleague are processing the same data but for different tasks. Without locks and deadlocks, you may end up corrupting each other’s interpretation of the data, and perform the wrong operations. As a result, you may end up paying the wrong salaries – or worse.
Given that, and more, deadlocks (and locks) need to be seen as friends helping us in keeping our data consistent. The problems arise when we have applications poorly designed and developed, and unfortunately, the wrong terminology (in my opinion) in MySQL.
Deadlock Detection
Let us start with MySQL. “Deadlock detection” is detecting an intrinsic inevitable condition in the RDBMS/ACID world. As such, defining it as an ERROR is totally misleading. A deadlock is a CONDITION, and its natural conclusion is the abortion of one of the transactions by reason of the deadlock. The message should be a NOTIFICATION, not an ERROR.
The problem in the apps, instead, is that normally the isolation and validation of the data are demanded to RDBMS, which is fine. But then only seldom can we see applications able to deal with messages like lock-timeout or deadlock. This is, of course, a huge pitfall, because while it is natural to have the RDBMS dealing with the data consistency, it is not, and should not, be responsible for the retry that is bound to the application logic.
Nowadays we have a lot of applications that require very fast execution, and locks and deadlocks are seen as enemies because they have a time cost.
But this is a mistake – a design mistake. Because if you are more willing to have speed instead of data consistency, then you should not use an RDBMS that must respect specific rules, at any (time) cost. Other systems to store data (eventually consistent) will be more appropriate in your case.
If you care about your data, then you need to listen to your RDBMS and write the code in a way that you will get all the benefits out of it, including when it comes to deadlocks.
Conclusion
MySQL deadlocks (and locks) should be seen as friends. They are mechanisms that exist to keep our data consistent. We should not bypass them unless willing to compromise our data. As previously indicated, if you want to understand, in detail, how to diagnose a deadlock, please review the links in the references below.
References
How to Deal With MySQL Deadlocks
Minimize MySQL Deadlocks with 3 Steps
by Marco Tusa via Percona Database Performance Blog
Comments
Post a Comment