Lock tree stalls

Description

Contribution from Rich Prohaska and Kristian Nielsen @ MariaDB

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Rich Prohaska Sep 14 to me, Kristian Hello George, Kristian Nielsen and I have been working on getting optimistic parallel replication working effectively with TokuDB. We needed to fix a couple of bogus stalls in the fractal tree's lock manager that occur due to critical section races. The stall manifest as a thread (transaction) waiting for an available lock until its lock timer pops. In addition, the optimistic parallel replication feature needs a new mechanism to terminate any client threads that are waiting on a lock. Please see this page for details: http://prohaska7.blogspot.com/2016/09/races-in-fractal-tree-lock-manager.html The code is here: https://github.com/percona/PerconaFT/pull/360 Thanks Rich Prohask
1 2 3 4 5 6 7 Rich Prohaska Sep 14 to me, Kristian Hello George, MariaDB's optimistic parallel replication uses a new MySQL API (something like THD_report_wait_for) to notify the optimistic scheduler that a lock conflict was encountered. In addition, it uses a handlerton api (something like kill_query) to cause the storage engine to wakeup any waiting client threads. These changes are not included in the push request as I thought that they were specific to MariaDB. However, I have not been following the algorithms being used in MySQL replication (especially 5.7 and 8.0), so maybe they have something similar.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 Kristian Nielsen <knielsen@knielsen-hq.org> Sep 14 to Rich, me Rich Prohaska <prohaska7@gmail.com> writes: > MariaDB's optimistic parallel replication uses a new MySQL API (something > like THD_report_wait_for) to notify the optimistic scheduler that a lock > conflict was encountered. In addition, it uses a handlerton api (something > like kill_query) to cause the storage engine to wakeup any waiting client > threads. These changes are not included in the push request as I thought > that they were specific to MariaDB. However, I have not been following the > algorithms being used in MySQL replication (especially 5.7 and 8.0), so > maybe they have something similar. The kill_query handlerton API is standard and has been in both MySQL and MariaDB for a long time. It would make sense to include it in TokuDB for Percona Server, it should work the same as in MariaDB without modifications. This allows a user to use KILL QUERY to interrupt a query that is waiting for a row lock. The thd_report_wait_for API is specific to MariaDB and optimistic parallel replication. So it will go in under #ifdef MARIADB_BASE_VERSION, and will not directly benefit Percona Server. MySQL 5.7 does have something similar to thd_report_wait_for(). It is called thd_report_row_lock_wait(), and actually is an identical API (except for the function name) to the MariaDB thd_report_wait_for(), as seen from the storage engine. However, MySQL 5.7 does not have optimistic replication, nor the ability to break engine row locks in case of conflict. The thd_report_row_lock_wait() only sets a flag that is used by the MySQL replication manager at the end of commit, to handle a few InnoDB corner cases. From my limited knowledge of the MySQL replication code, there would not be a benefit for TokuDB to implement the MySQL 5.7 thd_report_row_lock_wait() API currently. I do not know what the policy is for TokuDB to upstream MariaDB-specific code like what Rich mentions. From my side, we can upstream it, or I can maintain it separately in MariaDB, either way works fine for me. The changes are minor, anyway, after Rich's fixes to the locktree code. Hope this helps, - Kristian.
1 2 3 4 5 6 7 George O. Lorch III <george.lorch@percona.com> Sep 15 to Rich, Kristian, bcc: me Hey Rich, I found this old issue in the Tokutek JIRA that seems to describe the same issue you guys just fixed https://tokutek.atlassian.net/browse/FT-332
1 2 3 4 5 Rich Prohaska Sep 15 to me Yes, the first paragraph describes one of the problems that I fixed (the release before wait problem). The second paragraph describes a solution to a scaling and fairness problem with the lock retry algorithm. I would consider a different change that attaches pending lock requests to the lock that causes the conflict. When the lock is release, O(1) time is spent to reassign the lock.

Environment

None

Status

Assignee

George Lorch

Reporter

George Lorch

Labels

External issue ID

None

External issue ID

None

External issue ID

None

External issue ID

None

External issue ID

None

External issue ID

None

External issue ID

None

External issue ID

None

External issue ID

None

Freshdesk Tickets

None

Affects versions

Percona-Server-5.6.32-78.0
Percona-Server-5.7.14-7

Priority

Major
Configure