Lock tree stalls

Description

Contribution from Rich Prohaska and Kristian Nielsen @ MariaDB

Rich Prohaska
Sep 14

to me, Kristian 
Hello George,

Kristian Nielsen and I have been working on getting optimistic parallel replication working effectively with TokuDB.  We needed to fix a couple of bogus stalls in the fractal tree's lock manager that occur due to critical section races.  The stall manifest as a thread (transaction) waiting for an available lock until its lock timer pops.  In addition, the optimistic parallel replication feature needs a new mechanism to terminate any client threads that are waiting on a lock. 

Please see this page for details:
http://prohaska7.blogspot.com/2016/09/races-in-fractal-tree-lock-manager.html

The code is here:
https://github.com/percona/PerconaFT/pull/360

Thanks
Rich Prohask

Rich Prohaska
Sep 14

to me, Kristian 
Hello George,

MariaDB's optimistic parallel replication uses a new MySQL API (something like THD_report_wait_for) to notify the optimistic scheduler that a lock conflict was encountered.  In addition, it uses a handlerton api (something like kill_query) to cause the storage engine to wakeup any waiting client threads.  These changes are not included in the push request as I thought that they were specific to MariaDB.  However, I have not been following the algorithms being used in MySQL replication (especially 5.7 and 8.0), so maybe they have something similar.

Kristian Nielsen <knielsen@knielsen-hq.org>
Sep 14

to Rich, me 
Rich Prohaska <prohaska7@gmail.com> writes:

> MariaDB's optimistic parallel replication uses a new MySQL API (something
> like THD_report_wait_for) to notify the optimistic scheduler that a lock
> conflict was encountered.  In addition, it uses a handlerton api (something
> like kill_query) to cause the storage engine to wakeup any waiting client
> threads.  These changes are not included in the push request as I thought
> that they were specific to MariaDB.  However, I have not been following the
> algorithms being used in MySQL replication (especially 5.7 and 8.0), so
> maybe they have something similar.

The kill_query handlerton API is standard and has been in both MySQL and
MariaDB for a long time. It would make sense to include it in TokuDB for
Percona Server, it should work the same as in MariaDB without modifications.
This allows a user to use KILL QUERY to interrupt a query that is waiting
for a row lock.

The thd_report_wait_for API is specific to MariaDB and optimistic parallel
replication. So it will go in under #ifdef MARIADB_BASE_VERSION, and will
not directly benefit Percona Server.

MySQL 5.7 does have something similar to thd_report_wait_for(). It is called
thd_report_row_lock_wait(), and actually is an identical API (except for the
function name) to the MariaDB thd_report_wait_for(), as seen from the
storage engine.

However, MySQL 5.7 does not have optimistic replication, nor the ability to
break engine row locks in case of conflict. The thd_report_row_lock_wait()
only sets a flag that is used by the MySQL replication manager at the end of
commit, to handle a few InnoDB corner cases. From my limited knowledge of
the MySQL replication code, there would not be a benefit for TokuDB to
implement the MySQL 5.7 thd_report_row_lock_wait() API currently.

I do not know what the policy is for TokuDB to upstream MariaDB-specific
code like what Rich mentions. From my side, we can upstream it, or I can
maintain it separately in MariaDB, either way works fine for me. The changes
are minor, anyway, after Rich's fixes to the locktree code.

Hope this helps,

 - Kristian.

George O. Lorch III <george.lorch@percona.com>
Sep 15

to Rich, Kristian, bcc: me 
Hey Rich,

I found this old issue in the Tokutek JIRA that seems to describe the same issue you guys just fixed https://tokutek.atlassian.net/browse/FT-332

Rich Prohaska
Sep 15

to me 
Yes, the first paragraph describes one of the problems that I fixed (the release before wait problem). The second paragraph describes a solution to a scaling and fairness problem with the lock retry algorithm.  I would consider a different change that attaches pending lock requests to the lock that causes the conflict.  When the lock is release, O(1) time is spent to reassign the lock.

Environment

None

Status

Assignee

George Lorch

Reporter

George Lorch

Labels

locktree

5.6

5.7

External issue ID

None

External issue ID

None

External issue ID

None

External issue ID

None

External issue ID

None

External issue ID

None

External issue ID

None

External issue ID

None

External issue ID

None

Freshdesk Tickets

None

Affects versions

Percona-Server-5.6.32-78.0

Percona-Server-5.7.14-7

Priority

Major