Create utility to allow editing of ydb directory file

Description

Develop a new tool to allow viewing, adding and deleting of entries within the ydb directory file. Maybe also add an audit feature that reconciles the directory entries against the actual files on disk and report any discrepancies.

Maybe call it something tokudiredit. I can imagine some options like:
--attach dname iname // adds an entry that maps dname to the real file iname, file must exist
--move dname_old dname_new // move/renames dname
--remove dname // removes the entry and file specified by dname
--detach dname // removes only the entry dname but leaves the file intact
--list // lists all mappings
--reconcile // compare mappings to actual files and report any discrepancies

It will likely either need to use the toku_* functions directly or will need extension to the YDB API.

For full safety, I believe that the tool should force a checkpoint both before any changes and after any changes.

Environment

None

Activity

Show:
George Lorch
May 4, 2017, 7:07 PM
Vladislav Lesin
February 21, 2017, 3:36 PM

The initial bug can be fixed without utility, see https://bugs.launchpad.net/percona-server/+bug/1657908/comments/7 . So if motivation for creating such utility was this bug and it can be fixed, then... Do we still need the utility?

Vladislav Lesin
February 21, 2017, 8:39 AM

Vadim Tkachenko [10:33 PM]
so do you want to have this as part of server?

Vladislav Lesin [10:33 PM]
Yes.

[10:34]
But, actually, I can implement both variants, But server's variant allows to do changes without downtime.

Vadim Tkachenko [10:34 PM]
which one will be faster and easier?

Vladislav Lesin [10:35 PM]
I think server's variant will be faster because inside of tokudb plugin we already have necessary environment, there is no need to create it from scratch.

Vadim Tkachenko [10:40 PM]
look at this bug https://bugs.launchpad.net/percona-server/+bug/1657908
bugs.launchpad.net
Bug #1657908 “TokuDB does not allow to re-create table after one...” : Bugs : Percona Server
After some disaster one of partitions in a table was lost. Other tables were not affected. Restoring full database from backup is not an option due to size. Server crashed during start, so we deleted partition. After that server successfully stopped, but table could not be dropped. Then we dropped database which contained the table, re-created it. After that all tries to re-create the table failed with error 17. How to repeat: 1. cd into mysql-test directory 2. LD_PRELOAD=$HOME/mysql_pack...

[10:40]
this can be solved inside the server?

Vladislav Lesin [11:32 PM]
I think yes. Because in the bug description: 1) the server can be started on any step described 2) the table can not be created even if it's files were deleted, this looks like tokudb directory still contain record with such table name. But I will create corresponding mtr test and run it in debugger to be absolutely sure. So if this issue can be solved with 'embedded' approach, you approve such approach, right? (edited)

Vadim Tkachenko [11:46 PM]
yes

George Lorch
February 20, 2017, 7:16 PM

Correct, I have not seen a case where either prevented a server from starting unless it ran into case 3, in which case editing directory would not fix.

Fixed

Assignee

Vladislav Lesin

Reporter

George Lorch

Labels

None

External issue ID

None

Freshdesk Tickets

None

Priority

Major