Put TokuDB table files within appropriate schema directories

Description

Today all TokuDB files are located in the datadir. What with file-per-index, multiple tables and partitions, one can easily end up with 20,000 files in the datadir directory.

This is problematic in multiple senses:

  • Not all file systems are easy on large amounts of files in one directory

  • For all the trees one cannot see the wood, so to speak. One has to "ls -l | grep -v tokudb" to take a look at the binary logs, directories etc.

Environment

None

Activity

Show:
thatsafunnyname
September 25, 2013, 4:51 PM

Just to cross reference a thread in tokudb-user that has some discussion on this issue:
https://groups.google.com/d/msg/tokudb-user/iQRLqacjzNQ/rWHw_TEHXkAJ
and the wiki page on this:
https://github.com/Tokutek/ft-index/wiki/Separate-Database-Directories

Peter McLarty
January 7, 2015, 11:32 PM

I think a very powerful move would be to have a similar concept for Oracle and allow the creation of a tokudb database with a filepath allowing the folder to be anywhere rather than containing all the files under the default mysql datadir Then any tools to do backup and recovery would need to read that information but I think mysql needs to go this way in the future. Allows your toku databases to be stored on different filesystems, allowing for fast and slow storage for different purposes

Guo Feng
August 25, 2015, 2:57 AM

Could TokuDB put each schema in a dir and a table consistes table definition file and data file like InnoDB?
TokuDB's file distribution has two problems I think:
1.I couldn't confirm which file belongs to which schema and table,though information_schema.TokuDB_file_map could map file-schema relation,why a more step?
2.so many files,which means so many file handlers

Assignee

Vladislav Lesin

Reporter

Shlomi Noach

Labels

External issue ID

59

Freshdesk Tickets

None

Priority

Configure