Add lz4 based compression to the compression options.


I know you guys already have quicklz/lzo(I think I remember seeing that in the code), but the new hotness out there is lz4. It's compression ratio whilst not as good as quicklz it is faster. Since this database is geared at people with insanely huge datasets that require insane speeds, I think that lz4 compression would be a great addition to the already available options.

I've not digged through the code enough, and I've not yet compiled it yet(debian stable's gcc is way too old), and while I'm sure the storage engine's going to be overkill for anything that I or anyone I personally know is using, I do think that adding lz4 compression or replacing the quicklz based compression it'd be a great option for people as a default compression algorithm.

P.S. I also realize that the code and algorithm is not that old yet, and also doesn't have a streaming version of it, and although I doubt the compression in the database uses it, I do realize that this is likely a bit early for that algorithm to be placed in a product such as this one, which is used by extremely data critical systems. I would like to see it in there sometime in the future.

P.P.S. Thanks for open sourcing this thing, it's helped me and a friend feel even better about sticking with mariadb and not be envious of postgres' no locking fancy techonlogy. And I do realize that I'm a plebian when it comes to the source code/engines, I just thought it'd be interesting to make you guys aware of it in case you've not yet heard of it.

Edit: I just realized that lz4 is ~2yrs old now, and though it hasn't seen any widespread adoption like has been with quicklz and to a lesser extent lzo, I'd still like to see it in the database. if you wish to close this issue as "won't fix" or what have you, that is perfectly acceptable as quicklz is a damn fine algorithm and will likely be what I use.

Edit 2: Hadoop apparently uses lz4 as an option for their uses so that's a well known product that uses it.










External issue ID


Freshdesk Tickets