bulk_insert_buffer_size and InnoDB

Question

I read the following on this page

http://dev.mysql.com/doc/mysql/en/server-system-variables.html

bulk_insert_buffer_size MyISAM uses a special tree-like cache to make bulk inserts faster for INSERT ... SELECT, INSERT ... VALUES (...), (...), ..., and LOAD DATA INFILE. This variable limits the size of the cache tree in bytes per thread. Setting it to 0 disables this optimization. Note: This cache is used only when adding data to a non-empty table. The default value is 8MB. This variable was added in MySQL 4.0.3. This variable previously was named myisam_bulk_insert_tree_size.

I take this to mean that bulk_insert_buffer_size has no bearing on InnoDB tables, only on MyISAM tables. Is this correct?

Answer 1

Yes, you are correct. I wrote about this before: How can I optimize this mySQL table that will need to hold 5+ million rows?

WHen it comes to bulk loading InnoDB tables, one of the most frequently ignored settings to disable is innodb_doublewrite.

InnoDB Architecture

InnoDB Architecture

Please note the double write buffer inside the system tablespace file ibdata1. Every time a 16K page is written to an InnoDB table or transaction log, the page is written to the double wrote buffer first. All you need to do is disable it, load your data, and reenable it.

STEP 01 : Disable Fast Shutdown

mysql -uroot -p... -ANe"SET GLOBAL innodb_fast_shutdown = 0"

STEP 02 : Restart mysql without the double write buffer

service mysql restart --innodb-doublewrite=0

STEP 03 : Load the InnoDB data

STEP 04 : Restart mysql

service mysql restart

CAVEAT

You can disable the foreign key checks and the unique checks as well. Any standard mysqldump will do that for you already.

• noonex: unless it is (properly configured) zfs, disabling doublewrite may cause page corruptions (when partial write happens).
• Muhammad Omer Aslam: do not do this, this has been proven to corrupt data! percona.com/blog/2014/05/23/…
• RolandoMySQLDBA: Hey @MuhammadOmerAslam .I have been recommending this for many years (dba.stackexchange.com/…). I still do this. In fact, I did this for a personal client just 2 months ago. He was reloading a DB Server for Production but was not live. It dramatically cut the reload time by well over 50%. Other than reloading huge amounts of data, you are so right and so is that post you mentioned. I have read that post back then. Yet, I still disable innodb_doublewrite when other means of reloading fast do not work.

Answer 2

Yes it only works with MyISAM tables.

For Innodb we need to diable unique_checks and foreign_key_checks using

set unique_checks=0; set foreign_key_checks=0; diable auto_commit.

inserts......

set unique_checks=1; set foreign_key_checks=1; enable auto_commit.

Which improves bulk inserts performance.