{"id":3575,"date":"2026-04-04T17:01:09","date_gmt":"2026-04-04T17:01:09","guid":{"rendered":"https:\/\/kedar.nitty-witty.com\/blog\/?p=3575"},"modified":"2026-04-05T16:13:46","modified_gmt":"2026-04-05T16:13:46","slug":"how-to-fix-write-latency-in-mysql-8-4-upgrade","status":"publish","type":"post","link":"https:\/\/kedar.nitty-witty.com\/blog\/how-to-fix-write-latency-in-mysql-8-4-upgrade","title":{"rendered":"How to fix write latency in MySQL 8.4 Upgrade"},"content":{"rendered":"\n

During any MySQL major version upgrade<\/strong>, especially when moving from 8.0 to 8.4, it\u2019s not just about compatibility checks. Subtle default changes can directly impact MySQL performance tuning<\/strong>, and if you\u2019re not watching closely, they can show up as unexpected latency or throughput issues.<\/p>\n\n\n\n

This is a tale from a MySQL 8.4 upgrade causing write latency and the default changes<\/strong> of innodb_change_buffering<\/code>.<\/p>\n\n\n\n

InnoDB\u2019s change buffer (innodb_change_buffering<\/code>) used to be an important optimisation for write-heavy workloads. However, in MySQL 8.4, it is disabled by default<\/strong>. Remember, it is disabled<\/em>, not deprecated<\/strong> (at least as of now).<\/p>\n\n\n\n

This blog post walks through a small but practical use case and what it means for real-world systems.<\/p>\n\n\n\n

\n\n\n\n

What the change buffer was for<\/h1>\n\n\n\n

InnoDB\u2019s change buffer (formerly insert buffer) was designed for secondary indexes when the required leaf page is not present in the buffer pool.<\/p>\n\n\n\n

Instead of performing a random read to fetch that page just to apply an insert, delete-mark, or purge operation, InnoDB records the change in the change buffer and merges it later.<\/p>\n\n\n\n

On spinning disks, where random seeks were expensive and sequential I\/O was cheaper, this provided a real performance benefit and played a role in MySQL write optimization<\/strong>. I have it covered in my previous blog \/ podcast below.<\/p>\n\n\n\n

Reference: https:\/\/kedar.nitty-witty.com\/blog\/mysql-change-buffer-what-when-and-faq<\/a>
Podcast: https:\/\/www.youtube.com\/watch?v=vt78zI_fwfU<\/a><\/p>\n\n\n\n\n\n

\n\n\n\n

Why is innodb_change_buffering<\/code> off by default in 8.4?<\/h1>\n\n\n\n

(drive slow, sharp U turn ahead)<\/em><\/p>\n\n\n\n

1. Database hardware has changed<\/h2>\n\n\n\n

On most modern systems, storage is SSD\/NVMe or RAID-backed, where random I\/O is significantly faster compared to traditional HDDs.<\/p>\n\n\n\n

The original benefit of change buffering was to avoid slow random disk reads for secondary index updates. That bottleneck largely no longer exists.<\/p>\n\n\n\n

So effectively:<\/p>\n\n\n\n

\n

If random I\/O is no longer expensive, avoiding it brings less value for most MySQL workloads<\/strong>.<\/p>\n<\/blockquote>\n\n\n\n

\n\n\n\n

2. Additional overhead<\/h2>\n\n\n\n

Maintaining the change buffer is not free.
– Extra writes
– Additional metadata tracking
– Background merge operations<\/p>\n\n\n\n

All of this adds overhead to write operations and can impact MySQL query performance<\/strong>, especially under sustained write load.
Source: mysql-8-4-lts-new-production-ready-defaults-for-innodb<\/a><\/p>\n\n\n\n

So question arise:<\/p>\n\n\n\n

\n

Does the optimization still justify the cost?<\/p>\n<\/blockquote>\n\n\n\n

\n\n\n\n

In my understanding, this is not a case of \u201cChange buffer is useless<\/strong>\u201d but rather \u201cChange buffer is not beneficial enough<\/strong>\u201d<\/p>\n\n\n\n

\n\n\n\n

This aligns with MySQL 8.4 LTS philosophy of more practical defaults for MySQL performance optimization<\/strong>:<\/p>\n\n\n\n

innodb_change_buffering: all \u2192 none<\/pre>\n\n\n\n
Effectively disabling it out of the box. But why disable it and not remove it?<\/strong> I can\u2019t speak on behalf of MySQL, but I\u2019m actually glad it\u2019s still there. BUT wait there is more to this story.<\/strong> <\/p>\n\n\n\n
Let me first share what drew me to write this blog.<\/p>\n\n\n\n
8.4 Upgrade and Write Latency<\/h2>\n\n\n\n
A “friend” of mine upgraded from MySQL 8.0 to 8.4 and quickly hit the panic button, as their write latency<\/strong> almost doubled. From ~2ms to ~4ms. Now, I may joke about it saying “dude, that’s just 2ms” but for latency-sensitive systems, that\u2019s significant for database performance<\/strong>.<\/p>\n\n\n\n
I know, you being a smart reader would have doubted innodb_change_buffering<\/strong> behaviour and so did I.<\/p>\n\n\n\n
We reverted:<\/p>\n\n\n\n
innodb_change_buffering = all<\/pre>\n\n\n\n
\u2026and the latency dropped back to expected levels. <\/p>\n\n\n\n
You probably already know this, and maybe I did bait you a bit with the \u201chow to fix\u201d title. But in reality, I just wanted to put this out there for someone who might find it useful.<\/strong><\/em> <\/p>\n\n\n\n
and now it’s time for the “more to this story” or “sharp U turn”…<\/p>\n\n\n\n
\n\n\n\n
The Changing story of Change Buffering<\/h2>\n\n\n\n
I have been pointed out that in 9.5 release notes<\/a> covers that innodb_change_buffering<\/strong> is again defaults to all<\/strong>. (thanks JFG, JS)… I had totally over-looked this part and only checked that it was not deprecated in 9.6.<\/p>\n\n\n\n
We can take a hint from the commit log<\/a> that gives us some reasoning for this change.<\/p>\n\n\n\n
WL#16967 InnoDB : Enable innodb_change_buffer by default\n\nDescription: In 8.4, innodb_change_buffering was turned OFF<\/strong> (by setting it to NONE) by default due to certain stability issues<\/strong> identified with change buffering. \nNow that these issues are fixed in 9.3<\/strong>, innodb_change_buffering can be turned ON (by setting it to ALL) by default in the 9.x LTS.<\/code><\/pre>\n\n\n\n
Learning for MySQL users<\/h1>\n\n\n\n
What I understand here is, in MySQL 8.4 reason for change was “improved hardware” or “stability issues”. While in MySQL 9.5 \/ 9.6, the reason for the change was “stability improvements”.<\/p>\n\n\n\n
Regardless, If I were the user, I’d stay with the default and review the latency impact. If the impact is negative, I’d adjust the innodb_change_buffering to “all” (or whatever was my previous settings) and observe. Also remember, this default changes back to “all” in MySQL 9.6.<\/p>\n\n\n\n
Maybe it will stay, maybe it won\u2019t. Defaults have already shifted, and future versions may evolve further as MySQL continues to refine… <\/s>but this brings us to the important takeaway:<\/p>\n\n\n\n
\n
Always test every aspect of your workload during a MySQL upgrade<\/strong>.<\/p>\n<\/blockquote>\n\n\n\n
Especially when even a few milliseconds of latency change can have real impact on MySQL performance tuning and query optimization<\/strong>.<\/p>\n\n\n\n
\n\n\n\n
Final note<\/h1>\n\n\n\n
I recently covered a MySQL Major Version Upgrade Checklist<\/strong> in a podcast, focused on practical upgrade strategies, validation steps, and avoiding surprises like this one.<\/p>\n\n\n\n
If you\u2019re planning an upgrade, it might be a useful reference.<\/p>\n\n\n\n
\n