There were several reasons we asked Mark Callaghan to help.

MariaDB’s performance is stable over the years

Performance, by contrast, is what the blog entry is about. Or rather, it’s about a short summary of three thorough and detailed blog entries so far posted by Mark Callaghan, in April 2024:

Asking for a non-biased external benchmark

My short-hand summary is that performance in MariaDB 11.4 is better than in MySQL 8.0. Not in all situations, not in every single test, but as an approximation, the statement is fair.

  • First, we had a hunch that our performance is considerably better than that of MySQL.
  • Second, we want to give our users confidence to use MariaDB 11.4, as the newest LTS.
  • Third, we wanted our performance to be evaluated by a respectable and non-biased external benchmark expert.

Performance: Perception vs. reality

This blog entry is not about compatibility. The brief version is that MariaDB is more compatible with MySQL 5.7 than MySQL 8.0 itself is. And incompatibilities between MariaDB and MySQL 8.0 are more easy to overcome than between nearly any other two databases.

In the early days, the perception out there was “MariaDB is like MySQL, but faster“. Simplified, but not wrong. Now, we see statements which question both the compatibility and performance claims.

Compatibility? Sure, but off topic

Is performance important for you, along with the latest features and long-term support? Go with MariaDB 11.4. But don’t take our word for it. We asked well known benchmarking expert Mark Callaghan to check out a number of MariaDB and MySQL releases, hit them hard with a tool of his choice, and share his findings.

Three detailed blog entries by Mark Callaghan

I specifically encourage you to read the small print in the actual blog entries on https://smalldatum.blogspot.com/2024/04/ but let me cut and paste some tl;dr summary statements by Mark himself, further shortening them:

  1. Sysbench on a small server: MariaDB and MySQL (2024-04-04)
  2. Sysbench on a (less) small server: MariaDB and MySQL (2024-04-11)
  3. The Insert Benchmark: MariaDB, MySQL, small server, cached workload (2024-04-13)

The outcome: On the low concurrency load (high concurrency results are being prepared), MariaDB maintained stable performance over the last 10 years and 14 releases, while MySQL performance dropped almost by a third.

How the tests were done

An overarching reason, though, is that we see a misconception about how MariaDB Server is perceived. A discrepancy between perception and reality.

  • The work was done by Small Datum LLC (Mark Callaghan’s company) and sponsored by the MariaDB Foundation.
  • The results come from the sysbench benchmark for MariaDB and MySQL on a small server with a cached and low-concurrency workload (benchmarks for other workloads will follow).
  • For MariaDB, the target of the testing was LTS releases from 10.2 through 11.4.
  • For MySQL, the target was 5.6, 5.7 and 8.0.

The outcome: Hardly any performance regression

About Small Datum LLC (https://smalldatum.github.io/):

  1. Modern MariaDB is 13% to 36% faster than modern MySQL
  2. MariaDB is great at avoiding CPU regressions over time
  3. MySQL is less than great at avoiding CPU regressions over time
  4. Modern MariaDB (11.4.1) is faster than modern MySQL (8.0.36) on all INSERT benchmark steps except for two, where they have similar performance.
  5. Modern MariaDB (11.4.1) was in the worst case 15% slower at INSERT than older MariaDB (10.2.44). It is nice to see that MariaDB has done a great job of avoiding performance regressions over time.
  6. There are significant INSERT performance regressions from older MySQL (5.6) to modern MySQL (8.0)

Credits: Small Datum LLC

The setup for the testing is as follows:

  • Small Datum LLC provides consulting for databases with a focus on MySQL, RocksDB and Postgres.
  • It is lead by Mark Callaghan who spent 15 years as the tech lead for MySQL at Facebook and Google building great teams that scaled MySQL far beyond expectations.
  • Prior to that he worked on database internals at Oracle and Informix.
  • He has an MS in Computer Science from UW-Madison and is passionate about database performance and efficiency.

Similar Posts