This release reshapes how we organize, run, and report database workloads across MariaDB, MySQL, and now PostgreSQL.
We appreciate Hetzner’s outstanding support.
Additional improvements came from pull requests submitted by contributors across the community, and from sponsors who provided engineers, and hardware making this work possible.
These systems enabled full multi‑database workload testing, reproducible performance runs, and the scale required to expose real behavior across MariaDB, MySQL, and PostgreSQL.
Reporters and backend ingestion now rely on run metadata rather than external configuration files.
This resolves long‑standing provenance issues.
This makes test cases diff‑friendly and fully reproducible.
Before diving into the details of the TAF 4.0 release, we’d appreciate your input in the MariaDB Foundation Annual Survey
Major Implementations in TAF 4.0
1) PostgreSQL Support (Beta)
TAF 4.0 GitHub Release
- PostgreSQL Database Plugin (PR #7)
- Updated sysbench‑lua for PostgreSQL builds
- Updated RUN SQL library for PostgreSQL
These improve documentation, reproducibility, and automation.
TAF now supports embedding database configuration directly inside the user properties file:
2) Embedded DB Configuration in User Properties
Properties files:
In Closing
- CLI overrides
- Inline DB config block
- db_config_file reference
To MySQL, PostgreSQL, and all database vendors, please take TAF performance testing framework and use it.
3) Auto‑generated Database Configuration File
Recovery mode now re‑runs iteration 1 of the recovered data store to eliminate skew and ensure synchronization. (PR #9)
- built from the winning configuration
- used to start the database
- archived with results
- included in the auto‑generated test‑case properties
hz-bench
4) Auto‑generated Test Case User Properties Files
TAF now generates a temporary DB configuration file at runtime:
- Original properties (minus db_config_file)
- CLI overrides converted into properties
- The winning DB configuration
Virtuozzo is a MariaDB Foundation sponsor, and we appreciate their continued support and engineering contributions to TAF.
5) README.txt Improvements
Thanks to Alex Padula creator and author of TidesDB (PR #8):
- the DB configuration used
- the auto‑generated test‑case properties file
New comparable, minimal, default, analytics, and OLTP configuration files for:
6) Recovery Mode Improvements
The winning configuration is written into the auto‑generated test‑case properties file, archived with results, and passed to reporters and backend ingestion. This eliminates configuration drift and ensures each test case is fully self‑contained.
7) New TAF Usage Options
--test-case-tag/taf.test_case_tag--dump-run-state/taf.dump_run_state--exec-script-file-before-tests--exec-script-file-after-tests
TAF is the third benchmarking framework he has helped design and build.
8) New Database Configuration Files
Rewrote daemon wrapper to fix SSL initialization failures.
- MariaDB
- MySQL
- PostgreSQL
9) Backend and Reporter Improvements
This expands TAF’s multi‑database testing capabilities and ensures consistent behavior across MariaDB, MySQL, and PostgreSQL.
This feature is in beta — feedback and pull requests are welcome.
Each test‑case README now includes:
10) ResultsCompareRaw.pl (New)
hz-bench2
Each test case now produces a standalone properties file containing:
- added client_cpu_affinity
Monty’s recent MDEV‑39497 request also contributed to new features such as --test-case-tag, which helps organize large test runs and keep track of individual cases more reliably. Current results attached to MDEV39497 where generated with a beta TAF 4.0.
- added state directory cleanup
- removed timing options not applicable to HammerDB
Reporters use raw run data for configuration.
- added NormalizeDBType
13) New TidesDB Configuration and Test‑Case Properties
HammerDB TPROC‑C / TPROC‑H:
The MariaDB Foundation released it openly because performance change detection should benefit the entire database ecosystem, not just one project.
- innodb_compare_local.cnf
- tides_compare_local.cnf
- tides_compare_v10.cnf
If TAF helps you prevent regressions, validate improvements, or simply understand your engine more clearly, that’s exactly the point.
- sysbench_local_tidesdb_innodb.properties
- sysbench_tidesdb10_innodb_rocks_compare.properties
14) RenameResultsTestCaseTag Utility (New)
Backend now retrieves DB configuration from run metadata.
Patches and Fixes
Test Suite Framework
- fixed README metadata timing
- moved dynamic fields to WriteReadmeEnd
DatabaseSoftwareInstall
TAF 4.0 was engineered and validated on high‑performance hosts provided by Hetzner.
- correct package‑list parsing
- warnings → errors
- refuse install if directory exists
- hard fail on invalid package validation
- hard fail on missing install_root
- improved ACTIVE selection logic
MariaDB / MySQL Startup (SSL)
Corrected dictionary and parameter usage for SSL.
HammerDB TPROC‑C SSL Configuration
TAF now includes end‑to‑end PostgreSQL support:
Contributor Acknowledgements
Virtuozzo (PR #7, PR #9)
- Lukas Oliva — PostgreSQL plugin
- Viktor Ganeles — recovery‑mode improvements
A new tool for regenerating HTML from raw results using any metric:
PostgreSQL Testing & Patches
- Amrendra Kumar — early PG testing, .deb installation fixes
TidesDB (PR #8)
- Alex Padula — configuration files and properties
Hardware Sponsor Acknowledgment — Hetzner.com
Users deserve predictable performance, and TAF exists to make that possible for all.
Several of the changes in TAF 4.0 were driven directly by requests from Monty, the creator of both MySQL and MariaDB.
- Intel i5‑13500 (20 vCPUs / 14 cores)
- 62 GB RAM
- AlmaLinux 9.7
MariaDB Foundation’s Performance Test Automation Framework (TAF) 4.0 Release represents the biggest changes to the framework since its creation.
- AMD EPYC 9454 (96 vCPUs / 48 cores)
- 125 GB RAM
- AlmaLinux 9.8
[db_config_start]
…
[db_config_end]
Normalizes archived result directories and associated raw/text/CSV/JSON files based on the authoritative test_case_tag.
sysbench_lua:
The improvements are not just incremental — they fundamentally change how results are collected, tagged, and analyzed across the entire automation pipeline.
About the Author
Jonathan (Jeb) Miller has experience in military leadership, Fire/EMS leadership, computer operations leadership, teaching, and more than 27 years of database performance engineering (PervasiveSQL, MySQL, MariaDB).
This guarantees reproducibility and traceability.
Configuration files:
–metric-name=<metric_name>
11) System Saturation Collector Scripts
- taf_collect_mpstat.sh
- taf_collect_perf_sched.sh
- taf_collect_pidstat.sh
- taf_collect_sarq.sh
- taf_start_collectors.sh
- taf_stop_collectors.sh
12) Test Suite Improvements
Major hardening:
