DBTrail queries open-source database in DuckDB and speeds reports
The open-source relational database is very good at serving the fast, small reads and writes that applications need, but it gets into trouble when somebody runs a large analytical report on the same server. A full-table scan can push hot pages out of the buffer pool, the memory area that holds frequently used data. A long-running read can hold back the database's background cleanup, and a GROUP BY over millions of rows can make the server write temporary tables to disk. The traditional answer is a reporting replica, but that still runs another server of the same database, which remains a row-oriented database, one built to store and fetch whole rows rather than to scan columns quickly. A new open-source tool called DBTrail takes a different path. This article reports on a benchmark by Vadim Tkachenko at Percona, published in early October 2026.
DBTrail is open source under the Apache 2.0 license and is being built by Daniel Guzman-Burgos, who previously worked at Percona as a technical lead for the open-source relational database. It connects to the open-source relational database in a similar way to a replica. First it makes an initial copy of the tables using mydumper, a tool for dumping that database, and stores the data as Parquet files, a compact column-oriented format that analytical engines can read quickly. Those files can live locally or in S3. After that, DBTrail reads the database's binary log, the record of every change made to the database, and periodically applies those changes to the copy. You query the resulting data with DuckDB, an in-process analytical database designed for fast scans of large files. Nothing is installed inside the database server: no plugin, agent, or trigger. The setup keeps the production database untouched.
Tkachenko tested this using three AWS machines in the same subnet in us-west-2a. The source was a r7i.4xlarge running Percona Server 8.4.11-11, a distribution of the open-source relational database. DBTrail and DuckDB ran on a second r7i.4xlarge, and the load generator on a c7i.2xlarge. The source and DBTrail machines had Intel Xeon Platinum 8488C CPUs with 8 cores and 16 threads, 128 GB of memory, and EBS gp3 storage with 16,000 IOPS and 1,000 MB/s. The load machine had the same CPU with 4 cores and 8 threads, 16 GB of memory, and Ubuntu 24.04.5. The workload was sysbench-tpcc, a benchmark that simulates a wholesale supplier, with 200 warehouses. That produced 102 million rows and about 19 GB of data at the start. The test held the workload at 300 transactions per second (TPS). In this benchmark, one TPC-C transaction is roughly 28 SQL statements, which means about 8,500 queries per second (QPS) and 5,200 row changes per second. On the same server running flat out with 64 threads and nothing else attached, it reached 4,049 TPS, or about 115,000 QPS, so the 300 TPS test used only about 7% of available capacity. The light load was intentional, to see what analytical queries cost even when the source has plenty of headroom.
DBTrail installs with one command that starts its Docker Compose stack. The stack contains two containers: DBTrail itself and a version 8.4.9 instance of the open-source relational database used as an index of row changes. That internal instance needs attention because its default buffer pool is that database's 128 MB default, which is far too small for this workload. For the main tests, Tkachenko increased it to 48 GB, set the redo log capacity to 16 GB, and tuned several other settings. DuckDB ran on the same machine with its defaults: 16 threads and a 99 GB memory limit.
Does DBTrail slow the open-source relational database down? The short answer is almost not at all. The load ran continuously, and each phase was measured. With nothing attached for 10 minutes, the source database served 299.9 TPS, 8,554 QPS, and a 95th percentile latency of 41.9 ms. With DBTrail reading the binary log for 10 minutes, it served 300.5 TPS, 8,535 QPS, and 41.9 ms. During the initial copy of all tables, which took 4 minutes 40 seconds, it served 298.1 TPS and 8,460 QPS, with median latency 41.1 ms and a worst 95th percentile of 41.1 ms except for one spike. When the copy was refreshed every 5 minutes for 75 minutes, it served 300.2 TPS and 8,539 QPS. The spike happened exactly when the initial copy started, around second 1,500 of the workload. The 95th percentile latency jumped to 119 ms and throughput dropped to 271 transactions for that one second. This matches a brief global read lock that the initial copy takes when it starts. After that, DBTrail read 102 million rows in less than five minutes without any measurable slowdown in the application workload. CPU confirmation: the source normally sat around 18% CPU, increased to about 35% during the initial table copy, and was around 24% when the five analytical reports were run directly against the source database. The DBTrail machine itself writes roughly as much as the source because it keeps an index of every captured row change.
DBTrail creates a views.sql file containing a DuckDB view for each source table. From there you can query the data normally with SQL. Two sets of benchmarks were run. First, on the TPC-C data after it had grown from 102 million to 113 million rows, five reports were run against the source database and DuckDB while the transactional workload continued. The source database had essentially everything cached in memory, and the DBTrail copy was in its normal state: a base Parquet file plus accumulated change files that DuckDB must combine when running a query. The total time for all five reports was 6 minutes 35 seconds on the source database and 5.7 seconds on DuckDB. Individually, revenue per warehouse went from 9.7 seconds to 0.75 seconds; orders and revenue per district and month from 46.5 seconds to 2.8 seconds; ten best-selling items from 5 minutes 32 seconds to 1.7 seconds; customers by state and credit from 3.6 seconds to 0.18 seconds; and warehouses low on stock from 3.0 seconds to 0.23 seconds.
The second benchmark used TPC-H, a standard analytical workload, at scale factor 10. That dataset contained 86.6 million rows and 18.2 GB of InnoDB data, with normal indexes on join keys, order dates, and ship dates. The initial DBTrail copy took 6 minutes 26 seconds and produced 2.8 GB of Parquet files. All 22 TPC-H queries were run against both systems. The source database took 5 minutes 24 seconds; DuckDB, with a warm cache, took 10.3 seconds. The source database won only Q19 because an index can go directly to the relevant rows, and Q17 was effectively a tie. For the rest, DuckDB was substantially faster. Part 2 of Percona's report will look at how often DBTrail refreshes data to stay up to date.
For teams that run the open-source relational database and occasionally need analytical reports, this pattern removes the need to build and operate a full data pipeline into a separate analytical database.