Smaller LLMs Match Big Ones for DBA Tasks at Lower Cost
Smaller open-weight large language models (LLMs) can already complete database administration (DBA) tasks just as well as much larger models, while costing as little as one-tenth as much per run, according to a benchmark published by Percona on September 17, 2026. Vadim Tkachenko, the author of the evaluation, built a harness that lets different LLMs execute real DBA work on remote servers and reports the results openly.
The harness, called DBA AI Benchmark, follows a simple loop. It receives a task such as 'Install Percona's distribution of the open-source relational database, version 8.4', sends that task to the selected LLM, and asks the model to translate the request into command-line operations or Bash scripts. The harness then executes those commands over SSH (Secure Shell, a protocol for safely running commands on a remote server) on a target system, sends the command output, errors and system state back to the model, and lets the model decide what to do next. The process repeats until the task is completed, a timeout occurs, the maximum number of steps is reached, or a configured cost limit is hit. The project is available on GitHub under Percona-Lab/dbaai_bench.
Tkachenko is testing eight models in two groups. The heavy, full-size models are qwen/qwen3.8-max, moonshotai/kimi-k3, z-ai/glm-5.3 and deepseek/deepseek-v4-pro-0813. The lighter or flash models are deepseek/deepseek-v4-flash-0731, qwen/qwen3.8-27b, meta/muse-glimmer-30b and google/gemma-4-31b-it. All requests and responses are managed through OpenRouter. The author says his primary focus is on open-weight models, whose trained parameters are publicly available, because they offer deployment flexibility and cost efficiency, and because their token costs are often substantially lower than those of closed frontier models. That matters for automation workloads where an agent may need dozens of interactions to finish a single task.
One example run in the post launches the harness against two hosts with qwen/qwen3.8-max, the task 'Install the open-source relational database version 9.7 replication with encrypted traffic', a maximum of 200 steps and unattended mode. The model is responsible for interpreting the request, choosing commands, reacting to errors, and deciding when the task is done. Across the tests, almost all models successfully executed the tasks, with only a few individual exceptions. A striking finding is that smaller and cheaper models are often just as capable as much larger models; the main difference is usually the number of steps. Smaller models may require more iterations to reach a correct configuration, but their overall cost can still be dramatically lower. In several tests, smaller models completed the same work at roughly one-tenth of the cost of larger models.
A detailed leaderboard for a replication task shows this clearly. The task asked models to install Percona's distribution of the open-source relational database, version 8.4 on two freshly built servers, set up replication so one primary takes writes and one replica follows, route replication traffic over the private network, and make the replica refuse writes of its own. The results: deepseek/deepseek-v4-flash-0731 achieved 100% clean results in 35 steps at $0.0082; deepseek/deepseek-v4-pro-0813 achieved 100% in 27 steps at $0.2639; meta/muse-glimmer-30b achieved 100% in 74 steps at $0.0479; moonshotai/kimi-k3 achieved 100% in 17 steps at $0.2888; qwen/qwen3.8-max achieved 100% in 22 steps at $0.4200; z-ai/glm-5.3 achieved 100% in 20 steps at $0.2176; google/gemma-4-31b-it achieved 94% with no clean run after 46 steps at $0.0289; and qwen/qwen3.8-27b timed out with 0% after 78 steps at $0.5116. The cost comparison is especially instructive. meta/muse-glimmer-30b needed 74 steps but cost only $0.0479, while qwen/qwen3.8-max needed just 22 steps but cost $0.4200, and deepseek-v4-flash finished in 35 steps for only $0.0082. In agentic infrastructure workloads, the model that uses the fewest steps is not necessarily the most economical.
Tkachenko also tested how models handle requests that cannot be completed exactly. When asked to install version 9.7 of the open-source relational database, deepseek-v4-pro-0813 replied that version 8.4 of that database was available, not 9.7 as requested, but that 8.4 is what Ubuntu provides, so it would use that version, and it installed the database server on the primary node. Instead of failing, the model interpreted the situation, selected an available version and continued. Whether that behavior is desirable depends on the automation scenario, but it shows that LLM-driven infrastructure automation can make operational decisions rather than blindly following instructions. That makes validation and guardrails particularly important. In another deliberate test, the author asked the models to install Percona's distribution of the open-source relational database, version 9.7.2, a release that does not exist. Most models decided to install the closest available version, 9.7.1, while DeepSeek completed the run without performing an installation. These cases expose how a model reasons about ambiguity, unavailable software, conflicting requirements, and whether it should modify a user's intent.
The framework is not limited to one relational database. The same approach can be used for the open-source relational database, Percona's distribution of that same database, the object-relational database, the document-oriented database, and other database or Linux administration tasks. In practice, the harness tests whether an LLM can act as an iterative systems operator: plan, execute, observe, adjust and verify. Tkachenko expects evaluation to look beyond simple task completion. Useful dimensions include task completion rate, number of steps, total inference cost (the cost of generating model responses), execution time, correctness of the final configuration, number of unnecessary operations, recovery from command failures, handling of ambiguous and impossible requirements, security of generated configurations, and whether the final system actually matches the user's intent. A second project, DBA AI Batch Runner, runs the benchmark across multiple models and configurations, and results are published in a GitHub repository under Percona-Lab-results.
For teams that are starting to evaluate LLM-driven database automation, a managed cloud and data-infrastructure provider such as AEU Data can supply isolated test environments, monitoring and automated backups, which reduce the risk when a model uses its own judgment instead of following instructions exactly.