PgQue Brings Append-Only Event Queues to the Open-Source Database
Many production instances of the open-source database suffer from tables that receive millions of updates per day, triggering autovacuum hundreds of times daily and causing severe table bloat. This leads to poor performance, concurrency problems, and a database that becomes hard to manage. In a September 28, 2026 blog post for Percona, author Jobin Augustine explains how the open-source tool PgQue can solve these exact issues by replacing update-heavy status tracking with an append-only event log.
PgQue is a modernized version of PgQ, a queueing framework originally created by Marko Kreen as part of Skytools. Though largely forgotten for over 20 years, community members like Alexander Kukushkin brought it back into focus through conference talks such as 'Open-source database queues done right with PgQ' at POSETTE 2026. Nikolay Samokhvalov later refreshed the codebase as PgQue, improving documentation and making it easier to deploy. Crucially, PgQue does not require any binary extensions to be installed.
Traditional queue implementations in the open-source database often use SQL features like FOR UPDATE SKIP LOCKED, but that leads to massive table bloat and concurrency bottlenecks. PgQue takes a different approach: it stores events in an append-only partitioned table and uses the open-source database's snapshots to identify which transactions completed between two points in time. Consumers track their own position with a cursor, so no event rows are ever updated in place. This eliminates the bloat and update costs entirely, and allows multiple consumers to read the same event log independently.
Despite its name, PgQue is not a queue in the usual sense because events are never deleted when consumed. Instead, an acknowledgment simply advances that consumer's cursor. The model is a shared, append-only log, closer to a Kafka topic and especially similar to Pulsar's subscription model. Each consumer keeps its own position over the same events, which are stored only once. However, PgQue lacks Kafka's partitions, so it does not provide per-key ordering, and its cursor points at a tick (a time slice) rather than a specific message offset. A consumer commits through a tick number, not through an event ID. Space reclamation works by rotating and truncating old tables, but only after the slowest consumer has moved past them. The guaranteed rewind window is one rotation period, which is 2 hours by default; a stalled consumer blocks reclamation rather than causing data loss. PgQue also offers queue semantics that Kafka leaves to the application: per-consumer redelivery (a failed event is re-appended as a private copy visible only to that consumer), a built-in dead-letter queue, and cooperative consumers that allow multiple workers to share a single logical cursor. The cooperative-consumer feature is marked experimental as of the blog post.
Events are grouped into batches, which reduces network round trips because a single fetch request can return multiple events. After processing a batch, the consumer sends an acknowledgment, and the pointer advances to the next batch. Importantly, acknowledgment is per batch, not per message, so if one message in a batch fails, that message must be negatively acknowledged before the batch is acknowledged. Batch boundaries are defined by a ticker that runs at a scheduled interval, such as every 100 milliseconds (the default) or any exact divisor of 1000 ms. The ticker is triggered once per second by a cron-like scheduler extension for the database, and within that second it produces ticks at the configured rate. After each tick, the ticker emits a LISTEN/NOTIFY event, allowing consumers to wait for notifications instead of polling. On low-activity queues, where fewer than 500 events have accumulated since the last tick, the ticker may be throttled; users can manually force a tick with pgque.force_next_tick().
PgQue provides distinct security roles for producers and consumers: pgque_writer and pgque_reader, plus an administrative role pgque_admin. Core APIs include send(), send_batch(), subscribe(), unsubscribe(), receive(), ack(), and nack(). A set of dead-letter queue functions (dlq_inspect(), dlq_replay(), dlq_replay_all(), dlq_purge(), event_dead()) moves repeatedly failed messages to a separate area. Idempotent producers are supported through send_idem() and maint_idem(), which deduplicate publishes by key so that retrying producers do not double-publish. Management functions include start(), stop(), status(), version(), set_tick_period_ms(), and uninstall(). Observability functions such as queue_stats(), consumer_stats(), queue_health(), otel_metrics(), stuck_consumers(), throughput(), and error_rate() provide statistics, health scoring, stuck-consumer detection, and an OpenTelemetry-compatible metrics view. Delayed delivery is possible with send_at() and related maintenance functions.
To use PgQue, an application can call its raw SQL APIs or use official client libraries. As of the blog post, libraries exist for Python, Go, TypeScript, and Ruby under the 'client' directory of the repository. For testing from the database's command-line interface, a producer can publish an event to a queue with a JSONB payload. A consumer calls pgque.receive() to fetch up to a specified number of events. The receive function returns columns including msg_id, batch_id, type, payload, retry_count, and created_at. Empty batches do not need acknowledgment; receiving them automatically completes the batch and advances the cursor. Non-empty batches must be acknowledged with ack(), otherwise subsequent calls return the same unacknowledged batch. The standard loop is: receive a batch, process messages if any, acknowledge the batch, and repeat. A sample script in the blog shows how to drain events until the consumer catches up with the latest tick.
Jobin Augustine notes that the buzz around 'Just use the open-source database' is growing as system designers want to reduce complexity from external components like Kafka. He references Chandan Shukla's recent post 'The Open-Source Database as a Workflow Engine: Building Reliable Long-Running AI Jobs Without Kafka' as an example of this trend. PgQue provides a ready-made framework that avoids the need to reinvent snapshot-based queueing logic. For teams running the open-source database in production, adopting patterns like PgQue can reduce database bloat and improve scalability. A managed cloud environment such as AEU Data's managed databases can further help by providing automated backups, monitoring, and high availability, allowing teams to focus on application logic rather than infrastructure upkeep.