pgpipe Changelog

All notable changes to pgpipe are documented here.

The format is based on Keep a Changelog and pgpipe adheres to Semantic Versioning.

This file is the canonical source of truth. It ships inside every .deb / .rpm at /usr/share/doc/pgpipe/CHANGELOG.md and is rendered at https://www.pghorizon.com/pgpipe/changelog/.

“Unreleased” lists changes that are merged but have not yet shipped in a numbered release.

Unreleased

2.1.0 - 2026-06-13

A dashboard release: a top-to-bottom overhaul of the built-in web UI — visual system, information architecture, DLQ management, accessibility, and a sweep of honesty fixes so the dashboard never claims more than the engine actually does. No configuration, CLI, or wire changes; the replication engine is unchanged from v2.0.0, so upgrading is a drop-in binary/package replacement.

Added

• DLQ management on the dashboard: the unresolved-entry count is surfaced on the Errors card and as a nav badge, with a Discard-all drain action, per-entry copy buttons, and relative timestamps. “Mark Resolved” is renamed to Discard to match what it actually does.
• A status hero with a dedicated Throughput card and honest sparklines, plus a detail strip showing replication-slot liveness, a copyable LSN, and checkpoint age.
• Hash-based tab routing: the active tab lives in the URL, so refresh, the browser Back button, and deep links to a specific tab (e.g. #errors) all work, and screen readers get aria-current on the active nav item.
• A responsive breakpoint that collapses the sidebar to an icon rail at phone width, plus a keyboard-operability baseline across the dashboard.
• Logs tab: a catch-up pill, an honest pause state, a debounced filter, and a download button.
• Setup wizard: an unload guard, recovery from an expired token or a gone server, table search, copy-from-source, and a finish line that shows the saved config path and the exact start command.
• Config editor: tab ergonomics (a default-open section and expand-all), a persistent restart-required banner after a save, busy states on every async action button, and read-only fields that explain why they can’t be edited.
• The real build version is shown in the UI instead of a hardcoded string.

Changed

• A complete visual refresh built on semantic design tokens (color, type, spacing, mono), with inline styles swept into reusable components, pill radii standardized, emoji iconography replaced by inline SVGs (plus an SVG favicon), and native dark browser chrome via color-scheme / theme-color.
• Reworked navigation information architecture with a distinct identity per view; bespoke buttons folded into one .btn system; a skeleton first paint.
• htmx is now vendored into the binary and served from 'self'; the Content-Security-Policy is tightened to drop the unpkg CDN origin.
• The setup wizard is de-forked from its own stylesheet onto the shared theme.css design system.

Fixed

• Polling refreshes no longer destroy in-progress user state, and session expiry preserves mid-edit work and explains itself; stale content degrades visually and Retry is non-destructive.
• The login page no longer consumes login rate-limit budget on page load.
• Honesty: removed the DDL Detection section from the config editor (DDL replication is not implemented in this build), and corrected three false capability claims in the embedded Docs tab.
• Accessibility: ARIA live regions, proper input labels, a global reduced-motion respect, --text-muted raised to WCAG AA contrast, and tabular numerals on live values.
• The pre-auth flash of dashboard chrome is gated, and the user row is hidden when there is no session.
• Tables: the add-table form is clarified (labels, what “snapshot” means, Enter to submit), and per-table read counts are labeled “since start” so a restart reading zero is no longer mistaken for a stall.
• Lag and uptime now format with hour/day branches; severity thresholds are explained; a fully-caught-up pipeline is colored green; the initializing phase is styled, with a neutral fallback for unknown states.
• Error toasts persist until dismissed (hover pauses auto-dismiss); first-contact empty states for Tables and the DLQ; invalid duration edits are rejected instead of silently dropped; and the dead-end “Retry Save” button is replaced with “Back to editing”.

2.0.0 - 2026-06-10

Security

• BREAKING: /metrics is now closed by default. The endpoint honours metrics.{enabled,path,public,auth_token}; a default install requires the auto-generated bearer token printed on first run, instead of exposing metrics openly. Existing Prometheus scrapers must send the bearer token, or set metrics.public: true to restore the old open behaviour. This default change is the reason this release is a new major version.

Fixed

• Destination upsert-key correctness. A chain of fixes closes the SQLSTATE 42P10 (“no unique or exclusion constraint matching the ON CONFLICT specification”) class that previously surfaced per-row at write time (DLQ-quarantined or pipeline abort) instead of as a clear startup error:
  • Name the ON CONFLICT arbiter from the real primary key rather than the replica-identity keys.
  • Correct REPLICA IDENTITY FULL upserts and make destructive snapshots crash-safe.
  • Validate the destination upsert key at startup and mirror USING INDEX unique constraints, failing loudly with a copy-pasteable ALTER TABLE ... ADD CONSTRAINT ... UNIQUE fix.
  • Exclude DEFERRABLE constraints from the arbiter check — PostgreSQL rejects them as ON CONFLICT arbiters at write time (SQLSTATE 55000).
  • Rewrite the DLQ 42P10 quarantine hint to point at the destination constraint. It previously told operators to check that the source’s REPLICA IDENTITY is not FULL — a diagnosis for the original FULL → 42P10 bug that the arbiter registry has since fixed (FULL is now fully supported), so following the old hint led to fixing a non-problem.
• Honour all snapshot modes via a startup planner, and guard a destructive reset against an active replication slot; skip the heartbeat seed for snapshot_only.
• Reject an empty exported-snapshot name before truncating the destination. A destructive snapshot opens the slot, truncates the destination, then copies — but the copy is a no-op when the snapshot handle’s name is empty. An empty name would therefore truncate, copy nothing, checkpoint the empty result as success, and start streaming: silent data loss. BeginSnapshot now fails before the truncate, leaving the destination intact. (It cannot occur against real PostgreSQL, which always exports a snapshot name, but the invariant was previously unguarded.)
• Seed the destination heartbeat row after the snapshot rather than before. In initial + strict mode the snapshot’s COPY of the heartbeat row collided with a pre-seeded row, aborting the first start with a duplicate key.

1.1.4 - 2026-06-02

Fixed

• Wizard-saved configs no longer crash systemd installs with a “read-only file system” error on first start. The setup wizard previously wrote state.boltdb.path: pgpipe-state.db (a relative path). Under systemd’s ProtectSystem=strict, the service’s working directory / is read-only, so the relative path resolved to /pgpipe-state.db and bbolt could not create the file. The unit then restart-looped until StartLimitBurst gave up and the dashboard never came up.

Changed

• pgpipe.service now sets WorkingDirectory=/var/lib/pgpipe (the directory systemd already grants write access to via StateDirectory=pgpipe), and the setup wizard reads $STATE_DIRECTORY from systemd and writes an absolute state.boltdb.path into /etc/pgpipe/pgpipe.yaml. Configs written via the wizard now work whether started by systemd or run by hand.

Upgrade notes

Fresh installs: drop in the v1.1.4 .deb / .rpm. No extra steps.

Already running v1.1.3 or earlier on systemd with a config saved by the setup wizard — the relative path is baked into your pgpipe.yaml. Patch it once and restart:

sudo cp /etc/pgpipe/pgpipe.yaml /etc/pgpipe/pgpipe.yaml.bak
sudo sed -i 's|pgpipe-state\.db|/var/lib/pgpipe/pgpipe-state.db|' \
  /etc/pgpipe/pgpipe.yaml
sudo systemctl reset-failed pgpipe.service
sudo systemctl start pgpipe.service
sudo journalctl -u pgpipe -f      # expect: snapshot complete -> WAL streaming

The workaround is forward-compatible — the absolute path keeps working after you upgrade, so there is nothing to undo.

Docker / manual runs: not affected — no ProtectSystem=strict, the working directory was already writable.

1.1.3 - 2026-05-22

Fixed

• Setup wizard: the Test Connection result now displays and the Next button is correctly enabled after a successful test. The handler had set the result element’s display: none as an inline style before fetching, which left the result hidden and the wizard unable to advance.

1.1.2 - 2026-05-21

Fixed

• Setup wizard error reporting. The test-connection, list-schemas, and list-tables handlers now warn-log failures and info-log successes; connection errors are classified by SQLSTATE / transport and the UI shows a plain-English summary, remediation, and collapsible technical details.
• list-schemas / list-tables surface real errors instead of silently showing “No schemas/tables found”.
• systemd packaging: StartLimitBurst / StartLimitIntervalSec moved to the [Unit] section (fixes the restart-burst loop), and /etc/pgpipe is added to ReadWritePaths and owned pgpipe:pgpipe so the wizard’s Save endpoint can write pgpipe.yaml.

1.1.1 - 2026-05-19

Fixed

• Fix a Go 1.22 net/http.ServeMux pattern panic that crash-looped the setup wizard on startup.

1.1.0 - 2026-05-17

Added

• Opt-in session_replication_role: replica for the destination pool. Skips BEFORE / AFTER ROW triggers and FK re-checks on already-validated replicated rows via a pgxpool AfterConnect hook. Default empty preserves existing behaviour; requires SUPERUSER on the destination.
• fsync_mode knob for the BoltDB state store.
• Writer phase-level Prometheus timing histograms (pgpipe_writer_build_seconds, pgpipe_writer_send_seconds, pgpipe_writer_commit_seconds) for diagnostic visibility into build / send / commit latency.

Changed

• tx_group_size: 50 is now the default and is respected in strict ordering; the writer is op-type aware.
• Parallel-mode throughput scaling: hot-path pipeline counters are now lock-free atomics, removing the single-mutex cap that limited parallel apply to roughly 8 workers.
• Faster writer and snapshot paths: zero-copy string views in the writer hot path, per-row primary-key extraction with a single-table fast path, count(*) replaced with reltuples for snapshot sizing, and dedicated source connections for heartbeat / slot-lag / admin roles.
• Corrected the throughput claim: pgpipe sustains roughly 95% of native PostgreSQL logical-replication apply throughput, with strict per-transaction ordering preserved. Earlier “faster than native” figures are withdrawn.

Deprecated

• Runtime DDL replication is documented as not implemented and is no longer advertised as on-by-default. The internal/ddl package is scaffolding with no production code path: the WAL decoder never emits a DDL event, so source ALTER TABLE statements are not replicated. The destination schema is still created automatically on first run.

Fixed

• The source setup_schema and the heartbeat table it references are now created before the publication references them, instead of failing on a missing relation.

1.0.1 - 2026-04-30

Maintenance build over v1.0.0. There are no source-tracked changes between the v1.0.0 and v1.0.1 builds — see the v1.0.0 entry for the shipped feature set.

1.0.0 - 2026-04-30

First public release. pgpipe streams row changes from one PostgreSQL database to another in real time, with strict per-transaction ordering, a built-in web dashboard, a dead-letter queue, and Prometheus metrics.

Added

• Web dashboard with a login page and a first-run setup wizard.
• Live table management — add and remove replicated tables from the UI, with table-level filtering.
• Dead-letter queue with a REST API: failed events are captured with full context and can be inspected, retried, and replayed; the UI exposes expand / retry row actions.
• Observability: real-time sparkline charts, snapshot progress bars, a WAL-byte replication-lag metric, a stale-data banner, and an SSE log-tail viewer with a config diff / validation preview.
• Pause / resume for the pipeline.
• Prometheus metrics, plus /health and /ready endpoints for liveness and readiness probes.
• HTTPS / TLS for the dashboard via server.tls.cert_file and server.tls.key_file.
• Startup safety checks: PostgreSQL version detection and replica-identity validation, both reported clearly at startup.
• Bidirectional heartbeat verification and replication-slot drift detection.
• .deb and .rpm packages, and cross-compiled binaries for all 8 supported platforms.
• THIRD_PARTY_LICENSES.txt bundled alongside every binary.

Changed

• 5x–10x write throughput via prepared-statement caching and transaction grouping.
• Single-writer strict ordering by default for consistency, with async commit on the destination.

Fixed

• Total-rows counters now survive restarts, and the events-per-second readout no longer shows “idle” while replicating.

Security

• Dashboard authentication is enabled by default, with a random admin password generated and printed on first run.
• Persistent JWT secret so dashboard sessions survive restarts.
• Rate limiting on /api/auth/login, with trusted-proxy X-Forwarded-For handling.
• Security headers on all dashboard responses: CSP, HSTS, X-Frame-Options, X-Content-Type-Options, and Referrer-Policy.