Cheatsheet

Install

Single static binary, ~25 MB, no runtime dependencies. The package installs a hardened systemd unit that runs as a dedicated pgpipe system user.

1. Install the package

# arm64: swap amd64 → arm64
wget https://www.pghorizon.com/downloads/pgpipe/v5.2.0/pgpipe_5.2.0_amd64.deb
sudo apt install ./pgpipe_5.2.0_amd64.deb

# aarch64: swap x86_64 → aarch64
wget https://www.pghorizon.com/downloads/pgpipe/v5.2.0/pgpipe-5.2.0-1.x86_64.rpm
sudo dnf install ./pgpipe-5.2.0-1.x86_64.rpm

The package's postinst script creates the pgpipe system user, sets ownership on the config + state directories, and enables the systemd unit. You do not need to run systemctl enable by hand.

2. What got installed

3. Start it & open the setup wizard

# Start now (the package enabled the unit at install time)
sudo systemctl start pgpipe
sudo systemctl status pgpipe

# Follow structured-JSON logs from the systemd journal
sudo journalctl -u pgpipe -f

# Open http://<host>:8080 — the setup wizard walks you through
# source + destination credentials and writes /etc/pgpipe/pgpipe.yaml.

Need the first-run admin password? → Dashboard login section below.

More options? .deb / .rpm packages + Docker quickstart →

Docker quickstart

Spin up two Postgres databases plus pgpipe in one go — fastest way to see replication.

mkdir pgpipe-demo && cd pgpipe-demo

BASE="https://www.pghorizon.com/downloads/pgpipe/v5.2.0/docker"
for f in Dockerfile docker-compose.yml init-source.sql init-dest.sql; do
  curl -fsSL $BASE/$f -O
done
curl -fsSL $BASE/pgpipe.example.yaml -o pgpipe.yaml

docker compose up -d --build
docker compose logs -f pgpipe

Dashboard login

On first run pgpipe generates a random admin password, prints it once, and persists it to a 0600 file.

# Just the password (always works, even after first run)
docker compose exec pgpipe cat /var/lib/pgpipe/pgpipe-admin.password

# Or grep the full first-run banner from the logs
docker compose logs pgpipe | grep -A 6 "FIRST RUN — DASHBOARD"

# Then open the dashboard and log in as admin
open http://localhost:8080

Setting your own password? Edit pgpipe.yaml → server.auth.password. For production, prefer the PGPIPE_DASHBOARD_PASSWORD env var over a value baked into YAML.

Minimum config (pgpipe.yaml)

The smallest config that runs against your own databases.

pipeline:
  name: "prod-replica"

source:
  host: "source.example.com"
  database: "app"
  user: "pgpipe_repl"
  password: "${PGPIPE_SOURCE_PASSWORD}"
  ssl_mode: "require"
  replication:
    slot_name: "pgpipe_prod"
    publication_name: "pgpipe_prod"
  tables:
    - { schema: "public", name: "orders" }
    - { schema: "public", name: "customers" }

destination:
  host: "replica.example.com"
  database: "app_replica"
  user: "pgpipe_writer"
  password: "${PGPIPE_DEST_PASSWORD}"
  ssl_mode: "require"

state:
  backend: "boltdb"
  boltdb:
    path: "/var/lib/pgpipe/state.db"

Full reference: annotated pgpipe.example.yaml →

Source prerequisites

Run these on the source PostgreSQL before pointing pgpipe at it.

-- 1. postgresql.conf (requires restart for wal_level)
wal_level             = logical
max_replication_slots = 10
max_wal_senders       = 10

-- 2. pg_hba.conf — allow replication from pgpipe's IP
host  all  pgpipe_repl  10.0.0.0/8  scram-sha-256

-- 3. Replication role with least privilege
CREATE ROLE pgpipe_repl WITH LOGIN REPLICATION PASSWORD 'redacted';
GRANT USAGE ON SCHEMA public TO pgpipe_repl;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO pgpipe_repl;
ALTER DEFAULT PRIVILEGES IN SCHEMA public
    GRANT SELECT ON TABLES TO pgpipe_repl;

Walking through this end-to-end: PostgreSQL Logical Replication, Step by Step →

Start / stop / restart

# Foreground (development)
pgpipe start -c pgpipe.yaml

# Validate config without starting
pgpipe validate -c pgpipe.yaml

# Setup-only (creates publication, slot, schema — useful for CI dry-runs)
pgpipe setup -c pgpipe.yaml

# Under systemd (.deb / .rpm install)
sudo systemctl enable --now pgpipe
sudo systemctl status pgpipe
sudo journalctl -u pgpipe -f

# Under Docker Compose
docker compose restart pgpipe
docker compose logs -f pgpipe

Verify replication

-- On the source: slot health + lag
SELECT slot_name, active, wal_status,
       pg_size_pretty(pg_wal_lsn_diff(
         pg_current_wal_lsn(), confirmed_flush_lsn)) AS lag_bytes
FROM   pg_replication_slots;

-- On the source: tables in the publication
SELECT schemaname, tablename
FROM   pg_publication_tables
WHERE  pubname = 'pgpipe_pub';

-- Compare row counts source vs destination
SELECT count(*) FROM public.orders;

pgpipe also exposes everything as Prometheus metrics at http://<host>:8080/metrics. Since v2.0.0 the endpoint requires a bearer token by default — pgpipe prints the auto-generated token once on first run. Set your own via metrics.auth_token or the PGPIPE_METRICS_TOKEN env var, or restore open scraping with metrics.public: true.

# Scrape with the token (v2.0.0+ default)
curl -H "Authorization: Bearer $PGPIPE_METRICS_TOKEN" http://localhost:8080/metrics

# prometheus.yml — scrape job with the bearer token
scrape_configs:
  - job_name: "pgpipe"
    authorization:
      credentials: "<your-token>"
    static_configs:
      - targets: ["pgpipe-host:8080"]

Add / remove tables

Use the dashboard's table picker, or hit the REST API while pgpipe is running.

# Add a table to an active pipeline
curl -u admin:$PASSWORD -X POST http://localhost:8080/api/tables \
  -H 'Content-Type: application/json' \
  -d '{"schema":"public","name":"new_table"}'

# Remove a table
curl -u admin:$PASSWORD -X DELETE \
  'http://localhost:8080/api/tables/public/new_table'

# List configured tables
curl -u admin:$PASSWORD http://localhost:8080/api/tables | jq

Diagnose common issues

Upgrade

pgpipe is not (yet) published to a Debian repository, so sudo apt update && sudo apt install --only-upgrade pgpipe won't find an upgrade — there's no source list to pull from. Instead, download the new .deb and install it on top. The package preserves your config and state, keeps service enablement unchanged, and restarts pgpipe only when it was already running.

1. 1

   Stop pgpipe and snapshot the state DB

   Record whether the service is meant to be enabled and running, then stop it before copying BoltDB so the snapshot is coherent. The state backup contains authority evidence and must be protected like a credential.

   sudo systemctl is-enabled pgpipe
   sudo systemctl is-active pgpipe
   sudo systemctl stop pgpipe
   sudo cp --preserve=mode,ownership,timestamps \
     /var/lib/pgpipe/pgpipe-state.db \
     /var/lib/pgpipe/pgpipe-state.db.bak

2. 2

   Download the new .deb and install over the old one

   Substitute X.Y.Z with the version you're upgrading to (latest is v5.2.0). Apt accepts a local .deb the same way it accepts a remote one, while the package lifecycle replaces the binary without overwriting runtime-created config or state.

   # arm64: swap amd64 → arm64
   wget https://www.pghorizon.com/downloads/pgpipe/vX.Y.Z/pgpipe_X.Y.Z_amd64.deb
   sudo apt install ./pgpipe_X.Y.Z_amd64.deb

   RHEL / Rocky / Fedora equivalent: sudo dnf upgrade ./pgpipe-X.Y.Z-1.x86_64.rpm.

3. 3

   Validate the service state

   Because this checklist stopped pgpipe for a coherent backup, it remains stopped after installation. Validate the config, then restore only the enablement and running state you recorded in step 1. If you skipped the backup and left pgpipe running, the corrected package lifecycle restarts it automatically. On the first Debian transition from an older affected package, check both values because its old prerm can erase the prior service state.

   # Optional but recommended: validate the existing config against the
   # new binary, then inspect the preserved service state.
   sudo -u pgpipe -- /usr/bin/pgpipe validate -c /etc/pgpipe/pgpipe.yaml
   sudo systemctl is-enabled pgpipe
   sudo systemctl is-active pgpipe
   
   # First affected Debian transition only, and only when these
   # states were intended before the upgrade:
   sudo systemctl enable pgpipe
   sudo systemctl start pgpipe

4. 4

   Verify

   # Binary reports the new version
   pgpipe --version
   
   # Unit is active and not in restart loop
   sudo systemctl status pgpipe
   
   # Watch the first ~30s of logs — confirms the slot reattached
   # at the saved LSN and streaming is back
   sudo journalctl -u pgpipe -f

What gets preserved across the upgrade

Rollback (if the new version misbehaves)

Treat rollback as a coordinated recovery. Stop pgpipe first, preserve the current state directory, and restore the previous package together with the pre-upgrade state snapshot when that release's notes do not explicitly guarantee forward-state compatibility. Do not run an older binary against state already advanced by a newer release merely as a quick test.

sudo systemctl stop pgpipe
sudo apt install ./pgpipe_OLD-X.Y.Z_amd64.deb

# Restore the matching pre-upgrade state only as part of the
# reviewed rollback procedure for that release.
sudo cp /var/lib/pgpipe/pgpipe-state.db.bak /var/lib/pgpipe/pgpipe-state.db

sudo systemctl start pgpipe
pgpipe --version

Every release directory at /downloads/pgpipe/vX.Y.Z/ is kept indefinitely, so an older .deb is always reachable by URL.

Remove pgpipe safely

v5.2 separates Decommission pipeline, Remove software, and Erase local data. They affect different systems and never silently perform one another. If this host ran a pipeline, decommission it first so an abandoned slot cannot retain WAL.

1. 1

   Preview the database decommission

   sudo systemctl stop pgpipe
   sudo -u pgpipe -- /usr/bin/pgpipe teardown \
     -c /etc/pgpipe/pgpipe.yaml

   The preview is read-only. It reports the exact source identity, slot, publication, owned objects, preserved objects, and a plan digest without printing credentials.

2. 2

   Confirm the exact reviewed plan

   If the preview says it is ready, use the exact digest and slot name it printed. Omit --confirm-slot only when the preview does not request it.

   sudo -u pgpipe -- /usr/bin/pgpipe teardown \
     -c /etc/pgpipe/pgpipe.yaml \
     --execute \
     --yes \
     --operator "admin@example.com" \
     --reason "pipeline retired; CHG-1234" \
     --confirm-slot pgpipe_orders_slot \
     --plan-digest <64-lowercase-hex-characters>

   Execution rechecks the plan, refuses an active or changed slot, drops the exact inactive slot first, and removes only proven pgpipe-owned source objects. It never terminates PostgreSQL sessions, uses broad CASCADE deletion, or changes destination data and safety records.

3. 3

   Choose what happens locally

   Keep local data: Remove software

   Remove only the package when configuration and state should remain available for recovery or a future reinstall.

   # Debian / Ubuntu
   sudo apt remove pgpipe

   # RHEL / Rocky / Fedora
   sudo dnf remove pgpipe

   Both commands retain /etc/pgpipe, /var/lib/pgpipe, external TLS paths, and the stable pgpipe account.

   Discard local data: Erase local data

   This is permanent. Back up anything that must survive and complete the database decommission first. Debian has a guarded package purge; RPM requires a digest-bound local preview while the binary is still installed.

   # Debian / Ubuntu
   sudo apt purge pgpipe

   # RHEL / Rocky / Fedora: preview first
   sudo systemctl stop pgpipe
   sudo /usr/bin/pgpipe local-purge
   
   # Review the plan, then use its exact digest
   sudo /usr/bin/pgpipe local-purge \
     --execute --yes \
     --confirm-database-teardown \
     --operator "admin@example.com" \
     --reason "pipeline retired; CHG-1234" \
     --plan-digest <64-lowercase-hex-characters>
   sudo dnf remove pgpipe

   Guarded erasure refuses unknown paths, links, mounts, unsafe metadata, and live service processes. It preserves /etc/pgpipe-tls, configured external paths, and the stable service account. Do not replace it with broad rm -rf or account deletion.

4. 4

   Treat a refusal as a safe stop

   Do not bypass a guard. Correct only the named condition, rerun the read-only preview, and use the new digest. Shared, uncertain, conflicting, or legacy source ownership requires PostgreSQL administrator review; there is no safe automatic execution command for it.

   # Service running, slot active, or database plan changed
   sudo systemctl stop pgpipe
   sudo -u pgpipe -- /usr/bin/pgpipe teardown \
     -c /etc/pgpipe/pgpipe.yaml
   
   # RPM local-data plan changed
   sudo /usr/bin/pgpipe local-purge

Docker Compose teardown

For the bundled quickstart, stop the stack and delete its disposable database volumes:

# Stop containers and delete the data volumes
docker compose down -v

# Remove the working directory (Dockerfile, compose, configs)
cd .. && rm -rf pgpipe-demo

If the container used an external source, complete steps 1 and 2 with the v5.2 binary and its real configuration before deleting the container. Do not abandon an external replication slot.