Skip to content

This is the multi-page printable view of this section. .

Return to the regular view of this page.

Tutorials

End-to-end walkthroughs that take a pile of packages all the way to a signed repository your clients can install from.

Each tutorial starts from a new workspace. Commands are intended to be run in order; replace uppercase placeholders and package paths for your environment.

If you have not installed SOW yet, start with Installation and Quick Start. The tutorials below cover the managed repository path.

A managed RPM repository with per-architecture views, noarch projection, debuginfo filtering, version limits, and a working dnf client configuration.

A managed DEB repository with a Debian-style pool, by-hash indexes, and a deb822 client configuration.

Generate a dedicated GPG key, sign repository metadata and RPM packages, and configure clients to reject anything unsigned.

Serve a Repository with Nginx and publish a verified Generation to a configured filesystem target without exposing private workspace state.

Turn existing dual-architecture infra-pkg RPMs and DEBs into a real repository, then rehearse local installation, rolling updates, Stable promotion, and monthly snapshots.

Which one first

Your situation Start here
You ship RPMs to dnf clients Build a YUM Repository
You ship DEBs for Debian or Ubuntu Build an APT Repository
You need signed metadata or signed RPM payloads Sign Your Repository
The tree is built but nothing can reach it Serve Repositories
You want to turn a dual-architecture package pool into a maintained Infra repository Build the pigsty-infra Repository

The YUM and APT tutorials are independent fresh-workspace paths. A real Workspace may hold both RPM and DEB Dists in one Repository when that ownership boundary suits your operation.

Conventions used here

Shell blocks contain commands without a $ prefix so you can copy a whole block at once. Output appears in a separate block below the command, or as a comment when it is one line. Where a command needs a value you must substitute, it appears in UPPERCASE.

Every tutorial ends with a verification step. sow check returning 0 proves the selected Repository is complete and matches the recorded Generation. A nonzero result is not a release artifact.

1 - Build a YUM Repository

Create a managed RPM repository, apply membership policy, serve it, and configure dnf.

This tutorial creates a new Managed RPM repository. You need a writable directory, and one or more RPM files.

1. Create the workspace

mkdir -p /srv/sow
cd /srv/sow
sow init .
sow repo new pigsty
sow dist new el9 --format rpm -r pigsty

The Dist name is an identifier chosen by you. SOW does not infer an operating-system release from el9.

2. Set membership policy

Edit the generated sow.yml. This example keeps one version per package and architecture and excludes debug packages:

schema: sow/v3
architectures: [x86_64, aarch64]
repos:
  pigsty:
    dists:
      el9:
        format: rpm
        limit: 1
        exclude:
          - kind: [debuginfo, debugsource, llvmjit]
targets: {}

Validate every manual edit before writing repository state:

sow config check
sow config show --all

exclude runs before limit. Neutral noarch packages are projected into every enabled architecture view; they are not listed in architectures.

3. Add RPMs

sow add /path/to/packages/*.rpm -r pigsty -d el9
sow status -r pigsty
sow check -r pigsty

add parses the package headers, stores each accepted package once in the canonical pool, updates Desired membership, and materializes a new Generation. Excluded inputs are reported per item and are not command failures.

The public tree has this shape:

/srv/sow/pigsty/
├── pool/...
└── dists/el9/
    ├── x86_64/repodata/...
    └── aarch64/repodata/...

The rpm-md location href entries reach package bytes in the root pool/ by relative paths. Do not copy an architecture directory by itself: it is not a standalone repository.

4. Preview over HTTP

For a local preview:

cd /srv/sow
python3 -m http.server --bind 127.0.0.1 8080

Check the entry point from another shell:

curl --fail http://127.0.0.1:8080/pigsty/dists/el9/x86_64/repodata/repomd.xml >/dev/null

Use a maintained HTTP server for persistent service. It must expose the whole pigsty/ tree so client-resolved package URLs under pigsty/pool/ remain reachable.

5. Configure dnf

Replace REPO_HOST with an address the client can reach:

# /etc/yum.repos.d/pigsty.repo
[pigsty-el9]
name=Pigsty EL9
baseurl=http://REPO_HOST:8080/pigsty/dists/el9/$basearch/
enabled=1
gpgcheck=0
repo_gpgcheck=0

Then refresh and query the repository:

sudo dnf clean metadata
sudo dnf makecache --refresh
dnf --disablerepo='*' --enablerepo=pigsty-el9 list available

This configuration is deliberately unsigned. Enable client verification only after following Sign Your Repository.

6. Publish or export

Before delivery, require a successful deep check:

sow check -r pigsty

Use sow publish for a configured filesystem or R2 target. A whole-root copy is acceptable only into an offline staging location that is switched into service atomically; do not run an unordered in-place sync against a live repository.

Some mirroring tools, including default dnf reposync, reject rpm-md package locations that traverse to the root pool. Export a self-contained RPM leaf when such a consumer is required:

sow export rpm-leaf el9 x86_64 /srv/export/pigsty-el9-x86_64 -r pigsty

The destination must be absent or empty. The export duplicates package bytes by default; --hardlink is an explicit same-filesystem, trusted, read-only optimization.

Update the repository

sow add /path/to/new.rpm -r pigsty -d el9
sow rm PACKAGE_NAME -r pigsty -d el9
sow build -r pigsty
sow check -r pigsty

add and rm change Desired membership. build is useful after policy or signing configuration changes. check is the publication gate; status alone is not.

The automated client and platform scope is listed under Platforms & Integrations.

2 - Build an APT Repository

Create a managed DEB repository with by-hash indexes and configure an APT client.

This tutorial creates a new Managed DEB repository. You need a writable directory, and one or more DEB files.

1. Create the workspace

mkdir -p /srv/sow
cd /srv/sow
sow init .
sow repo new pigsty
sow dist new trixie --format deb -r pigsty

The Dist name becomes the APT suite. It is an identifier chosen by you; SOW does not infer distribution semantics from trixie.

2. Set membership policy

Edit the generated sow.yml if you need filtering or version limits:

schema: sow/v3
architectures: [x86_64, aarch64]
repos:
  pigsty:
    dists:
      trixie:
        format: deb
        limit: 1
        exclude:
          - kind: [dbgsym, dbg]
targets: {}

Then validate it:

sow config check
sow config show --all

SOW stores canonical architecture families in configuration and renders Debian names in the repository: x86_64 becomes amd64, aarch64 becomes arm64, and neutral all packages are included in both views.

3. Add DEBs

sow add /path/to/packages/*.deb -r pigsty -d trixie
sow status -r pigsty
sow check -r pigsty

Accepted package bytes are stored once. The public tree is:

/srv/sow/pigsty/
├── pool/...
└── dists/trixie/
    ├── Release
    └── main/
        ├── binary-amd64/
        │   ├── Packages
        │   ├── Packages.gz
        │   └── by-hash/SHA256/...
        └── binary-arm64/...

Package paths beneath pool/ are grouped by normalized source package. Packages uses archive-root-relative Filename values. SOW writes SHA-256 by-hash copies and advertises them from Release.

4. Preview over HTTP

For a local preview:

cd /srv/sow
python3 -m http.server --bind 127.0.0.1 8080

Check the entry points:

curl --fail http://127.0.0.1:8080/pigsty/dists/trixie/Release >/dev/null
curl --fail http://127.0.0.1:8080/pigsty/dists/trixie/main/binary-amd64/Packages.gz >/dev/null

Use a maintained HTTP server for persistent service and expose the complete pigsty/ tree.

5. Configure APT

Replace REPO_HOST with an address the client can reach. For an unsigned test repository, use a deb822 source with explicit trust:

# /etc/apt/sources.list.d/pigsty.sources
Types: deb
URIs: http://REPO_HOST:8080/pigsty
Suites: trixie
Components: main
Architectures: amd64
Trusted: yes

Then refresh and query it:

sudo apt update
apt-cache policy

Trusted: yes disables authenticity checking and is suitable only for a controlled test. For a signed repository, remove that line and configure a keyring:

Types: deb
URIs: https://repo.example.com/pigsty
Suites: trixie
Components: main
Architectures: amd64
Signed-By: /usr/share/keyrings/pigsty-archive-keyring.gpg

Follow Sign Your Repository before enabling Signed-By.

6. Publish safely

Require a successful deep check before delivery:

sow check -r pigsty

Use sow publish for a configured filesystem or R2 target. If you use another transport, copy the entire repository into an offline staging location and switch it into service atomically. Do not update a live dists/ tree file by file: clients may observe metadata and package state from different generations.

Update the repository

sow add /path/to/new.deb -r pigsty -d trixie
sow rm PACKAGE_NAME -r pigsty -d trixie
sow build -r pigsty
sow check -r pigsty

Use build after policy or signing configuration changes. Use check, not status alone, as the publication gate.

The automated client and platform scope is listed under Platforms & Integrations.

3 - Sign Your Repository

Sign RPM and APT metadata, optionally sign RPM packages, and enable client verification.

SOW has two independent signing paths:

Path Output Client control
RPM metadata repodata/repomd.xml.asc repo_gpgcheck=1
APT metadata InRelease and Release.gpg Signed-By
RPM package body embedded RPM signature gpgcheck=1

APT trusts package hashes through the signed Release; SOW does not re-sign DEB package bodies. Start with metadata signing. Add RPM package signing only when you own the signing policy for those package bytes.

1. Create a dedicated key

The commands below create an unencrypted example key. Use a protected key and a passphrase reference for production; see the configuration reference.

SIGNING_UID='SOW Repository <[email protected]>'
gpg --batch --pinentry-mode loopback --passphrase '' \
  --quick-generate-key "$SIGNING_UID" rsa3072 sign 2y

FPR="$(gpg --batch --with-colons --list-secret-keys "$SIGNING_UID" \
  | awk -F: '$1 == "fpr" {print $10; exit}')"
test -n "$FPR"

sudo install -d -m 0700 /srv/sow-secrets
sudo chown "$(id -u):$(id -g)" /srv/sow-secrets
gpg --batch --pinentry-mode loopback --passphrase '' --armor \
  --export-secret-keys "$FPR" > /srv/sow-secrets/repo-signing.asc
gpg --armor --export "$FPR" > /srv/sow-secrets/repo-signing.pub
chmod 600 /srv/sow-secrets/repo-signing.asc

Keep the secret key outside the Workspace’s public Repository tree and outside every web root. If a dedicated service account runs SOW, make that account—not the interactive user—the directory owner. Distribute only repo-signing.pub to clients.

2. Configure metadata signing

In /srv/sow/sow.yml, add the relevant blocks under the Repository. Omit the ecosystem you do not use:

repos:
  pigsty:
    signing:
      rpm:
        metadata:
          key: file:///srv/sow-secrets/repo-signing.asc
      deb:
        metadata:
          key: file:///srv/sow-secrets/repo-signing.asc
    dists:
      # existing Dist definitions remain here

Validate the key reference, rebuild, and run the publication gate:

cd /srv/sow
sow config check
sow build -r pigsty
sow check -r pigsty

For a protected key, add passphrase: env://SOW_METADATA_PASSPHRASE or a bounded file reference next to key. SOW never writes key or passphrase material into configuration, SQLite, JSON, or logs.

3. Verify metadata manually

Use the exact paths for your Dists and architectures:

gpg --verify \
  pigsty/dists/el9/x86_64/repodata/repomd.xml.asc \
  pigsty/dists/el9/x86_64/repodata/repomd.xml

gpg --verify pigsty/dists/trixie/InRelease
gpg --verify \
  pigsty/dists/trixie/Release.gpg \
  pigsty/dists/trixie/Release

sow check verifies the configured signing identity as part of its deeper consistency checks. Manual verification is still useful when establishing a client trust root.

4. Optional: sign RPM package bodies

Add rpm.packages only if clients require embedded package signatures:

repos:
  pigsty:
    signing:
      rpm:
        packages:
          mode: fill
          key: agent://REPLACE_WITH_THE_FINGERPRINT
        metadata:
          key: file:///srv/sow-secrets/repo-signing.asc

Replace the placeholder with the 40-hex fingerprint printed in $FPR. For this operation:

  • rpm and gpg must be installed;
  • the matching secret key must be available in the ambient GPG environment used by rpm;
  • fill preserves packages already signed by the configured or trusted_keys identities;
  • always re-signs everything not already signed by the configured identity;
  • never leaves input bytes unchanged.

SOW invokes rpm --addsign or rpm --resign on a private staged copy, not on the input file. Revalidate and rebuild after changing the policy:

sow config check
sow build -r pigsty
sow check -r pigsty

Inspect a resulting package with rpmkeys --checksig /path/to/package.rpm.

5. Enable dnf verification

Transfer the public key to the client through a trusted channel:

sudo install -m 0644 /path/to/repo-signing.pub /etc/pki/rpm-gpg/RPM-GPG-KEY-pigsty
sudo rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-pigsty

Then enable the checks that correspond to what you signed:

[pigsty-el9]
name=Pigsty EL9
baseurl=https://repo.example.com/pigsty/dists/el9/$basearch/
enabled=1
repo_gpgcheck=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-pigsty

Set gpgcheck=0 if package-body signing is not configured. Do not disable repo_gpgcheck after configuring metadata signing.

6. Enable APT verification

Install the public key as a dedicated keyring:

sudo gpg --dearmor --yes \
  --output /usr/share/keyrings/pigsty-archive-keyring.gpg /path/to/repo-signing.pub

Reference it from deb822 configuration and do not set Trusted: yes:

Types: deb
URIs: https://repo.example.com/pigsty
Suites: trixie
Components: main
Architectures: amd64
Signed-By: /usr/share/keyrings/pigsty-archive-keyring.gpg

Run apt update and treat any signature error as a failed deployment, not as a reason to weaken the client configuration.

Plain-mode RPM signing

Plain mode can sign RPM package bodies, but it does not sign repository metadata or create an APT Release:

sow create /srv/flat --sign-with 0123456789ABCDEF

The key must be exactly 16, 40, or 64 hexadecimal characters, with no 0x prefix, and the matching private key must be usable by the ambient rpm/GPG setup. Without --overwrite, already signed RPMs keep their bytes; adding --overwrite explicitly re-signs every RPM. SOW signs private staged copies before replacing package bytes and metadata.

Key changes

Changing a key reference or resolved fingerprint marks affected Dists dirty. A metadata key can be changed by distributing the new public key, rebuilding, checking, and then switching client enforcement. RPM package keys need a staged rollover: Package Objects are immutable, and build rejects stored RPMs that do not satisfy the new policy instead of re-signing them in place. Use fill with the old public key in trusted_keys until old package coordinates have been withdrawn or replaced. Finish with a real client acceptance test in the target environment.

Run the final signed repository through the exact dnf/APT versions and trust policy used in production. The automated scope is listed under Platforms & Integrations.

4 - Serve and Publish Repositories

Serve a public Repository with Nginx and publish verified Generations to a filesystem target.

SOW writes static files; it is not an HTTP server. This guide keeps the writable Workspace separate from the path Nginx serves.

Public and private paths

Mode Public unit Never serve
Plain the directory passed to sow create transient .sow-plain-stage-* output; no durable journal
Managed one Repository’s complete pool/ + dists/ tree Workspace sow.yml, .sow/, SQLite, locks, journals, staging

For the workspace in First Workspace, the source Repository is /srv/sow/local. Do not make /srv/sow the document root.

1. Gate the source Generation

cd /srv/sow
sow build -r local
sow check -r local

Continue only when check returns 0. status is useful for diagnosis, but check is the full read-only delivery proof.

2. Configure a filesystem target

Create the endpoint directory first. It must be a real, canonical directory, not a symlink; SOW refuses to create a missing endpoint for you.

sudo install -d -m 0755 /srv/repo-public
sudo chown "$(id -u):$(id -g)" /srv/repo-public

The second command gives the current operator write access; use the account that will run sow publish if publication runs under a dedicated service user.

Add a target to /srv/sow/sow.yml:

targets:
  public:
    repository: local
    provider: filesystem
    endpoint: file:///srv/repo-public
    prefix: local
    public_endpoint: file:///srv/repo-public/local/
    max_cache_ttl: 0s
    authoritative_workspace: true
    single_writer: true
    exclusive_write_authority: true

The three booleans are mandatory safety acknowledgements. The endpoint and prefix combine to /srv/repo-public/local; SOW creates and owns the prefix below the pre-existing endpoint.

Validate and publish:

sow config check
sow publish public

Publication copies immutable payloads and metadata before mutable protocol pointers, verifies the result, and records a target checkpoint. Repeating the command for an unchanged Generation is an idempotent no-op.

Do not let another tool write into the same target prefix. The target contract is single-writer and exclusive.

3. Serve the target with Nginx

server {
    listen 80;
    server_name repo.example.com;

    root /srv/repo-public;
    autoindex off;

    location / {
        try_files $uri $uri/ =404;
    }

    location ~ (^|/)\. {
        deny all;
    }
}

Reload Nginx after validating its configuration. The client URLs are:

DNF baseurl: http://repo.example.com/local/dists/el9/x86_64/
APT source:  deb http://repo.example.com/local bookworm main

If metadata or packages are signed, publish the corresponding public key separately and configure gpgkey/Signed-By; private keys never belong under the document root.

4. Verify the served entry points

curl --fail --head \
  http://repo.example.com/local/dists/el9/x86_64/repodata/repomd.xml

curl --fail --head \
  http://repo.example.com/local/dists/bookworm/Release

curl --fail --head \
  http://repo.example.com/local/dists/bookworm/main/binary-amd64/Packages.gz

Then run the actual package manager from a client host. HTTP reachability is not package client verification; both checks matter.

The complete Repository prefix must have one access policy. RPM metadata may resolve a package through ../../../pool/..., and APT Filename fields point to pool/... directly. Protecting dists/ while accidentally exposing or blocking pool/ breaks the repository.

Manual and air-gapped delivery

If sow publish cannot reach the destination:

  1. run sow check on the source;
  2. copy the complete Repository into a new, non-live staging or release directory;
  3. verify transport checksums against sow changes 0 or an archive manifest;
  4. atomically switch an operator-owned parent reference to the new directory;
  5. keep the previous release until clients and caches have moved past it.

Do not run an unordered rsync --delete directly against a live Repository root. That does not preserve SOW’s pointer ordering, target checkpoint, cache grace, or recovery state. sow changes describes Generation differences; it is not an authorization to mutate a live target without those controls.

R2 targets

provider: r2 uses an S3-compatible storage transport and report-only target GC. The transport integration covers list, HEAD, GET, and conditional PUT against a pinned MinIO fixture. Validate credentials, bucket policy, public endpoint, cache behavior, replay, and recovery on a nonproduction prefix before enabling a production target. See Platforms & Integrations.

Next

5 - Build the pigsty-infra Repository

Turn an existing dual-architecture RPM and DEB package pool into an infra repository, then validate installs, roll updates, promote to Stable, and take monthly snapshots.

pgsty/infra-pkg is the upstream build source for Pigsty Infra packages. This tutorial assumes the dual-architecture RPMs and DEBs already exist and covers the second half of the job: turning that pile of packages into a real, consumable, maintainable SOW repository named infra.

1. Put the packages under ~/repo

This tutorial uses the fixed path ~/repo throughout. Copy the existing packages into two input directories:

mkdir -p ~/repo/packages/rpm ~/repo/packages/deb
cp ~/pgsty/infra-pkg/dist/rpm/*.rpm ~/repo/packages/rpm/
cp ~/pgsty/infra-pkg/dist/deb/*.deb ~/repo/packages/deb/

Confirm that all four format-by-architecture cells contain real payloads:

find ~/repo/packages/rpm -maxdepth 1 -type f -name '*.x86_64.rpm' | wc -l
find ~/repo/packages/rpm -maxdepth 1 -type f -name '*.aarch64.rpm' | wc -l
find ~/repo/packages/deb -maxdepth 1 -type f -name '*_amd64.deb' | wc -l
find ~/repo/packages/deb -maxdepth 1 -type f -name '*_arm64.deb' | wc -l

All four results must be greater than zero. At this point, the tree is only an input package pool:

~/repo/
└── packages/
    ├── rpm/                         # x86_64 + aarch64 RPMs
    └── deb/                         # amd64 + arm64 DEBs

2. Create the infra Repository and two Dists

Initialize a Workspace and create the Repository named infra:

sow init ~/repo
cd ~/repo
sow repo new infra
sow dist new rpm --format rpm -r infra
sow dist new deb --format deb -r infra

The model is now fixed:

Repository: infra
├── Dist: rpm    format=rpm    policy=latest
└── Dist: deb    format=deb    policy=latest

Open ~/repo/sow.yml and reduce it to this configuration:

schema: sow/v3
architectures: [x86_64, aarch64]
repos:
  infra:
    dists:
      rpm:
        format: rpm
        limit: 1
      deb:
        format: deb
        limit: 1

limit: 1 keeps only the newest version for each package name and native architecture. The rpm and deb Dists are therefore rolling latest channels while still retaining both x86-64 and ARM64.

sow config check
sow config show --all -r infra

3. Ingest once and build once

Update Desired Membership first, then build a single time:

cd ~/repo
sow add ~/repo/packages/rpm --recursive -r infra -d rpm --skip
sow add ~/repo/packages/deb --recursive -r infra -d deb --skip
sow build -r infra -d rpm -d deb
sow check -r infra

Initialization is complete only when sow check returns 0. Verify the formats and architectures that SOW read from the package headers:

sow ls -r infra -d rpm -d deb --json |
  jq -r '.result.packages | group_by(.format + "/" + .canonical_arch)[] |
    "\(.[0].format)\t\(.[0].canonical_arch)\t\(length) packages"'

The output must include at least:

deb     aarch64   ... packages
deb     x86_64    ... packages
rpm     aarch64   ... packages
rpm     x86_64    ... packages

SOW reports canonical architecture names, so DEB amd64/arm64 appear here as x86_64/aarch64.

4. Read the generated filesystem tree

Print the actual directories:

find ~/repo -maxdepth 6 -type d | LC_ALL=C sort

The important structure is:

~/repo/
├── sow.yml                            # configuration; never serve it
├── .sow/                              # database, locks, recovery; never serve it
├── packages/                          # original input pool; archive as desired
│   ├── rpm/
│   └── deb/
└── infra/                             # complete public Repository Root
    ├── pool/                          # one shared payload pool for RPM and DEB
    └── dists/
        ├── rpm/
        │   ├── x86_64/repodata/
        │   └── aarch64/repodata/
        └── deb/
            ├── Release
            └── main/
                ├── binary-amd64/
                └── binary-arm64/

One path rule is easy to miss and essential to remember: SOW always places Dists below dists/. The logical infra/rpm and infra/deb channels are therefore served from /infra/dists/rpm/ and /infra/dists/deb/, while package payloads live in /infra/pool/. Always publish or mount the complete ~/repo/infra tree, never an individual Dist.

5. Serve the repository read-only with Nginx

Use the official nginx:alpine image and mount the Repository Root read-only at /infra:

docker network create --internal infra-lab
docker run --detach \
  --name infra-nginx \
  --network infra-lab \
  --publish 8080:80 \
  --volume "$HOME/repo/infra:/usr/share/nginx/html/infra:ro" \
  nginx:alpine

Check both metadata entry points directly:

curl -fsS http://127.0.0.1:8080/infra/dists/rpm/x86_64/repodata/repomd.xml | head
curl -fsS http://127.0.0.1:8080/infra/dists/deb/Release | head

Nginx can see only ~/repo/infra; it cannot see sow.yml or .sow/, and it cannot modify the repository.

6. Install an RPM from only infra on EL9

The Rocky Linux 9 container below is attached to the --internal network. The script removes every preconfigured repository and enables only infra, so a successful installation cannot fall back to a public mirror.

docker run --rm --interactive --network infra-lab rockylinux:9 bash -s <<'ROCKY'
set -euxo pipefail

rm -f /etc/yum.repos.d/*.repo
cat >/etc/yum.repos.d/infra.repo <<'REPO'
[infra]
name=Pigsty Infra RPM
baseurl=http://infra-nginx/infra/dists/rpm/$basearch/
enabled=1
gpgcheck=0
repo_gpgcheck=0
REPO

dnf clean all
dnf --disablerepo='*' --enablerepo=infra makecache
dnf --disablerepo='*' --enablerepo=infra install -y pg-exporter
rpm -q --qf '%{NAME}\t%{VERSION}-%{RELEASE}\t%{ARCH}\n' pg-exporter
command -v pg_exporter
ROCKY

An RPM baseurl must point to a concrete architecture view. dnf expands $basearch to x86_64 or aarch64.

7. Install a DEB from only infra on Ubuntu 24.04

APT points URIs at the Repository Root and uses the Dist name deb as Suites:

docker run --rm --interactive --network infra-lab ubuntu:24.04 bash -s <<'UBUNTU'
set -euxo pipefail

rm -f /etc/apt/sources.list
rm -f /etc/apt/sources.list.d/*.list /etc/apt/sources.list.d/*.sources
cat >/etc/apt/sources.list.d/infra.sources <<'SOURCE'
Types: deb
URIs: http://infra-nginx/infra
Suites: deb
Components: main
Trusted: yes
SOURCE

apt-get clean
apt-get update
apt-get install -y --no-install-recommends pg-exporter
dpkg-query -W -f='${Package}\t${Version}\t${Architecture}\n' pg-exporter
command -v pg_exporter
UBUNTU

This isolated HTTP lab temporarily disables verification. A production service should sign RPM and APT metadata and remove both gpgcheck=0 and Trusted: yes.

Docker validates the host architecture by default. To run the complete four-cell matrix, repeat each docker run once with --platform linux/amd64 and once with --platform linux/arm64; cross-architecture execution requires Docker binfmt/QEMU support. Repository inventory and client installation are separate acceptance gates.

8. Routine maintenance: add a new version

The normal update operation is add, not removing the old package first. Suppose the four new pg-exporter payloads are ready:

cp ~/pgsty/infra-pkg/dist/rpm/pg-exporter-*.rpm ~/repo/packages/rpm/
cp ~/pgsty/infra-pkg/dist/deb/pg-exporter_*.deb ~/repo/packages/deb/

cd ~/repo
sow add ~/repo/packages/rpm/pg-exporter-*.rpm -r infra -d rpm --skip
sow add ~/repo/packages/deb/pg-exporter_*.deb -r infra -d deb --skip
sow build -r infra -d rpm -d deb
sow check -r infra

Because the latest Dists have limit: 1, the new version wins and the old version automatically leaves that Dist’s Desired Membership. The old bytes are not deleted immediately, and relaxing policy later does not make the old membership reappear automatically.

Rerun the EL9 and Ubuntu clients from sections 6 and 7, refresh metadata, then install or upgrade to complete the update acceptance test.

Removal is only for hard corrections

Do not begin a normal release with sow rm. If a bad package must be withdrawn, use sow ls -r infra -d rpm --json (or the corresponding -d deb) to find its exact SHA-256, run sow rm sha256:... -r infra -d rpm --check, inspect the plan, then run the same command without --check. rm removes only Dist Membership; conservative sow gc handles pool bytes separately. Avoid a bare package name that could remove every version and architecture.

9. Two retention layers: latest and stable

limit: 1 makes rpm and deb good rolling channels but cannot express “keep every formally released version.” Create two more Dists for that purpose:

cd ~/repo
sow dist new rpm-stable --format rpm -r infra
sow dist new deb-stable --format deb -r infra
sow config show --all -r infra -d rpm-stable -d deb-stable

The new Dists default to limit: 0, which means retain every version. The resulting policy is:

Dist Format limit Role
rpm RPM 1 RPM latest
deb DEB 1 DEB latest
rpm-stable RPM 0 Accumulate promoted RPMs
deb-stable DEB 0 Accumulate promoted DEBs

Stable does not automatically resurrect every historical object that happens to remain in the pool. It starts accumulating versions that you explicitly promote from this point forward.

10. Promote latest into stable

SOW does not expose a dedicated promote command. The reliable current procedure is to freeze writers, export the exact source Dist Membership, and add those objects to the target Dist. Inputs come directly from infra/pool; SOW verifies and reuses each existing Package Object without repackaging it or storing a second copy.

First require clean source state and save the promotion manifests:

cd ~/repo
sow check -r infra
mkdir -p ~/repo/manifests

sow ls -r infra -d rpm --json |
  jq -r '.result.packages[].pool_path' > ~/repo/manifests/rpm-latest-202608.list
sow ls -r infra -d deb --json |
  jq -r '.result.packages[].pool_path' > ~/repo/manifests/deb-latest-202608.list

Pause writes to rpm and deb until promotion finishes, then reuse the pool objects:

cd ~/repo
(
  set -e
  while IFS= read -r pool_path; do
    sow add "$HOME/repo/infra/$pool_path" -r infra -d rpm-stable --skip
  done < ~/repo/manifests/rpm-latest-202608.list

  while IFS= read -r pool_path; do
    sow add "$HOME/repo/infra/$pool_path" -r infra -d deb-stable --skip
  done < ~/repo/manifests/deb-latest-202608.list

  sow build -r infra -d rpm-stable -d deb-stable
  sow check -r infra
)

Each add should report reused. If the loop stops midway, the source Dists are unchanged; correct the failure and rerun the same manifest. Over subsequent promotions, rpm/deb retain only the latest version while rpm-stable/deb-stable accumulate release history.

11. Take the 2026-08 snapshot from stable

A client-visible monthly snapshot is another pair of Dists:

cd ~/repo
sow dist new rpm-202608 --format rpm -r infra
sow dist new deb-202608 --format deb -r infra
sow config show --all -r infra -d rpm-202608 -d deb-202608

Pause stable writes during the snapshot window and first persist its exact Membership as manifests:

sow check -r infra
sow ls -r infra -d rpm-stable --json |
  jq -r '.result.packages[].pool_path' > ~/repo/manifests/rpm-stable-202608.list
sow ls -r infra -d deb-stable --json |
  jq -r '.result.packages[].pool_path' > ~/repo/manifests/deb-stable-202608.list

Add those manifests to the corresponding snapshot Dists:

cd ~/repo
(
  set -e
  while IFS= read -r pool_path; do
    sow add "$HOME/repo/infra/$pool_path" -r infra -d rpm-202608 --skip
  done < ~/repo/manifests/rpm-stable-202608.list

  while IFS= read -r pool_path; do
    sow add "$HOME/repo/infra/$pool_path" -r infra -d deb-202608 --skip
  done < ~/repo/manifests/deb-stable-202608.list

  sow build -r infra -d rpm-202608 -d deb-202608
  sow check -r infra
)

Also retain the complete verified Repository Generation so later GC treats it as a safety root:

sow retain add "$(sow status -r infra --json | jq -r '.result.built_generation')" -r infra
sow retain ls -r infra

retain protects a whole Repository Generation for recovery and GC; the fixed client-visible URLs are still provided by the rpm-202608 and deb-202608 Dists. SOW does not enforce Dist immutability, so never running add or rm against snapshot Dists after creation is part of the operating contract.

12. Client address map

One Nginx service and one shared infra/pool support every channel:

Channel dnf baseurl APT URIs / Suites
latest http://infra-nginx/infra/dists/rpm/$basearch/ http://infra-nginx/infra / deb
stable http://infra-nginx/infra/dists/rpm-stable/$basearch/ http://infra-nginx/infra / deb-stable
2026-08 http://infra-nginx/infra/dists/rpm-202608/$basearch/ http://infra-nginx/infra / deb-202608

Run the final acceptance checks:

cd ~/repo
sow dist ls -r infra
sow status -r infra
sow check -r infra

The result is not a disposable demo directory. It is a real Infra Repository that can continue ingesting packages, promoting releases, and producing monthly snapshots: rpm/deb move quickly, rpm-stable/deb-stable accumulate formal history, monthly Dists provide fixed endpoints, and every view reuses the same immutable package objects.

Stop the temporary service when the lab is complete:

docker rm --force infra-nginx
docker network rm infra-lab