Skip to content

This is the multi-page printable view of this section. .

Return to the regular view of this page.

Design Records

Why SOW uses its current repository layout and publication workflow, and how the design changed.

Design records explain the choices behind SOW’s repository layout, state model, and publication workflow. Article dates record when a decision was documented; modification dates record later revisions.

Start with these current design explanations:

The historical series follows the v0.1 program, its design decisions and tests, the v0.2 reset, and later releases. It links to dated source snapshots. Use SOW Docs for current commands and release notes for version-specific changes.

1 - Coordinated Publication Decision

The v0.4.0 decision to keep SOW’s native publication providers, and the earlier rclone proposal that did not ship.

SOW 0.4.0 kept the native filesystem and r2 publication backends. An earlier proposal used rclone and added remote audit and preview commands. The native design was selected because SOW needs to track each conditional write and its result.

Current command syntax is in sow publish.

Why keep native backends

A publication plan binds each file to an expected SHA-256 and operation. The backend must create immutable objects without overwriting existing content and update mutable pointers conditionally. SOW records the result so it can resume after an interruption.

A bulk-copy tool would need another identity and checkpoint model to preserve these rules. The 0.4 work instead improved the existing backend:

  • conditional multipart uploads;
  • separate response-header and body-idle timeouts;
  • retries for operations that can be repeated;
  • verification of changed files through the public URL.

This retained the existing publication state and avoided a transport migration.

Publication workflow

The native backend uploads packages and immutable metadata before updating the repository indexes. It then checks the public files and records a checkpoint for the target. The next publication compares the current build with that checkpoint.

Change target settings

A binding revision records changes to the target name, public URL, or cache lifetime. The target keeps its publication history. A different storage location requires a new target.

Resume an interrupted publication

SOW saves the plan and progress for each attempt. Before the commit decision, it can abandon the attempt after checking that the indexes have not changed. After that decision, recovery completes the saved plan. This gives each target one history for upload, verification, and recovery.

Configure serving separately

The operator configures HTTP access, caching, and client trust. SOW verifies publication through the configured public endpoint.

See Publication and recovery for the complete sequence. Use sow publish for configuration and commands.

2 - SOW Design Evolution: From Route Graphs to the Repository Core

How SOW moved from the v0.1 distribution platform to a repository engine, and how v0.2 through v0.5 refined that model.

SOW moved from a Git-based distribution control plane in v0.1 to a repository engine in v0.2. This article traces that change and the refinements through v0.5.0.

Version numbers below refer to released Git tags. Some archived plans used “v0.2” and “v0.3” for development designs. In particular, the C2 layout with RPM hardlinks inside each view was replaced before v0.2.0 shipped.

Four articles cover the earlier work in detail:

The short history

Release Main changes Public layout
v0.1 Git/CAS state, Routes, Edge authorization, cloud workflows, and Pigsty migration Retired
v0.2 Plain/Managed modes, one package pool per Repository, metadata-only views, retained Generations, and target publication Introduced the current pool/ + dists/ layout
v0.3 Removed V1, cached package facts, simplified Plain builds, and batched payload commits Unchanged
v0.4 Single-pass verification, independent RPM trust rings, target rebind, and migration/recovery fixes Unchanged
v0.5 Package-path reservations, removal and recovery fixes, signing checks, and reproducible Plain timestamps Unchanged

The source tags preserve each implementation: v0.1.0, v0.2.0, v0.3.0, v0.4.0, and v0.5.0.

v0.1: useful questions, too many product concepts

The first plan covered Git-backed state, content-addressed storage, Routes and Views, snapshots, remote providers, Edge authorization, and legacy migration. Forty-four ADRs, numbered through 0045, documented the decisions and their failure cases.

The combined scope made ownership difficult to explain. Several objects could describe the same package bytes and published paths. Local repository generation also depended on cloud, Edge, and migration concepts.

The v0.2 reset removed Git as the authoritative product database, the separate CAS layer, the Route/View/Snapshot command model, and Edge access control. It made local repository construction the core task.

v0.2: establish a small ownership model

v0.2 split the product into two execution paths:

  • Plain scans a caller-owned directory and rebuilds RPM or DEB metadata in place.
  • Managed owns a Workspace containing independent Repositories. Each Repository has its own Pool, Dists, SQLite state, locks, and recovery records.

This reset introduced the terms still used today: Package Object, Membership, Desired and Built state, Generation, and Changeset. Stable lock files coordinate access. Generated files replace the protocol pointers last. Managed journals record how to recover an interrupted operation.

The first implementation, called C2, still placed package hardlinks in each RPM view. That layout changed before release.

v0.2.0: make Repository root the delivery unit

The single-payload decision was written on 2026-08-05 and shipped in v0.2.0 on August 8. It removed package aliases from RPM views. One Repository has one canonical pool/ and metadata-only dists/. The complete root is the unit to copy, host, authorize, and publish.

v0.2.0 also separated local and remote state:

  • Package Objects, Desired/Built state, Generations, Changesets, and local retention belong to the Repository.
  • Publication Attempts, Applied Checkpoints, remote inventories, cache grace, and deletion records belong to one Repository and one target prefix.

Repositories do not deduplicate packages across their boundaries. A target prefix has one owner. Compatibility exports are separate copies outside the canonical Repository.

v0.3: simplify and optimize the same model

v0.3 kept the v0.2.0 public layout. It removed the V1 runtime and migration harness, introduced the package-facts cache, and made Plain generation a rebuildable one-pass operation. It also expanded Membership in batches and grouped payload commits to reduce filesystem work.

v0.4: strengthen verification and recovery

v0.4 made Managed deep verification a bounded single pass. RPM verification must succeed within one trust ring; it cannot combine partial results from unrelated rings. An explicit rebind operation can update mutable target settings. Migration, publication recovery, and public-delivery checks also became stricter.

An early coordinated-publication proposal considered using rclone for bulk transfer. The final design kept native providers so SOW could check object identity, record receipts, control commit order, and recover publication from the same state machine.

v0.5: preserve package URLs and committed state

v0.5.0, released on 2026-09-29, kept the same public layout and upgraded managed state to schema 13. The new pool-path indexes prevent different package bytes from taking over an existing URL, including a URL recorded by an earlier publication.

The release also fixed recovery after Dist removal and package re-addition, preserved committed changes after interruption, and checked conflicting signing policies before changing Desired state. Plain create --metadata-timestamp added an explicit timestamp for reproducible RPM metadata; its default is zero. See the v0.5.0 changelog for the full changes.

What survived every redesign

Several v0.1 rules continue in the smaller model:

  • Give each durable fact one owner.
  • Keep package bytes separate from rebuildable metadata.
  • Prepare immutable payload and metadata before replacing protocol pointers.
  • After a durable commit intent, complete the operation through forward recovery.
  • Check references and exact identity before deletion. Remote deletion also checks cache grace and provider capabilities.
  • Record compatibility results with the source version, client, provider, and environment that produced them.

The current rationale is in Design Principles, System Model, Single-Payload Repository, and Publication & Recovery. Use SOW Docs for commands and operating procedures, and Release Notes for version-specific changes.

3 - Publication & Recovery

How SOW uploads a repository, resumes interrupted publication, and removes unused files.

A build creates a local snapshot (Generation). Publication uploads its files to a target and records progress for recovery. Each target keeps a checkpoint that identifies its last completed publication.

State ownership

Repository records Target records
Packages, Desired state, and Built state Publication attempts and checkpoints
Generations and changesets Remote inventory
Retained package and metadata references Grace periods and deletion records

Publication order

freeze the plan
  -> upload packages
  -> upload immutable metadata
  -> record commit intent
  -> update protocol pointers, one view at a time
  -> verify storage and public URLs
  -> record the applied checkpoint

Package uploads create new objects or reuse identical bytes. They cannot overwrite an immutable path with different content. SOW records commit intent before it changes the first mutable APT index alias or protocol pointer.

Within a view, referenced content is uploaded before the pointer:

RPM: checksum-named metadata -> repomd.xml.asc -> repomd.xml
APT: by-hash and direct indexes -> Release.gpg -> InRelease/Release

During an update, views can briefly use indexes from different builds. A detached signature and its index also change in separate writes. Clients can therefore need to retry an index refresh. Public verification completes before SOW records the checkpoint.

Public verification

SOW downloads changed files and affected index entry points through public_endpoint, then checks their contents. Unchanged packages do not need another download.

The public endpoint can still return cached files. SOW waits and retries within max_cache_ttl. If verification fails, SOW does not record a new completed checkpoint.

Resume or abandon an attempt

Rerun the original sow publish TARGET command to resume. Recovery follows the saved plan rather than starting from a newer local build.

Saved state Next action
Before commit intent Retry publication, or use --abort to reconcile and abandon the attempt
After commit intent Retry publication to complete the saved update
Checkpoint recorded Finish remaining maintenance; a completed target needs no update
Conflicting file identity Inspect the named file and target configuration before retrying

--abort removes eligible private filesystem staging. It keeps remote objects and records abandoned uploads so a later attempt can reuse matching bytes.

Target identity

Storage identity consists of the provider, endpoint, region, and bucket. Target identity also includes the prefix. The first publication binds these values to the Repository. Later attempts use that binding to find the correct history.

To change the target name, public_endpoint, or max_cache_ttl, edit sow.yml and run sow publish TARGET --rebind. SOW appends a binding revision. Changes to the Repository, storage identity, or prefix require a new target.

Before changing max_cache_ttl, complete pending target maintenance. For filesystem targets, finish conditional deletion before changing public_endpoint. A shorter TTL does not shorten an already recorded grace period.

Remove a published view

A configured target with an applied checkpoint prevents withdrawal of its existing Dist, architecture, or signature pointers. To retire that target, remove its entry from targets in sow.yml, then run sow config check. This keeps the old remote files and publication history; it does not delete the remote repository.

You can then change the local Dist configuration and rebuild. To publish the reduced set of views, configure a new target with a different prefix and update clients to its URL. --rebind cannot change a target prefix. See sow publish for the configuration rules.

Retain and collect packages

A retained Generation stores metadata, manifests, and references to packages in the shared Pool. Local garbage collection checks all of these references:

  • current Desired and Built membership;
  • retained Generations;
  • active operations and recovery journals;
  • publication recovery and grace periods.

It removes a package only when no reference remains and the file still matches the recorded identity. Keeping a Generation therefore keeps its packages without making another package tree.

Filesystem target deletion requires a known inventory, exclusive ownership, and an expired grace period. SOW conditionally deletes the recorded file, checks public absence, then records the result. R2 target GC reports candidates and keeps the objects.

4 - System Model

The objects and state transitions that connect packages, distributions, generations, and publication targets.

SOW uses configuration for repository settings, SQLite for managed state, and a public directory for client access. The model separates the packages you select from the files that have been built and published.

Objects

Workspace
├── Repository
│   ├── Pool: package objects
│   ├── Dists: package memberships and policies
│   ├── Desired state
│   ├── Built Generation
│   └── Retained Generations
└── Publication targets
    └── Repository + storage backend + prefix

Workspace

A Workspace contains sow.yml, the private .sow/ directory, and stable lock files. It groups configuration and lets commands discover the selected Repository.

Repository

A Repository owns one pool/, one dists/ tree, one state database, and one Generation sequence. Package deduplication is local to this Repository. Two Repositories with the same input packages each keep their own copy.

Package object

SOW identifies a package object by its SHA-256 after package signing. Name, version, release, and architecture describe the package; the digest identifies its bytes. Re-adding the same object reuses it. Different bytes at the same Pool path are a conflict.

Dist and membership

A Dist selects packages and settings for one format, RPM or DEB. A package can belong to several Dists while keeping one Pool file. Architecture-neutral packages (noarch or all) appear in each matching architecture view without another package copy.

Desired, Built, and Generation

Desired state records the selected packages and configuration. Built state records the last completed build. A Generation is an immutable inventory of that build, and a changeset compares two inventories.

An add --skip operation can change Desired state without rebuilding indexes. An interrupted build can also leave unfinished work. Use sow status to inspect the difference and sow build to apply pending changes or finish recovery.

Publication target

A target connects one Repository to a storage backend and prefix. Each target has its own publication attempts, checkpoints, inventory, and cleanup records.

See Publication and recovery for target identity and configuration changes.

State flow

package input
    -> inspect, sign if configured, and hash
    -> package object and membership
    -> Desired state
    -> render and validate
    -> Built Generation and changeset
    -> target publication attempt
    -> applied checkpoint

Managed operations record their progress in transactions and journals. Recovery uses these records to finish or roll back the interrupted step.

Public and private files

Serve as a complete Repository Keep in the private workspace
<repo>/pool/ sow.yml
<repo>/dists/ .sow/ databases and journals
Generated indexes and signatures Locks, staging, and recovery files

A copy of the public tree can serve packages. To continue managing that Repository, keep the matching private workspace too. A public copy alone has no membership, retention, or publication history.

Locks and writers

The Workspace and each Repository have separate locks. Most reads use shared locks; writes use exclusive locks to prevent concurrent changes to the same state. sow status queries the lock state directly so you can inspect an active writer.

One workspace must own writes to each target prefix. These locks coordinate local processes, not independent workspaces on different hosts.

5 - Design Principles

Why SOW separates package storage, repository builds, and publication.

SOW separates package storage, repository builds, and publication. These principles explain where each kind of state belongs and how operations use it.

One owner for each record

Record Owner
Package bytes and identity Repository
Desired membership and Built state Repository
Generations and changesets Repository
Target bindings and revisions Repository and target
Publication attempts and checkpoints Repository and target prefix
Remote inventory and deletion records Repository and target prefix

Each Repository stores its own copy of a package. Each publication target stores its own copy too. This keeps package retention and deletion independent across repositories and targets.

Store packages; rebuild indexes

In Managed mode, pool/ holds the package bytes. In Plain mode, the input directory holds them. Indexes and architecture views describe those packages and can be rebuilt.

Managed garbage collection removes a package only after its active, retained, and recovery references are gone. Plain mode rebuilds indexes directly from the current directory.

Serve the complete public tree

A Managed Repository root contains pool/ and dists/. Copy or serve that complete root so clients can resolve package paths from the indexes. Use sow export rpm-leaf when a consumer needs a self-contained RPM architecture directory.

Keep the workspace’s sow.yml and .sow/ outside the HTTP root. They contain configuration, databases, locks, journals, and recovery files.

Upload packages before indexes

Publication uses this order:

packages -> immutable metadata -> protocol pointers -> grace period -> eligible deletion

Protocol pointers include repomd.xml, Release, and InRelease. SOW uploads referenced files before updating these pointers. Old files remain through the grace period so clients with cached indexes can still use them.

Recover according to the recorded state

After an interrupted Plain build, run sow create again. It scans the package directory and rebuilds the indexes.

A Managed publication records its plan and progress. Before commit intent, SOW can reconcile an attempt and abandon it. After commit intent, rerun sow publish TARGET to complete the recorded update. Conflicting file identities stop recovery and identify the object to inspect.

Test the layout with its consumers

A package manager resolves package URLs; a mirror tool can also impose rules on its output directory. Test the complete Repository and an exported RPM leaf separately. Record the client version and layout with each result. The compatibility reference lists automated coverage.

Keep coordination local

Use one workspace as the writer for each target prefix. SOW coordinates local processes with locks. It does not arbitrate between independent workspaces.

R2 publication supports incremental uploads. R2 garbage collection reports candidates and keeps remote files because this backend does not implement the required conditional deletion.

6 - Why SOW Uses One Payload Tree per Repository

One package pool per repository, with metadata-only distribution views that share its files.

This decision was drafted on 2026-08-05 and shipped in SOW v0.2.0 on 2026-08-08. SOW 0.5.0 still uses this layout. Each Repository stores one package tree and lets its distribution views reference that tree.

Decision

Each SOW Repository owns one public tree:

<repository>/
  pool/                         # package files
  dists/
    <dist>/
      <architecture>/           # RPM metadata only

The diagram shows an RPM view. APT indexes use dists/<dist>/main/binary-<architecture>/; its Release files sit at dists/<dist>/.

A package object has one payload path inside one Repository. Adding the same object to several Dists or architecture views adds membership and metadata references.

The complete pool/ + dists/ root is the supported copy, static-hosting, authorization, and publication unit. A single architecture leaf is a client view, not a complete package repository.

From the C2 development layout to v0.2.0

An earlier pre-release C2 development layout stored each package once in a root Pool, then created package hardlinks inside RPM views. This gave selected reposync clients a self-contained leaf without duplicating local bytes.

Hardlinks saved local disk space, but had costs elsewhere:

  • object storage expands each path into a separate key, so aliases duplicate payloads;
  • archives and cross-filesystem copies can expand or lose hardlink identity;
  • a package path inside a view looks independent, although the Repository still owns the file;
  • retention and garbage collection can confuse link counts with package references;
  • the repository layout has to follow a particular mirror tool’s output-directory rules.

Before v0.2.0 was tagged, the final single-payload design removed package aliases from RPM views. The archived C2 documents describe that development line.

How metadata reaches the Pool

APT already treats Filename as an archive-root-relative path, so a package entry names its pool path directly:

Filename: pool/p/pkg/pkg_1.0_amd64.deb

For RPM, each <location href> is computed from the actual view directory to the Pool path:

view:    dists/el9/x86_64/
payload: pool/p/pkg/pkg-1.0-1.x86_64.rpm
href:    ../../../pool/p/pkg/pkg-1.0-1.x86_64.rpm

The renderer and checker share the same path function. It calculates the relative path and escapes the file name. It then resolves the result and checks that it points to the original package inside the Repository.

This keeps the public tree relocatable. Keep the relative locations of pool/ and dists/ when you copy the repository. Then the same metadata works under another filesystem directory, HTTP origin, bucket, or prefix.

Rejected alternatives

Alternative Why it was not used
Package hardlink/copy in every RPM view Creates path aliases and duplicate object keys
Symlinked payloads Object storage does not support them; static servers and archives handle them differently
Leading /pool/... href Different clients interpret it as host-root or repository-relative; it also breaks non-root prefixes
Absolute URL or xml:base Binds metadata to one origin and prevents unchanged relocation
CDN rewrite or redirect objects Requires extra routing configuration for the repository
Workspace- or bucket-global pool Requires cross-Repository coordination for locks, retention, and deletion

Mirror-tool compatibility is an export concern

An ordinary package client only needs to resolve metadata references and fetch package bytes. A mirror tool may additionally insist that every package be stored below the view directory it was given.

For consumers that require a standalone RPM leaf, use sow export rpm-leaf. It creates a directory outside the Repository with package files and regenerated local metadata. Copy is the default. Use --hardlink on the same filesystem and keep exported package files read-only, because writing through a hardlink also changes the Pool file. The export is not a Generation, publication source, membership owner, or garbage collection root.

Client and mirror-tool results remain in the compatibility matrix. Use sow check to verify local paths, digests, and file references. Use the listed client tests to check package installation and mirroring.

Consequences

  • Package identity, membership, Generation, retention, and local GC stay Repository-scoped.
  • Publication attempts, checkpoints, remote inventory, grace, and deletion evidence stay target-prefix-scoped.
  • Each package file maps to one static object key.
  • Before deleting a payload, check its references and file identity. Do not use hardlink count to determine whether it is still needed.
  • Copy the Repository root, or use the export command to create a standalone RPM architecture directory.

The current mechanism is documented in Pool & Metadata Views, while System model and Publication and recovery define the surrounding ownership and lifecycle rules.

7 - The v0.2 Reset: Plain and Managed Instead of a Universal Control Plane

How SOW replaced the v0.1 Git/CAS/Edge platform with two small execution modes, explicit ownership, SQLite state, and a repository-focused release.

On 2026-07-31, after sealing the v0.1 source baseline, SOW began a product reset. The goal was a small repository engine that could replace Pigsty’s daily package-build workflow. It kept the earlier lessons about integrity and recovery while removing much of the control plane.

The result was the split still used today: Plain rebuilds caller-owned directories; Managed maintains repositories and their history.

The development design initially used C2 hardlinks inside RPM views. These were replaced before the v0.2.0 release on August 8. Some archived plans called the C2 build “v0.2.0,” but the released tag already had one canonical Pool and metadata-only views.

Why reset instead of incrementally simplifying V1

The v0.1 model had coupled five different jobs:

  1. build valid APT/YUM/asset repositories;
  2. maintain package history and snapshots;
  3. migrate a large legacy Pigsty topology;
  4. publish to multiple cloud/CDN targets;
  5. enforce commercial Edge access.

Removing one feature at a time would leave the same Git/CAS/Route ownership model in place. The reset instead asked which facts the core repository engine truly needed to own.

The first phase excluded:

  • the V1 Git database, Route/Manifest/Quarantine model, or global content-addressed product layer;
  • upstream sync, pull/push/copy/move/promote surface, Edge entitlement, CDN bootstrap, or commercial topology;
  • remote publication/state, object storage, GC, snapshot/freeze, Managed export, or serving in the first phase;
  • modulemd, source-package building, Web UI, cross-Repository deduplication, or multi-writer operation.

Upstream synchronization and channel promotion were deliberately removed even though V1 had real streaming, fuzz, and deterministic-proof evidence for them. Package acquisition became an external task. Operators can prepare package sets with dedicated mirror or download tooling, then let SOW own admission, membership, build, and publication. Some phase-one exclusions—publication, retention, GC, and external export—returned before the final v0.2.0 tag under the smaller Repository ownership model.

Two isolated execution modes

Plain: rebuild what the caller owns

Plain mode operates on a caller-owned directory of RPM or DEB files:

sow create /srv/repo

It scans packages, renders protocol metadata, validates the result, and installs pointers last. Plain owns its generated metadata but not a Workspace, Repository database, Generation history, or long-lived membership state.

Because the desired state is simply “the packages currently in this directory,” the correct recovery strategy is to rerun the build. Later releases made that principle even more explicit by removing Plain journals and repeated package reads.

Managed: own lifecycle and history

Managed mode introduces a Workspace with independent Repositories:

Workspace
├── sow.yml
├── .sow/                    private state, locks, SQLite, journals
└── <repository>/
    ├── pool/                canonical package bytes
    └── dists/               client protocol views

Each Repository owns its Pool, Package Objects, Membership, Desired State, Built Generation, Changeset, lock, database, and recovery state. Different Repositories do not silently deduplicate or share counters.

A Dist is a named RPM or DEB membership set. Architecture Views are projections of that membership, not new owners. Neutral packages (noarch / all) project into every applicable architecture without duplicating logical membership.

Decisions that replaced V1 concepts

V1 concept v0.2 replacement
Git commit/ref as canonical state Per-Repository SQLite plus explicit filesystem/journal evidence
Global CAS and Route materialization Repository-owned Pool and protocol views
View/Snapshot/Stable ref graph Dist membership, Desired revision, Built Generation, retained state
One universal command interface Separate Plain and Managed commands
Implicit legacy topology Strict sow.yml plus explicit names and selectors
Cross-product publication saga Repository build kept separate from target publication
Reuse V1 packages directly Narrow parser/renderer/version/signing ports only

The critical simplification was that private control state and public repository bytes could be understood without Git or an Edge router.

Desired versus Built

Managed mode made interrupted state explicit:

  • Desired State is the current package and membership intent in SQLite.
  • Built Generation is the last complete public tree that rendered and validated successfully.
  • Changeset is the exact delta between two built states.

Desired can advance before a new Built Generation completes. The operation journal records unfinished work, and status shows the difference between Desired and Built. Recovery uses the recorded operation phase to decide what remains to be completed.

P0 through P3

The work was intentionally sequenced:

Phase Contract
P0 Plain create, real RPM/DEB metadata, signing, Pigsty-compatible filtering, pointer-last replacement
P1 Workspace/Repository/Dist lifecycle, strict config, package identity, membership, and query
P2 Add/remove/policy/build operations, journals, crash recovery, and safe mutation
P3 Check, Changeset, handoff, scale, compatibility, and clean-delivery evidence

Each phase had an API contract, implementation specification, adversarial review, and acceptance record.

The first Managed RPM layout kept canonical files in a root Pool and added hardlinks below each architecture View. Metadata used a safe local pool/... href. This C2 layout made a pinned AlmaLinux 9 reposync flow work and avoided duplicate local bytes because aliases shared an inode.

The design had a hidden deployment cost:

  • object storage expands each path into a separate full object;
  • archive and cross-filesystem copies can lose hardlink identity;
  • every Dist/architecture alias looks like another canonical payload;
  • retention and GC risk depending on inode/link-count behavior;
  • the whole Repository no longer maps one-to-one to static object keys.

The 2026-08-05 snapshot records this C2 implementation and its local test results.

The single-payload contract was written on 2026-08-05, committed as the “next” design on August 7, and implemented before the August 8 release tag. The final release therefore shipped:

<repository>/
  pool/       one canonical payload per Package Object
  dists/      metadata-only APT/RPM views

Default EL reposync became an accepted compatibility limitation; a standalone RPM leaf moved to an explicit external export. The complete reasoning is preserved in Single-Payload Repository.

What the reset reused

The team did not start from blank code. It retained only narrow components that had no V1 ownership assumptions:

  • RPM/DEB parsing and native version comparison;
  • metadata rendering, compression, signing, and validation;
  • safe filesystem primitives;
  • selected client fixtures and fault patterns;
  • exact package identity and provenance checks.

The new core had no dependency on Git state, Route receipts, Edge components, legacy migration, or cloud control code. Those components were kept for reference or removed.

From acceptance tests to release

The development line recorded P0/P1 acceptance on August 1, P1–P3 acceptance on August 2, and a local release-gate run on August 5. The final v0.2.0 source was tagged on August 8 after the single-payload implementation, packaging, and CI corrections landed.

The final release also shipped filesystem and R2 publication, retained generations, external RPM leaf export, and local GC. R2 remote deletion was disabled from the start; GC for R2 only reported candidates.

What still defines SOW

  • Plain and Managed are isolated modes with different recovery costs.
  • Workspace is configuration/discovery scope; Repository is package/history scope.
  • Package Object identity is immutable; Membership is logical and many-to-many.
  • Desired and Built states are distinct.
  • Writers use stable lock files and install protocol pointers last. Managed journals record interrupted operations; Plain recovers by rebuilding.
  • CLI syntax, JSON envelopes, and exit classes provide a defined automation interface.
  • Protocol, client, mirror-tool, and provider compatibility are reported separately.

The current forms are documented in Get Started, System Model, and Design Principles.

Primary source snapshots

Continue with Design Evolution for the v0.3 through v0.5 changes that followed.

8 - What SOW v0.1 Actually Proved

A summary of 112 dated v0.1 reports covering client compatibility, scale, recovery, migration, and cloud-provider behavior.

The v0.1 archive contains 112 dated reports from 2026-07-11 through July 31. They record client tests, scale measurements, failure recovery, migration, and cloud-provider experiments. This article summarizes the main results and links to representative reports.

Each result belongs to the source revision, fixtures, clients, and environment recorded in its report. Current compatibility is listed in the compatibility reference.

Evidence timeline

Window Reports What the work concentrated on
2026-07-11 3 Real APT/DNF client baseline, 72,310-package manifest scan, 50k upstream streaming
2026-07-12 35 APT/YUM protocol behavior, CAS/GC, publication ordering, recovery, MinIO, performance, migration, build/supply-chain gates, and cloud harness design
2026-07-13 2 RPM trust closure and selected-set materialization recovery
2026-07-14–16 14 Legacy topology, family migration, physical ownership, full-copy adoption, and package-signature inventory
2026-07-17–20 39 Cloudflare/R2/COS/Edge safety, provider attestation, cutover receipts, configuration bounds, and checkpoint-fenced deletion
2026-07-22 10 Descriptor/path identity across stage, apply, prune, rollback, and residue cleanup
2026-07-26–29 7 Recursive durability, hostile-writer limits, replacement outcomes, preserved audits, and provider corrections
2026-07-30–31 2 Clean-room APT/YUM/asset MVP and final legacy-source baseline

What had strong supporting evidence

Real package clients and large repositories

The first evidence set used real APT and DNF clients rather than validating only generated XML or text. It also measured scale explicitly:

These measurements supported streaming processing and bounded concurrency for large repositories.

Protocol limits found in testing

Failed experiments identified limits in the proposed protocols:

These experiments showed why protocol syntax and client transactions need separate tests.

Failure and recovery

The archive exercised interrupted sync, publication, materialization, snapshot, archive, and remote-restore paths. Later reports bound file operations to descriptors and exact path identities, then covered rollback, quarantine, and preserved evidence.

Representative records include:

These tests led to distinct recovery paths: abandon uncommitted work, complete work after commit intent, or stop when recorded evidence conflicts.

Legacy migration

The migration evidence enumerated old Make targets, physical layouts, repository families, consumer configuration, package-signature inventory, and rollback. It used read-only copies and exact receipts before mutation.

This work validated a specific Pigsty cutover path. It also established a method: record migration assumptions as data, check them before each step, and retire the migration code after the cutover.

Provider and Edge experiments

The archive contains real and simulated R2, MinIO, Cloudflare, COS, and Edge tests. Representative reports include:

Some tests used designated nonproduction resources; others ran offline or against mocks. The reports distinguish these environments. Storage behavior measured on R2, MinIO, or COS applies to that service and the API tested.

What became part of the design method

The work established six practices:

  1. State what was tested. Distinguish source checks, simulated failures, client tests, and tests against a real service.
  2. Keep failed test results. They identify conditions where the design does not work.
  3. Record inputs with each result. Source revision, container/client version, provider, route, and fixture belong with the result.
  4. Measure scale. Large repositories need measured memory, call count, and concurrency behavior.
  5. Test each new layout. Keep historical results with the model they tested.
  6. Check each release build. Test supported platforms, scan dependencies, and run static analysis. Check that the build can be reproduced.

These practices now appear in Design Principles and the current compatibility reference.

Primary source

The immutable v0.1.0 evidence directory contains every dated report, including commands, results, and test conditions.

Continue with The v0.2 Reset to see how those lessons were kept while the product model became much smaller.

9 - SOW v0.1 Decision Ledger: What Survived and What Was Retired

All 44 surviving v0.1 architecture decisions: their original purpose and how each changed after the repository model reset.

The v0.1.0 tag contains 44 architecture decision records (ADRs), numbered 0001 through 0045; 0006 is absent. Each ADR records a problem, the decision made at the time, and the conditions that implementation had to meet. This table shows what happened to those decisions after the repository model changed.

Fate Meaning
Retained The principle remains in current SOW with substantially the same boundary.
Evolved The problem remains, but ownership or implementation changed.
Migration history The decision governed the Pigsty/V1 cutover.
Retired The decision belonged to the removed Git/CAS/Route/Edge model.

Decision dates run from 2026-07-11 through July 28. ADR-0035 also has amendments from July 29 and July 31. Most early ADRs entered Git together on July 19, after implementation had begun. ADR-0032 is summarized without a direct link because it names obsolete infrastructure details.

Core state, publication, and trust

ADR Original decision Later fate
0001 Git was canonical state, SHA-256 CAS owned package bytes, refs expressed views/history, and publication used target sagas. Evolved. Git/CAS/Route ownership retired; single ownership, pointer-last publication, independent target state, and evidence gates survived.
0002 Public routes, immutable generations, and client/provider compatibility had separate acceptance gates. Evolved. Route machinery retired; protocol/client/provider evidence remains separate.
0003 Snapshots required verified generation copies and inventory-bound retention. Evolved. CopyObject snapshot trees retired; retained generations now store metadata and reference sets without payload copies.
0004 Edge token verification and deployment had one versioned cross-provider contract. Retired. Edge entitlement is no longer part of the SOW repository engine.
0005 Every legacy Make target was mapped to SOW, retirement, or policy rejection. Migration history. The mapping completed its cutover role and left the active product.
0007 Public and gated packages shared one canonical repository instead of a separate Pro bucket. Retired. The commercial Edge topology left product scope; the more general single-owner lesson remains.
0008 GC followed complete reachability and required explicit confirmation for destructive work. Evolved. Current GC checks references and exact identity. The old --apply --confirm interface retired; remote deletion also checks cache grace and provider capability.
0009 Once a remote generation became visible, recovery rolled forward rather than inventing rollback. Retained. Commit intent remains the boundary between abandon/reconcile and forward-only recovery.
0010 Metadata signing bound exact public/private identities and treated rotation as a repository-wide transition. Evolved. Current signer evidence and trust-ring rules preserve exact identity without the V1 ref topology.
0011 Remote deletion required target ownership, inventory, checkpoint, grace, and conditional-delete evidence. Retained. R2 remains report-only because it cannot satisfy the required atomic delete capability.
0012 Private origins, cache topology, and exact purge mapping were explicit deployment contracts. Retired. CDN/private-origin deployment is outside current SOW.
0013 A persisted plan was recovery input to revalidate, never authority to replay blindly. Retained. Managed recovery still rebinds plans, files, target identity, and public evidence.
0014 RPM provenance recorded exact packet evidence and distinguished ingestion policy from historical proof. Evolved. Current package authentication and independent trust rings replace the V1 receipt encoding.
0015 Configuration named a target; published refs and checkpoints belonged to canonical state. Retained. Target-neutral generations and target-prefix attempts/checkpoints remain separate owners.
0016 Cloud adapters used concrete, bounded SDK/API contracts instead of a generic provider abstraction. Evolved. Concrete provider boundaries survived; V1 Edge/COS details retired.
0017 Accepted RPM signatures had to close against a stable repository keyring and opened payload identity. Evolved. v0.4 independent multi-ring verification strengthens the same trust boundary.
0018 A selected package set staged, validated, committed, and recovered as one bounded local transaction. Evolved. Managed operations use recovery journals. Plain stages metadata and replaces pointers last; an interrupted Plain build is rebuilt.

Legacy topology, compatibility, and migration

ADR Original decision Later fate
0019 EL7 metadata format and compressor behavior were frozen for legacy consumers. Migration history. Current compatibility documentation, not this freeze, defines supported platforms.
0020 Every legacy physical path and selector group received an explicit owner. Migration history. It prevented ambiguous cutover but is not a current repository model.
0021 Cross-EL yum/infra/{arch} projections were reproduced with exact frozen evidence. Migration history. The special projection left active product scope.
0022 Package history could not be broken merely because physical ownership moved. Evolved. Generations, retained references, and migration journals now preserve continuity without V1 routes.
0023 Canonical Git objects were rebound to exact bytes before admission and use. Evolved. Git authority retired; descriptor/path/digest identity checks remain.
0024 Materialized route receipts acted as narrow read/retirement capabilities. Evolved. Route receipts retired; capability-bound inspection and deletion survived.
0025 Locks bound an exact process instance and stable lock inode rather than elapsed time. Evolved. Stable lock inodes and rejection of timeout-based lock stealing survived; the V1 process-instance lease mechanism did not.
0026 Offline archive creation used a durable intent and strict archive admission. Retired. Offline archive projection left current SOW scope.
0027 Legacy bytes entered canonical state only after exact path, package, and provenance admission. Migration history. The adoption program completed; strict package admission remains in narrower form.
0028 DEB inspection opened only the control archive needed for metadata and rejected ambiguous containers. Retained. Narrow parsing and bounded input remain part of the APT package boundary.
0029 Client floors and the EL8 freeze were explicit owner policy, not inferred from code. Migration history. Current platform policy lives in the compatibility reference and release notes.
0030 Missing legacy YUM bodies could be repaired only from a reviewed blocker-set digest. Migration history. The narrow negative-provenance exception did not become a general ignore flag.
0031 Gated legacy content required exact checksum repair and activation evidence. Retired. Pro activation left product scope; fail-closed repair remained a general lesson.

Provider and Edge control plane

ADR Original decision Later fate
0032 Only one exact owner-designated Cloudflare test tuple could bypass normal production rejection. Retired. It was a narrow historical test exception, never a general deployment rule.
0033 Read-only provider readiness had its own registry and ownership evidence. Retired. Provider bootstrap registry left current SOW.
0034 Worker bootstrap used leases, two-phase recovery, exact resources, and reversible state. Retired. Cloudflare deployment is no longer repository-engine responsibility.
0035 Provider identity, runtime bindings, log sink, and lease ownership were attested before use. Retired. The exact Edge control plane left product scope.
0036 R2 lacked conditional DeleteObject, so an explicit checkpoint-fenced unconditional-delete fallback was allowed behind a deterministic capability probe and repeated identity/fence proofs. Evolved; fallback not inherited. Capability probing and fail-closed defaults survived, but current SOW disables R2 remote deletion and keeps target GC report-only.
0037 Gated publication had to prove denial at the Edge before uploading confidential bytes. Retired. Edge access control is now an external deployment responsibility.
0038 YUM cutover required an expiring receipt bound to exact endpoints, generation, and trust bytes. Migration history. It made a dangerous consumer cutover auditable and then retired.
0039 Caret had one canonical URL spelling across Go, Edge, logs, and origin routing. Evolved. The route was retired; canonical encode-once path handling remains.

Bounded configuration and derived-state recovery

ADR Original decision Later fate
0040 Configuration cardinality and expanded topology were bounded before allocation or execution. Retained. Current parsers and state wires keep explicit size/cardinality limits.
0041 Unknown final projection stages were preserved for audit rather than guessed away. Evolved. Current recovery still preserves contradictory evidence and fails closed.
0042 Derived-state replacement had explicit success, rollback, preserved, and blocked outcomes. Evolved. Managed journals record operation phases; recovery stops when recorded state and files disagree.
0043 File mutation checked descriptor identity and defined the limit against a hostile writer using the same OS user. Retained. Path checks reject identity changes; OS permissions remain the boundary against other writers.
0044 Preserved audit copies could be retired only through an exact capability and confirmation token. Evolved. The V1 command retired; exact capability-bound deletion remains a design rule.
0045 Unjournaled residue had a bounded classifier and could never be silently adopted or deleted. Retained. Current recovery distinguishes known state, safe residue, and contradictory evidence.

The pattern behind the ledger

The decisions that continued share a few rules:

  • Give each fact one owner.
  • Check exact identity before mutation.
  • Prepare immutable content before committing pointers.
  • Recover forward after commit intent.
  • Check references and deletion conditions before removing data.
  • Set input limits and define the product scope.
  • Record compatibility against the client and provider tested.

Their current implementation is described in Design Principles, System Model, and Publication & Recovery.

The v0.1.0 ADR directory remains the immutable primary source. The next article summarizes the separate v0.1 evidence record.

10 - The Original SOW v0.1 Program: A Git for Package Repositories

Why SOW began as a Git/CAS/Route/Edge control plane, what that model achieved, and why the product later reset around a smaller Repository core.

The v0.1 program began on 2026-07-11 and was sealed as v0.1.0 on July 31. It aimed to replace Pigsty’s Makefile-and-rclone workflow with one Go control plane for APT, YUM, and static assets. That control plane covered local state, package history, upstream sync, channels, views, snapshots, two clouds, CDN behavior, commercial access control, and migration.

The guiding phrase was “Git for artifact repositories.” Package distribution has immutable content, mutable names, history, promotion, publication, and rollback. The analogy helped describe those needs, but it also expanded the scope of the product. The Git/CAS/Route/Edge model was later retired; the current model is described in System Model.

The problem the first design tried to solve

The legacy system already published dozens of stable URLs used by APT, DNF, scripts, and installation endpoints. A replacement had to preserve those URLs while adding properties the Makefile workflow could not express safely:

  • one identity for every accepted package body;
  • desired state separated from what each cloud had actually published;
  • snapshots and retained history without guessing which files were still live;
  • incremental upload and purge proportional to the change set;
  • recovery after a crash between metadata preparation and pointer publication;
  • provenance for upstream indexes, package signatures, and migration exceptions;
  • private commercial objects that could not leak through a public index or cache key;
  • compatibility evidence from real APT/DNF clients rather than metadata syntax alone.

The original program deliberately treated these as one connected ownership problem.

The selected v0.1 model

Git was canonical state

.sow/state was a normal Git worktree operated through an embedded Go library. Manifests, refs, provenance, configuration hashes, target checkpoints, and security labels were Git content. SQLite was only a rebuildable query projection.

This gave every state transition a durable history and made one ref update the local commit boundary. It also meant that package-management state, publication state, migration state, and operator intent all had to fit one Git-shaped authority.

CAS owned the bytes

Package bodies lived in an immutable SHA-256 pool. Published trees used hardlinks into that pool, so package references determined whether a file could be removed. This made local deduplication and history inexpensive, but required one filesystem and made materialized paths, receipts, and inode identity part of the product contract.

Refs expressed product meaning

Repository, View, Snapshot, Stable, History, and per-target remote refs described the meaning and retention of content. A mutable view could move without erasing stable or historical reachability. Removing a view changed references; garbage collection remained a separate operation that checked all remaining references.

Each target had its own publication history

Cloudflare R2 and Tencent COS were independent targets. Each target prepared immutable objects, persisted commit intent, moved protocol pointers, purged the CDN, verified public visibility, and advanced its own checkpoint. One target succeeding could not be rolled back merely because the other failed.

The publication order was:

immutable payload
  -> immutable metadata
  -> durable commit intent
  -> mutable protocol pointer
  -> public verification
  -> checkpoint
  -> grace
  -> deletion after reference and identity checks

Current publication retains this ordering. Cache grace and provider deletion checks apply to remote garbage collection.

Routes and Edge were part of repository correctness

The public tree preserved legacy paths, while generation-aware routes selected immutable metadata. Private objects lived behind an Edge contract shared by Cloudflare Worker and EdgeOne. Authentication had to run before origin access; tokens could not enter origin URLs, cache keys, or logs.

This design made CDN routing, token verification, provider deployment, log sinks, and cache topology prerequisites of the repository model.

Why the design was attractive

The model gave each part of the system explicit rules:

  • every durable fact had an explicit owner;
  • immutable bytes and mutable names were separated;
  • local and remote publication states were observable independently;
  • pointer-last publication made partial work recoverable;
  • GC checked all package references before deletion;
  • plans and receipts were revalidated instead of blindly trusted;
  • client compatibility, provider compatibility, and implementation evidence were distinct;
  • the exact legacy migration surface was treated as data, not tribal knowledge.

These rules continued in later releases with fewer supporting mechanisms.

Why the product reset

By the end of July, SOW v0.1 had accumulated:

  • Git commits and refs as an internal database protocol;
  • a global content-addressed store and hardlink materialization layer;
  • Route, View, Snapshot, Generation, Projection, Receipt, and Lease concepts;
  • APT/YUM/asset synchronization and legacy migration inside the core binary;
  • cloud SDK, edge bootstrap, provider attestation, purge, token, and private-origin logic;
  • forty-four surviving ADR files and 112 dated evidence reports.

Each individual addition answered a real failure mode. Together they created several problems.

First, too many objects could appear to own the same package and public path. A package was simultaneously a CAS object, a manifest entry, a View member, a materialized Route, a Snapshot reference, and a remote target object.

Second, local repository generation was coupled to cloud and Edge lifecycle decisions. An operator who only wanted to build a YUM or APT repository inherited the conceptual cost of commercial access control and distributed publication.

Third, the migration program had become a permanent product subsystem. Exact legacy topology, Make targets, CDN behavior, and old provider exceptions were valuable during cutover but did not belong in the long-term repository abstraction.

Finally, recovering every derived path under hostile-writer and crash conditions required more identity, journal, quarantine, and retirement machinery than the core job justified.

Upstream synchronization and channel promotion were another deliberate scope cut. V1 had tests for streaming downloads, malformed input, URLs, and verification order, but acquisition introduced its own policy, retry, provenance, and remote-ownership model. The reset made package acquisition an external input concern so SOW could own repository admission and publication without also becoming a universal mirror orchestrator.

What survived the reset

v0.1 lesson Maintained form
Every durable fact has one owner Repository and target-prefix ownership in System Model
Canonical data differs from projections One pool/ plus metadata-only dists/
Pointers commit after payload preparation Publication & Recovery
Post-commit recovery is forward-only Managed operation and publication journals
Plans and paths are untrusted input Exact identity, containment, and rebind checks
Deletion requires complete evidence References and exact object identity; cache grace and provider capability for remote deletion
Generated server config is derived state The complete Repository root is the hosting unit; serving remains explicit operator configuration
Compatibility is a matrix Platforms & Integrations
Historical evidence never upgrades itself Dated Design and Release records

What was retired

  • Git as product state and SQLite as a disposable cache;
  • a cross-product CAS and hardlink-based materialization layer;
  • the Route/View/Snapshot public command model;
  • generated Nginx includes, Route receipts, and V1 serving-control state;
  • built-in upstream synchronization, promotion, and legacy Make-target migration;
  • Edge token entitlement and private-origin topology as repository prerequisites;
  • the multi-cloud publication control plane and provider bootstrap/attestation system;
  • V1-specific projection receipts, leases, quarantine commands, and recovery surfaces.

Some implementation ideas later returned under a narrower boundary: SOW manages repositories and their publication. Package acquisition and serving remain external tasks.

Timeline

Date Milestone
2026-07-11 Goal, brainstorm, technical research, client and scale baselines
2026-07-12 Core Git/CAS/publication contract and first protocol/provider evidence
2026-07-13–16 Package trust, transaction, legacy topology, and migration closure
2026-07-17–20 Provider bootstrap, Edge confidentiality, deletion capability, and input bounds
2026-07-22–29 Path-identity, hostile-writer, quarantine, and residue-recovery hardening
2026-07-30–31 Clean-room MVP evidence and v0.1.0 source seal
2026-08-01 onward Plain/Managed reset and eventual retirement of the V1 runtime

Primary sources and next records

The immutable v0.1.0 docs/ tree preserves the original architecture contract, requirements traceability, migration material, ADRs, and evidence for the historical implementation.

Continue with the v0.1 Decision Ledger, v0.1 Evidence Ledger, and v0.2 Reset.