Design Records

• 1: Coordinated Publication Decision
• 2: SOW Design Evolution: From Route Graphs to the Repository Core
• 3: Publication & Recovery
• 4: System Model
• 5: Design Principles
• 6: Why SOW Uses One Payload Tree per Repository

design -> implementation -> focused tests -> real client/provider run -> release artifact

1 - Coordinated Publication Decision

This page preserves the decision boundary so a historical proposal cannot be mistaken for the current product. The normative behavior is documented in sow publish and Publication & Recovery.

Decision

SOW 0.4.0 keeps its native filesystem and r2 publication providers. SOW owns both the Repository meaning and the transport operations needed to preserve it:

• the frozen Generation and exact change plan;
• create-only immutable objects and conditional mutable writes;
• durable commit intent and deterministic pointer order;
• attempt, checkpoint, inventory, grace, and recovery evidence;
• provider verification and canonical public visibility checks.

The public guarantee remains deliberately narrow: publication is ordered, restartable, and eventually converges to one frozen Generation. It is not an atomic transaction across all object keys, and it does not create DNS, bucket policy, CDN configuration, or client trust policy.

Why the executor proposal was not adopted

A generic bulk-copy executor would introduce another versioned dependency while weakening the one-to-one relationship between a planned object and its conditional-write receipt. It would also require a second checkpoint identity model for providers whose bulk tool cannot preserve SOW’s per-object SHA-256 metadata.

The 0.4 work instead hardened the existing, smaller transport boundary: bounded conditional multipart upload, phase-specific header and idle-progress deadlines, retryable replay-safe operations, exact changed-closure verification, and a shared public HTTP verifier. This preserved the existing checkpoint model and avoided a transport migration disguised as a routine upgrade.

What 0.4.0 shipped

Native incremental publication

publish computes the exact Generation delta, writes payload and checksum-addressed metadata before pointers, persists commit intent, advances views in deterministic order, verifies provider and public evidence, and records an Applied Checkpoint. A current target is an idempotent no-op.

Operator-confirmed rebind

publish TARGET --rebind can revise the target name, public_endpoint, or max_cache_ttl while preserving stable storage and target identities. Storage endpoint, provider, region, bucket, Repository, and prefix require a new target. Every accepted rebind appends an immutable audit revision and retains forward recovery for an active commit-intent attempt.

Hardened public visibility

Filesystem and R2 HTTP(S) targets share canonical-GET verification, cache-TTL handling for stale content, short bounded transient retries, independent header/body-idle deadlines, and an oversize guard. Filesystem conditional deletion additionally waits for canonical public 404/410 evidence. R2 deletion remains disabled and report-only.

Operator workflow

sow build -r pgsql
sow check -r pgsql
sow changes -r pgsql
sow publish prod

changes is the read-only local Generation delta; it is not a remote dry run. publish accepts only a configured target and owns the remote preflight. Use --json for machine-readable results.

If configuration drift is reported, inspect the fields first. Use --rebind only for a permitted mutable correction; create a new target for any storage or prefix change.

Recovery decision

Was durable commit intent recorded?
├── no  -> rerun publish, or reconcile and publish --abort
└── yes -> rerun publish; forward recovery is the only legal direction

Repeating the original command is the resume operation. There is no --resume flag. Contradictory attempt, checkpoint, provider, or public evidence fails closed rather than selecting a convenient history.

Explicit boundaries

• There is no full-prefix remote sow audit command. sow check proves the local Repository; normal publication proves its exact changed closure and affected public pointers.
• R2 target GC records exact report-only candidates and never deletes remote objects.
• Multiple independent writers, distributed locking, automatic cache purge, DNS management, and arbitrary executor plugins are outside the current contract.
• A successful provider write is not package-manager acceptance. Run the deployed dnf/APT client and signing policy as a separate release gate.

References

• Current Publication & Recovery model
• Current sow publish command
• Platforms & Integrations

2 - SOW Design Evolution: From Route Graphs to the Repository Core

This record dates to the 2026-08-10 consolidation of SOW around its Repository core. It distills the useful decisions from the earlier planning, ADR, review, migration, and verification trees without turning every intermediate artifact into maintained product documentation.

Some sealed folders use “v0.2” and “v0.3” for consecutive pre-release design lines. The version labels below refer only to released Git tags: the C2 hardlink line was replaced before v0.2.0 was tagged.

The short history

The exact code and claims for each released line remain available in the v0.1.0, v0.2.0, v0.3.0, and v0.4.0 source tags.

v0.1: useful questions, too many product concepts

The first program explored a broad package-distribution control plane: Git-backed canonical state, content-addressed storage, Routes and Views, materialized snapshots, remote providers, edge authorization, legacy migration, and evidence-bound deletion. Forty-five ADRs and a large verification tree made individual failure modes explicit.

That work produced reusable safety ideas, but the combined product surface was too large for the core job. Too many concepts could own or reinterpret the same package bytes and published paths. Edge, provider, migration, and repository concerns were coupled before a small local repository model had been stabilized.

The v0.2 reset therefore retired Git as canonical product state, the separate CAS product layer, route/view/snapshot abstractions, edge entitlement as a repository prerequisite, and the expectation that one design should simultaneously own local generation, remote distribution, legacy migration, and CDN policy.

v0.2: establish a small ownership model

v0.2 deliberately split the product into two execution paths:

• Plain scans a caller-owned directory and rebuilds RPM or DEB metadata in place.
• Managed owns a Workspace containing independent Repositories, each with its own Pool, Dists, SQLite state, locks, operations, and recovery evidence.

This reset established the durable object vocabulary still used today: Package Object, Membership, Desired and Built state, Generation, Changeset, and explicit operation journals. It also established stable lock files, pointer-last file replacement, fail-closed recovery, and the rule that tests or specifications do not upgrade themselves into live-client evidence.

During development, the C2 layout still placed package hardlinks in each RPM view. The sealed archive accurately records that pre-release design line; it must not be relabeled as behavior shipped by the v0.2.0 tag.

v0.2.0: make Repository root the delivery unit

The 2026-08-05 single-payload decision removed canonical package aliases from RPM views. It shipped three days later in v0.2.0. One Repository has one canonical pool/ and metadata-only dists/; the complete root is the copy, hosting, authorization, and publication unit.

v0.2.0 also made the state split explicit:

• Package Object, Desired/Built, Generation, Changeset, and local retention belong to the Repository.
• Publication Attempt, Applied Checkpoint, remote inventory, grace, and deletion evidence belong to one Repository plus one target prefix.

The design rejected cross-Repository deduplication and distributed prefix arbitration. Those are not missing optimizations: they would change the ownership and failure model. Compatibility exports remain outside canonical state.

v0.3: simplify and optimize the same model

v0.3 kept the v0.2.0 public layout. It removed the retired V1 CLI/runtime and migration harness, introduced the package-facts cache, made Plain generation a rebuildable one-pass projection, bulk-expanded Membership, and replaced per-object promotion with bounded group commits. These changes reduced work and code surface without moving ownership away from Repository pool/ + dists/.

v0.4: harden evidence rather than add another model

v0.4 kept the public layout introduced in v0.2.0 and optimized in v0.3, then concentrated on integrity and recovery. Managed deep verification became a bounded single-pass contract; RPM trust requires one independently verified trust ring instead of evidence assembled across unrelated keys; mutable target settings can be corrected only through an explicit, audited rebind; and migration, publication recovery, and public-delivery checks were tightened.

An early coordinated-publication proposal considered handing bulk transfer to rclone. The final decision kept native providers because publication correctness depends on provider receipts, exact object identity, commit ordering, checkpoint state, public visibility, and recovery — not only byte transport.

What survived every redesign

The useful v0.1 lessons were not discarded with its product model:

• every durable fact needs one explicit owner;
• package bytes and rebuildable projections must not share authority;
• immutable payload and metadata prepare before mutable pointers commit;
• post-commit recovery moves forward; contradictory evidence fails closed;
• deletion requires reachability, grace, ownership, and provider capability;
• path, client, mirror, provider, release, and deployment evidence are separate gates;
• historical PASS results stay attached to their version, source, and environment.

These principles now live in a much smaller object model. See Design Principles, System Model, Single-Payload Repository, and Publication & Recovery.

Documentation authority

The maintained boundary is now simple:

• SOW Docs define current user-facing commands, configuration, formats, and operating behavior.
• This Design column defines maintained rationale and dated decision history.
• Release Notes define version-level delivery and upgrade claims.
• Source tags preserve exact historical implementation, tests, and superseded prose.

Raw planning prompts, intermediate reviews, generated evidence summaries, and abandoned specifications remain historical material. They may explain how a conclusion was reached, but they are not parallel documentation authorities.

3 - Publication & Recovery

Building and publishing are separate state transitions. A build produces a target-neutral Generation. Publication applies that Generation to one provider prefix and records enough evidence to recover without guessing.

Ownership split

The split prevents a successful filesystem publication from being treated as proof about R2, and prevents one target’s partial attempt from contaminating another target.

Durable target identity and rebind

A target has two nested identities. Storage identity covers provider, endpoint, region, and bucket; target identity adds the public-tree prefix. Repository identity, both target identities, and the three authority acknowledgements are immutable after the first bind. This is what lets attempts, checkpoints, grace, and maintenance survive a configuration-file edit without being reinterpreted as evidence for another namespace.

Target name, public_endpoint, and max_cache_ttl are mutable only through explicit publish --rebind. Initial bind, migration backfill, and every accepted rebind append an immutable binding revision. Rebind holds the same exclusive locks as publish and rechecks immutable fields inside the transaction; changing storage or prefix requires a new target.

Pending target maintenance freezes TTL in both directions. A filesystem conditional-delete workflow also freezes public_endpoint. Changing the current TTL never rewrites historical grace: deletion eligibility is the later of the stored deadline and the applied checkpoint plus the current grace minimum.

Publication phases

plan
  -> create-only payload
  -> checksum-addressed metadata
  -> durable commit intent
  -> protocol pointers, one view at a time
  -> provider and canonical public verification
  -> applied checkpoint
  -> grace
  -> evidence-gated deletion

Before commit intent, only add-only objects may be written. publish --abort may reconcile and remove private filesystem staging, but it does not delete remote objects. Exact abandoned-object evidence is retained so a later attempt can safely recognize and reuse matching bytes.

Commit intent is persisted before the first mutable APT stable alias or protocol pointer. After that point, the only recovery direction is forward. Object stores do not provide a multi-key atomic commit, so SOW permits a bounded mixed-generation window while it rolls individual views forward in a deterministic order.

Pointer order

Within a view, immutable content is installed first. Signature companions are installed before their corresponding mutable pointer. Examples:

RPM: checksum-named repodata -> repomd.xml.asc -> repomd.xml
APT: by-hash/direct indexes -> Release.gpg -> InRelease/Release

A client that sees a new pointer can therefore reach every object and signature it names.

Public verification

Routine publication verifies the exact changed closure and every affected final pointer; it does not publicly download every unchanged package. Provider receipts and public visibility are separate proofs. Filesystem and R2 HTTP(S) targets share a canonical-GET verifier with independent response header and body-idle deadlines, a size-plus-one oversize guard, short bounded retries for transient 408/425/429/5xx responses, and cache-TTL retries for stale content or missing objects.

No-cache requests are revalidation hints only. A later ordinary canonical GET must observe the expected bytes before publication can checkpoint. For filesystem conditional deletion, HTTP 404 or 410 proves public absence; a stale 200 is retried through the TTL. file:// instead uses exact, descriptor-bound absence. R2 public absence and remote deletion remain disabled.

Filesystem aliases are checked on prospective canonical paths before durable bind, including case aliases on case-insensitive volumes. A failed backend/path preflight creates no binding row and no target prefix.

Published-pointer fence

Once a configured target has an Applied Checkpoint, local configuration cannot silently withdraw a public Dist, architecture, or signing pointer that target still owns. The operator must first retire or unbind the target, or publish a replacement under a new name and prefix.

This turns a dangerous omission into an explicit lifecycle decision.

Retention without payload copies

A retained Generation stores metadata, manifests, and reference sets — not another package tree. Repository-local reachability includes:

• current Desired/Built memberships;
• retained Generation references;
• active operation and recovery journals;
• publication grace and recovery roots.

Local garbage collection may remove a canonical Pool object only when it is outside that complete closure and the exact file identity still matches the recorded candidate.

Remote deletion is a capability

Remote physical deletion additionally requires authoritative inventory, target ownership, grace expiry, cache-absence evidence when applicable, and an atomic conditional delete primitive. A provider that cannot satisfy the primitive can still publish, but SOW must report unreachable candidates rather than issue an unsafe unconditional delete.

The r2 provider is handled this way: publication is implemented, while remote physical deletion is deliberately disabled and target GC remains report-only.

Recovery outcomes

4 - System Model

The model is deliberately layered. Configuration expresses intent, the database records owned state, and the public tree is a deterministic projection. None of those layers may quietly become a substitute for another.

Object hierarchy

Workspace
├── Repository
│   ├── Package Object
│   ├── Dist
│   │   └── Membership -> Package Object
│   ├── Desired state
│   ├── Built Generation
│   └── Retained Generation references
└── Publication Target
    └── Repository + provider + prefix

Workspace

A Workspace supplies discovery, configuration, and coordination. It owns sow.yml, the private .sow/ directory, and stable lock paths. It is not a package deduplication domain.

Repository

A Repository is the smallest self-contained public archive and the unit of package identity. It owns one pool/, one set of dists/, one state database, and one Generation sequence. Two Repositories share no package bytes or counters even when their inputs match.

Package Object

A Package Object is identified by its final SHA-256 after any package signing. Logical coordinates such as name, version, release, and architecture are metadata; the digest is the byte identity. Re-adding the same digest is idempotent. Different bytes at the same canonical pool path are a hard conflict.

Dist and Membership

A Dist is an APT or RPM publication policy and a collection of memberships. Membership is many-to-many: one Package Object may belong to several Dists without acquiring another canonical payload. Neutral packages (all or noarch) project into every matching architecture view but remain one logical membership.

Desired, Built, and Generation

Desired state is what configuration and package operations ask for. Built state is the last fully rendered and validated public tree. A Generation is an immutable inventory of that Built state; a Changeset is the exact difference between two inventories.

Keeping Desired and Built separate lets an interrupted operation be described honestly: the intent may have changed while the last committed public tree remains valid.

Publication Target

A target binds one Repository to a provider endpoint and prefix. Publication attempts, applied checkpoints, remote inventory, grace, and delete evidence are target-scoped. Building a Repository is target-neutral; publishing it is not.

Storage identity (provider/endpoint/region/bucket), target identity (storage plus prefix), and Repository identity are durable. Only target name, public endpoint, and cache TTL may change through operator-confirmed rebind, and every accepted change appends an immutable binding revision.

State flow

package input
    -> inspect/sign/hash
    -> Package Object + Membership
    -> Desired state
    -> render and validate
    -> Built Generation + Changeset
    -> target publication attempt
    -> applied checkpoint

Every arrow is journaled or transactional. A later stage consumes an immutable identity from the previous one rather than reinterpreting mutable paths.

Public and private state

A copy of only the public tree is a valid static repository. It is not an authoritative writer: without the matching private state it cannot safely resume publication history, retention, or garbage collection.

Locking boundary

Workspace lifecycle operations take the Workspace lock. Repository mutations take a stable Repository lock. When both are required, acquisition order is Workspace then Repository, released in reverse. Stable lock paths prevent a rename or recovery operation from accidentally creating a second writer on a new inode.

The model assumes one authoritative Workspace and exclusive write authority for every configured target prefix. Distributed arbitration between independent Workspaces is a non-goal.

5 - Design Principles

SOW is not primarily a metadata generator. It is an ownership and state-transition system whose output happens to be APT and RPM repositories. The following principles keep that system small enough to reason about.

One owner for every durable fact

Every durable fact has one scope and one authority:

State is not silently shared across Repositories or publish prefixes. The same package may therefore exist once in each Repository or target. That is intentional: local deduplication must not create distributed ownership.

Canonical data, rebuildable projections

Managed package bytes in pool/ are canonical data. In Plain mode, the top-level package files are canonical instead. Protocol indexes, architecture views, reports, and compatibility exports are projections; they never become a second owner of package bytes.

The durability rule follows the authority. Managed removes a projection through its recorded operation and removes canonical data only after reachability across every live and retained owner. Plain simply regenerates its owned index paths from the current package directory.

The public tree is the delivery unit

A Repository root contains pool/ + dists/. That complete tree is the unit for static hosting, copying, authorization, and publication. A single RPM architecture leaf is a client view, but it is not an independently owned Repository.

Private state such as sow.yml, .sow/, locks, journals, credentials, and recovery files must never be served as part of that tree.

Pointers commit; payloads prepare

Publication follows this order:

payload -> immutable/checksum-named metadata -> mutable pointer -> grace -> delete

Payload and immutable metadata may arrive before they are visible. A protocol pointer such as repomd.xml, Release, or InRelease is the commit boundary. Nothing is deleted until the new pointer is durable and the old reader/cache window is closed.

Recovery cost follows state cost

Plain has no desired-state history to preserve. Its cheapest correct recovery is a fresh one-pass scan and overwrite rebuild, so it stores no transaction journal and does not spend package-size I/O proving an old attempt.

Managed state is different. Before commit intent, an operation may be abandoned if exact reconciliation proves that no public pointer changed. After commit intent, recovery is forward-only. Managed does not guess whether a half-finished publication “probably worked”; it compares journals, manifests, checkpoints, provider identities, and the public tree.

Contradictory evidence stops the operation. A visible refusal is safer than an invisible fork in repository history.

Compatibility is a matrix

Standards compliance, ordinary client behavior, mirror-tool behavior, object-storage layout, proxy normalization, and filesystem semantics are different questions. SOW records them separately and uses a real client or provider for the claim being made.

The canonical Repository and an exported RPM mirror leaf are therefore separate artifacts; evidence for one does not establish compatibility for the other.

Evidence never upgrades itself

A specification is not implementation. A unit test is not a live-client result. A local Hugo build is not a published site. A dated result remains attached to its source revision, environment, and version; it cannot be reused as a PASS for a later layout without rerunning the relevant gate.

Non-goals keep the model honest

The current contract does not promise cross-Repository deduplication, overlapping writers, bucket-global coordination, arbitrary third-party mirror compatibility, or safe remote deletion on providers without an atomic conditional delete primitive. Excluding these is part of the safety contract, not an unfinished implementation detail.

6 - Why SOW Uses One Payload Tree per Repository

This decision was drafted on 2026-08-05 during the pre-v0.2 release design consolidation and shipped in SOW v0.2.0 on 2026-08-08. It remains the layout contract in v0.3 and v0.4. It records an architecture choice, not a blanket compatibility claim: protocol closure, ordinary package clients, mirror tools, static hosting, and object storage are verified as separate evidence layers.

Decision

Each SOW Repository owns one public tree:

<repository>/
  pool/                         # canonical package bytes
  dists/
    <dist>/
      <architecture>/           # protocol metadata only

A Package Object has one canonical payload path inside one Repository. Adding the same object to several Dists or architecture views adds membership and metadata references; it does not create another canonical package file.

The complete pool/ + dists/ root is the supported copy, static-hosting, authorization, and publication unit. A single architecture leaf is a client view, not an independently owned repository.

From the C2 development layout to v0.2.0

An earlier pre-release C2 development layout stored each package once in a root Pool, then created package hardlinks inside RPM views. This gave selected reposync clients a self-contained leaf without duplicating local bytes.

The local inode optimization did not survive the full delivery model:

• object storage expands each path into a separate key, so aliases duplicate payloads;
• archives and cross-filesystem copies can expand or lose hardlink identity;
• a view-local package path looks canonical even though ownership remains at Repository scope;
• retention and garbage collection risk treating link count as product state;
• preserving mirror-tool behavior would force the canonical tree to follow one client’s local-output assumptions.

Before v0.2.0 was tagged, the final single-payload design removed package aliases from canonical RPM views. The archived C2 documents remain accurate for that development line, but they do not describe the v0.2.0 release tag.

How metadata reaches the Pool

APT already treats Filename as an archive-root-relative path, so a package entry names its canonical path directly:

Filename: pool/p/pkg/pkg_1.0_amd64.deb

For RPM, each <location href> is computed from the actual view directory to the canonical Pool path:

view:    dists/el9/x86_64/
payload: pool/p/pkg/pkg-1.0-1.x86_64.rpm
href:    ../../../pool/p/pkg/pkg-1.0-1.x86_64.rpm

The renderer and checker share the same path function. It normalizes Repository-relative paths, computes literal parent navigation, escapes data bytes exactly once, resolves the result against a directory URI, and requires the round trip to land on the original Pool path without leaving the Repository.

This keeps the public tree relocatable. The same bytes can be copied under another filesystem directory, HTTP origin, bucket, or prefix without rewriting metadata, provided the relative relationship between pool/ and dists/ is preserved.

Rejected alternatives

Mirror-tool compatibility is an export concern

An ordinary package client only needs to resolve metadata references and fetch package bytes. A mirror tool may additionally insist that every package be stored below the view directory it was given. Those are different contracts.

SOW does not distort the canonical Repository to make every mirror tool happy. When an operator needs a standalone RPM leaf, sow export rpm-leaf creates an external compatibility artifact with regenerated leaf-local metadata. Copy is the default; --hardlink is an explicit same-filesystem optimization for a controlled, read-only boundary. The export is not a Generation, publication source, membership owner, or garbage collection root.

Client and mirror-tool results remain in the compatibility matrix. A successful structural sow check proves path, digest, and closure invariants; it does not silently upgrade an unrun client or provider cell to PASS.

Consequences

• Package identity, membership, Generation, retention, and local GC stay Repository-scoped.
• Publication attempts, checkpoints, remote inventory, grace, and deletion evidence stay target-prefix-scoped.
• Every canonical file maps one-to-one to a static object key.
• Payload deletion follows reference closure and exact identity, never inode link count.
• Copying only an RPM architecture leaf is unsupported; copy the Repository root or create an explicit leaf export.

The current mechanism is documented in Pool & Metadata Views, while System Model and Publication & Recovery define the surrounding ownership and lifecycle contracts.