This is the multi-page printable view of this section. .
SOW Blog
- 1: Design Records
-
2: Release Notes
- 2.1: SOW v0.4.0
- 2.2: SOW v0.3.0
- 2.3: SOW v0.2.0
Release notes, design notes, and project news for SOW — the self-contained APT / YUM package repository manager by Pigsty.
1 - Design Records
Design records explain why SOW adopted a contract, what it rejected, and which evidence
boundary applies. They are ordered by date, which records when the rationale first took
its maintained form; lastmod records later editorial or implementation-alignment
updates. Release notes and source tags remain authoritative for when behavior actually
shipped.
Authority and evidence
This column is the maintained authority for design rationale and decision history. Current commands, configuration, and operator behavior remain in SOW Docs; exact behavior of a historical release remains attached to that release’s source tag and release note.
Each operational claim should match the evidence layer it has actually reached:
Passing one layer does not imply the next. The platform and integration reference records the automated client, Provider, and filesystem coverage; release artifacts are a separate delivery gate.
1.1 - Coordinated Publication Decision
An earlier version of this record proposed an rclone executor, publish --dry-run, and a
separate sow audit command. None of those surfaces shipped in 0.4.0. SOW has no
rclone dependency, publish has no --dry-run, and sow audit is not a command.
This page preserves the decision boundary so a historical proposal cannot be mistaken for the
current product. The normative behavior is documented in
sow publish and
Publication & Recovery.
Decision
SOW 0.4.0 keeps its native filesystem and r2 publication providers. SOW owns both the
Repository meaning and the transport operations needed to preserve it:
- the frozen Generation and exact change plan;
- create-only immutable objects and conditional mutable writes;
- durable commit intent and deterministic pointer order;
- attempt, checkpoint, inventory, grace, and recovery evidence;
- provider verification and canonical public visibility checks.
The public guarantee remains deliberately narrow: publication is ordered, restartable, and eventually converges to one frozen Generation. It is not an atomic transaction across all object keys, and it does not create DNS, bucket policy, CDN configuration, or client trust policy.
Why the executor proposal was not adopted
A generic bulk-copy executor would introduce another versioned dependency while weakening the one-to-one relationship between a planned object and its conditional-write receipt. It would also require a second checkpoint identity model for providers whose bulk tool cannot preserve SOW’s per-object SHA-256 metadata.
The 0.4 work instead hardened the existing, smaller transport boundary: bounded conditional multipart upload, phase-specific header and idle-progress deadlines, retryable replay-safe operations, exact changed-closure verification, and a shared public HTTP verifier. This preserved the existing checkpoint model and avoided a transport migration disguised as a routine upgrade.
What 0.4.0 shipped
Native incremental publication
publish computes the exact Generation delta, writes payload and checksum-addressed metadata
before pointers, persists commit intent, advances views in deterministic order, verifies provider
and public evidence, and records an Applied Checkpoint. A current target is an idempotent no-op.
Operator-confirmed rebind
publish TARGET --rebind can revise the target name, public_endpoint, or max_cache_ttl while
preserving stable storage and target identities. Storage endpoint, provider, region, bucket,
Repository, and prefix require a new target. Every accepted rebind appends an immutable audit
revision and retains forward recovery for an active commit-intent attempt.
Hardened public visibility
Filesystem and R2 HTTP(S) targets share canonical-GET verification, cache-TTL handling for stale content, short bounded transient retries, independent header/body-idle deadlines, and an oversize guard. Filesystem conditional deletion additionally waits for canonical public 404/410 evidence. R2 deletion remains disabled and report-only.
Operator workflow
changes is the read-only local Generation delta; it is not a remote dry run. publish accepts
only a configured target and owns the remote preflight. Use --json for machine-readable results.
If configuration drift is reported, inspect the fields first. Use --rebind only for a permitted
mutable correction; create a new target for any storage or prefix change.
Recovery decision
Repeating the original command is the resume operation. There is no --resume flag. Contradictory
attempt, checkpoint, provider, or public evidence fails closed rather than selecting a convenient
history.
Explicit boundaries
- There is no full-prefix remote
sow auditcommand.sow checkproves the local Repository; normal publication proves its exact changed closure and affected public pointers. - R2 target GC records exact report-only candidates and never deletes remote objects.
- Multiple independent writers, distributed locking, automatic cache purge, DNS management, and arbitrary executor plugins are outside the current contract.
- A successful provider write is not package-manager acceptance. Run the deployed dnf/APT client and signing policy as a separate release gate.
References
1.2 - SOW Design Evolution: From Route Graphs to the Repository Core
This record dates to the 2026-08-10 consolidation of SOW around its Repository core. It distills the useful decisions from the earlier planning, ADR, review, migration, and verification trees without turning every intermediate artifact into maintained product documentation.
Some sealed folders use “v0.2” and “v0.3” for consecutive pre-release design lines. The version labels below refer only to released Git tags: the C2 hardlink line was replaced before v0.2.0 was tagged.
The short history
| Line | Design center | Historical status |
|---|---|---|
| v0.1 | Git/CAS ownership, route-aware materialization, edge authorization, provider workflows, and Pigsty migration | Released as v0.1.0; superseded |
| v0.2 | Local Plain/Managed engine, Repository-scoped single-payload tree, metadata-only views, retained Generations, and target-scoped publication | Released as v0.2.0; public layout remains current |
| v0.3 | V1 removal, package-facts caching, one-pass Plain builds, and bounded payload commits | Released as v0.3.0; same public layout |
| v0.4 | Single-pass verification, independent RPM trust rings, explicit target rebind, and migration/recovery hardening | Current released baseline when this record was updated |
The exact code and claims for each released line remain available in the v0.1.0, v0.2.0, v0.3.0, and v0.4.0 source tags.
v0.1: useful questions, too many product concepts
The first program explored a broad package-distribution control plane: Git-backed canonical state, content-addressed storage, Routes and Views, materialized snapshots, remote providers, edge authorization, legacy migration, and evidence-bound deletion. Forty-five ADRs and a large verification tree made individual failure modes explicit.
That work produced reusable safety ideas, but the combined product surface was too large for the core job. Too many concepts could own or reinterpret the same package bytes and published paths. Edge, provider, migration, and repository concerns were coupled before a small local repository model had been stabilized.
The v0.2 reset therefore retired Git as canonical product state, the separate CAS product layer, route/view/snapshot abstractions, edge entitlement as a repository prerequisite, and the expectation that one design should simultaneously own local generation, remote distribution, legacy migration, and CDN policy.
v0.2: establish a small ownership model
v0.2 deliberately split the product into two execution paths:
- Plain scans a caller-owned directory and rebuilds RPM or DEB metadata in place.
- Managed owns a Workspace containing independent Repositories, each with its own Pool, Dists, SQLite state, locks, operations, and recovery evidence.
This reset established the durable object vocabulary still used today: Package Object, Membership, Desired and Built state, Generation, Changeset, and explicit operation journals. It also established stable lock files, pointer-last file replacement, fail-closed recovery, and the rule that tests or specifications do not upgrade themselves into live-client evidence.
During development, the C2 layout still placed package hardlinks in each RPM view. The sealed archive accurately records that pre-release design line; it must not be relabeled as behavior shipped by the v0.2.0 tag.
v0.2.0: make Repository root the delivery unit
The 2026-08-05 single-payload decision removed canonical package aliases from RPM views.
It shipped three days later in v0.2.0. One Repository has one canonical pool/ and
metadata-only dists/; the complete root is the copy, hosting, authorization, and
publication unit.
v0.2.0 also made the state split explicit:
- Package Object, Desired/Built, Generation, Changeset, and local retention belong to the Repository.
- Publication Attempt, Applied Checkpoint, remote inventory, grace, and deletion evidence belong to one Repository plus one target prefix.
The design rejected cross-Repository deduplication and distributed prefix arbitration. Those are not missing optimizations: they would change the ownership and failure model. Compatibility exports remain outside canonical state.
v0.3: simplify and optimize the same model
v0.3 kept the v0.2.0 public layout. It removed the retired V1 CLI/runtime and migration
harness, introduced the package-facts cache, made Plain generation a rebuildable one-pass
projection, bulk-expanded Membership, and replaced per-object promotion with bounded group
commits. These changes reduced work and code surface without moving ownership away from
Repository pool/ + dists/.
v0.4: harden evidence rather than add another model
v0.4 kept the public layout introduced in v0.2.0 and optimized in v0.3, then concentrated on integrity and recovery. Managed deep verification became a bounded single-pass contract; RPM trust requires one independently verified trust ring instead of evidence assembled across unrelated keys; mutable target settings can be corrected only through an explicit, audited rebind; and migration, publication recovery, and public-delivery checks were tightened.
An early coordinated-publication proposal considered handing bulk transfer to rclone. The final decision kept native providers because publication correctness depends on provider receipts, exact object identity, commit ordering, checkpoint state, public visibility, and recovery — not only byte transport.
What survived every redesign
The useful v0.1 lessons were not discarded with its product model:
- every durable fact needs one explicit owner;
- package bytes and rebuildable projections must not share authority;
- immutable payload and metadata prepare before mutable pointers commit;
- post-commit recovery moves forward; contradictory evidence fails closed;
- deletion requires reachability, grace, ownership, and provider capability;
- path, client, mirror, provider, release, and deployment evidence are separate gates;
- historical PASS results stay attached to their version, source, and environment.
These principles now live in a much smaller object model. See Design Principles, System Model, Single-Payload Repository, and Publication & Recovery.
Documentation authority
The maintained boundary is now simple:
- SOW Docs define current user-facing commands, configuration, formats, and operating behavior.
- This Design column defines maintained rationale and dated decision history.
- Release Notes define version-level delivery and upgrade claims.
- Source tags preserve exact historical implementation, tests, and superseded prose.
Raw planning prompts, intermediate reviews, generated evidence summaries, and abandoned specifications remain historical material. They may explain how a conclusion was reached, but they are not parallel documentation authorities.
1.3 - Publication & Recovery
Building and publishing are separate state transitions. A build produces a target-neutral Generation. Publication applies that Generation to one provider prefix and records enough evidence to recover without guessing.
Ownership split
| Repository-scoped | Target-prefix-scoped |
|---|---|
| Package Object | Publication Attempt |
| Desired and Built state | Applied Checkpoint |
| Generation and Changeset | Remote inventory |
| Retained payload/metadata references | Grace and deletion evidence |
The split prevents a successful filesystem publication from being treated as proof about R2, and prevents one target’s partial attempt from contaminating another target.
Durable target identity and rebind
A target has two nested identities. Storage identity covers provider, endpoint, region, and bucket; target identity adds the public-tree prefix. Repository identity, both target identities, and the three authority acknowledgements are immutable after the first bind. This is what lets attempts, checkpoints, grace, and maintenance survive a configuration-file edit without being reinterpreted as evidence for another namespace.
Target name, public_endpoint, and max_cache_ttl are mutable only through explicit
publish --rebind. Initial bind, migration
backfill, and every accepted rebind append an immutable binding revision. Rebind holds the same
exclusive locks as publish and rechecks immutable fields inside the transaction; changing storage
or prefix requires a new target.
Pending target maintenance freezes TTL in both directions. A filesystem conditional-delete
workflow also freezes public_endpoint. Changing the current TTL never rewrites historical grace:
deletion eligibility is the later of the stored deadline and the applied checkpoint plus the
current grace minimum.
Publication phases
Before commit intent, only add-only objects may be written. publish --abort may reconcile
and remove private filesystem staging, but it does not delete remote objects. Exact
abandoned-object evidence is retained so a later attempt can safely recognize and reuse
matching bytes.
Commit intent is persisted before the first mutable APT stable alias or protocol pointer. After that point, the only recovery direction is forward. Object stores do not provide a multi-key atomic commit, so SOW permits a bounded mixed-generation window while it rolls individual views forward in a deterministic order.
Pointer order
Within a view, immutable content is installed first. Signature companions are installed before their corresponding mutable pointer. Examples:
A client that sees a new pointer can therefore reach every object and signature it names.
Public verification
Routine publication verifies the exact changed closure and every affected final pointer; it does not publicly download every unchanged package. Provider receipts and public visibility are separate proofs. Filesystem and R2 HTTP(S) targets share a canonical-GET verifier with independent response header and body-idle deadlines, a size-plus-one oversize guard, short bounded retries for transient 408/425/429/5xx responses, and cache-TTL retries for stale content or missing objects.
No-cache requests are revalidation hints only. A later ordinary canonical GET must observe the
expected bytes before publication can checkpoint. For filesystem conditional deletion, HTTP 404 or
410 proves public absence; a stale 200 is retried through the TTL. file:// instead uses exact,
descriptor-bound absence. R2 public absence and remote deletion remain disabled.
Filesystem aliases are checked on prospective canonical paths before durable bind, including case aliases on case-insensitive volumes. A failed backend/path preflight creates no binding row and no target prefix.
Published-pointer fence
Once a configured target has an Applied Checkpoint, local configuration cannot silently withdraw a public Dist, architecture, or signing pointer that target still owns. The operator must first retire or unbind the target, or publish a replacement under a new name and prefix.
This turns a dangerous omission into an explicit lifecycle decision.
Retention without payload copies
A retained Generation stores metadata, manifests, and reference sets — not another package tree. Repository-local reachability includes:
- current Desired/Built memberships;
- retained Generation references;
- active operation and recovery journals;
- publication grace and recovery roots.
Local garbage collection may remove a canonical Pool object only when it is outside that complete closure and the exact file identity still matches the recorded candidate.
Remote deletion is a capability
Remote physical deletion additionally requires authoritative inventory, target ownership, grace expiry, cache-absence evidence when applicable, and an atomic conditional delete primitive. A provider that cannot satisfy the primitive can still publish, but SOW must report unreachable candidates rather than issue an unsafe unconditional delete.
The r2 provider is handled this way: publication is implemented, while remote physical
deletion is deliberately disabled and target GC remains report-only.
Recovery outcomes
| Durable boundary | Legal outcome |
|---|---|
| No commit intent | reconcile, then abort or retry |
| Commit intent present | roll forward only |
| Applied checkpoint present | converge and enter grace |
| Evidence contradicts | fail closed; do not invent state |
1.4 - System Model
The model is deliberately layered. Configuration expresses intent, the database records owned state, and the public tree is a deterministic projection. None of those layers may quietly become a substitute for another.
Object hierarchy
Workspace
A Workspace supplies discovery, configuration, and coordination. It owns sow.yml, the
private .sow/ directory, and stable lock paths. It is not a package deduplication domain.
Repository
A Repository is the smallest self-contained public archive and the unit of package
identity. It owns one pool/, one set of dists/, one state database, and one Generation
sequence. Two Repositories share no package bytes or counters even when their inputs match.
Package Object
A Package Object is identified by its final SHA-256 after any package signing. Logical coordinates such as name, version, release, and architecture are metadata; the digest is the byte identity. Re-adding the same digest is idempotent. Different bytes at the same canonical pool path are a hard conflict.
Dist and Membership
A Dist is an APT or RPM publication policy and a collection of memberships. Membership is
many-to-many: one Package Object may belong to several Dists without acquiring another
canonical payload. Neutral packages (all or noarch) project into every matching
architecture view but remain one logical membership.
Desired, Built, and Generation
Desired state is what configuration and package operations ask for. Built state is the last fully rendered and validated public tree. A Generation is an immutable inventory of that Built state; a Changeset is the exact difference between two inventories.
Keeping Desired and Built separate lets an interrupted operation be described honestly: the intent may have changed while the last committed public tree remains valid.
Publication Target
A target binds one Repository to a provider endpoint and prefix. Publication attempts, applied checkpoints, remote inventory, grace, and delete evidence are target-scoped. Building a Repository is target-neutral; publishing it is not.
Storage identity (provider/endpoint/region/bucket), target identity (storage plus prefix), and Repository identity are durable. Only target name, public endpoint, and cache TTL may change through operator-confirmed rebind, and every accepted change appends an immutable binding revision.
State flow
Every arrow is journaled or transactional. A later stage consumes an immutable identity from the previous one rather than reinterpreting mutable paths.
Public and private state
| Public, copy as one unit | Private, never serve |
|---|---|
<repo>/pool/ |
sow.yml |
<repo>/dists/ |
.sow/ databases and journals |
| protocol signatures and indexes | locks, stages, recovery pre-images |
| Generation-described regular files | credentials and provider receipts |
A copy of only the public tree is a valid static repository. It is not an authoritative writer: without the matching private state it cannot safely resume publication history, retention, or garbage collection.
Locking boundary
Workspace lifecycle operations take the Workspace lock. Repository mutations take a stable Repository lock. When both are required, acquisition order is Workspace then Repository, released in reverse. Stable lock paths prevent a rename or recovery operation from accidentally creating a second writer on a new inode.
The model assumes one authoritative Workspace and exclusive write authority for every configured target prefix. Distributed arbitration between independent Workspaces is a non-goal.
1.5 - Design Principles
SOW is not primarily a metadata generator. It is an ownership and state-transition system whose output happens to be APT and RPM repositories. The following principles keep that system small enough to reason about.
One owner for every durable fact
Every durable fact has one scope and one authority:
| Fact | Owner |
|---|---|
| Package bytes and package identity | Repository |
| Desired memberships and Built state | Repository |
| Generation and Changeset | Repository |
| Publication target binding and revisions | Repository + target storage/prefix |
| Publication attempt and applied checkpoint | Repository + target prefix |
| Remote inventory, grace, and delete evidence | Repository + target prefix |
State is not silently shared across Repositories or publish prefixes. The same package may therefore exist once in each Repository or target. That is intentional: local deduplication must not create distributed ownership.
Canonical data, rebuildable projections
Managed package bytes in pool/ are canonical data. In Plain mode, the top-level package
files are canonical instead. Protocol indexes, architecture views, reports, and compatibility
exports are projections; they never become a second owner of package bytes.
The durability rule follows the authority. Managed removes a projection through its recorded operation and removes canonical data only after reachability across every live and retained owner. Plain simply regenerates its owned index paths from the current package directory.
The public tree is the delivery unit
A Repository root contains pool/ + dists/. That complete tree is the unit for static
hosting, copying, authorization, and publication. A single RPM architecture leaf is a
client view, but it is not an independently owned Repository.
Private state such as sow.yml, .sow/, locks, journals, credentials, and recovery files
must never be served as part of that tree.
Pointers commit; payloads prepare
Publication follows this order:
Payload and immutable metadata may arrive before they are visible. A protocol pointer such
as repomd.xml, Release, or InRelease is the commit boundary. Nothing is deleted until
the new pointer is durable and the old reader/cache window is closed.
Recovery cost follows state cost
Plain has no desired-state history to preserve. Its cheapest correct recovery is a fresh one-pass scan and overwrite rebuild, so it stores no transaction journal and does not spend package-size I/O proving an old attempt.
Managed state is different. Before commit intent, an operation may be abandoned if exact reconciliation proves that no public pointer changed. After commit intent, recovery is forward-only. Managed does not guess whether a half-finished publication “probably worked”; it compares journals, manifests, checkpoints, provider identities, and the public tree.
Contradictory evidence stops the operation. A visible refusal is safer than an invisible fork in repository history.
Compatibility is a matrix
Standards compliance, ordinary client behavior, mirror-tool behavior, object-storage layout, proxy normalization, and filesystem semantics are different questions. SOW records them separately and uses a real client or provider for the claim being made.
The canonical Repository and an exported RPM mirror leaf are therefore separate artifacts; evidence for one does not establish compatibility for the other.
Evidence never upgrades itself
A specification is not implementation. A unit test is not a live-client result. A local Hugo build is not a published site. A dated result remains attached to its source revision, environment, and version; it cannot be reused as a PASS for a later layout without rerunning the relevant gate.
Non-goals keep the model honest
The current contract does not promise cross-Repository deduplication, overlapping writers, bucket-global coordination, arbitrary third-party mirror compatibility, or safe remote deletion on providers without an atomic conditional delete primitive. Excluding these is part of the safety contract, not an unfinished implementation detail.
1.6 - Why SOW Uses One Payload Tree per Repository
This decision was drafted on 2026-08-05 during the pre-v0.2 release design consolidation and shipped in SOW v0.2.0 on 2026-08-08. It remains the layout contract in v0.3 and v0.4. It records an architecture choice, not a blanket compatibility claim: protocol closure, ordinary package clients, mirror tools, static hosting, and object storage are verified as separate evidence layers.
Decision
Each SOW Repository owns one public tree:
A Package Object has one canonical payload path inside one Repository. Adding the same object to several Dists or architecture views adds membership and metadata references; it does not create another canonical package file.
The complete pool/ + dists/ root is the supported copy, static-hosting, authorization,
and publication unit. A single architecture leaf is a client view, not an independently
owned repository.
From the C2 development layout to v0.2.0
An earlier pre-release C2 development layout stored each package once in a root Pool, then
created package hardlinks inside RPM views. This gave selected reposync clients a
self-contained leaf without duplicating local bytes.
The local inode optimization did not survive the full delivery model:
- object storage expands each path into a separate key, so aliases duplicate payloads;
- archives and cross-filesystem copies can expand or lose hardlink identity;
- a view-local package path looks canonical even though ownership remains at Repository scope;
- retention and garbage collection risk treating link count as product state;
- preserving mirror-tool behavior would force the canonical tree to follow one client’s local-output assumptions.
Before v0.2.0 was tagged, the final single-payload design removed package aliases from canonical RPM views. The archived C2 documents remain accurate for that development line, but they do not describe the v0.2.0 release tag.
How metadata reaches the Pool
APT already treats Filename as an archive-root-relative path, so a package entry names its
canonical path directly:
For RPM, each <location href> is computed from the actual view directory to the canonical
Pool path:
The renderer and checker share the same path function. It normalizes Repository-relative paths, computes literal parent navigation, escapes data bytes exactly once, resolves the result against a directory URI, and requires the round trip to land on the original Pool path without leaving the Repository.
This keeps the public tree relocatable. The same bytes can be copied under another
filesystem directory, HTTP origin, bucket, or prefix without rewriting metadata, provided
the relative relationship between pool/ and dists/ is preserved.
Rejected alternatives
| Alternative | Why it is not canonical |
|---|---|
| Package hardlink/copy in every RPM view | Creates path aliases and duplicate object keys; makes a filesystem optimization look like durable ownership |
| Symlinked payloads | Not portable to object storage and unsafe across static servers, archives, and trust boundaries |
Leading /pool/... href |
Different clients interpret it as host-root or repository-relative; it also breaks non-root prefixes |
Absolute URL or xml:base |
Binds metadata to one origin and prevents unchanged relocation |
| CDN rewrite or redirect objects | Makes an external router part of repository correctness |
| Workspace- or bucket-global CAS | Introduces cross-Repository ownership, locking, retention, and deletion coordination that SOW does not promise |
Mirror-tool compatibility is an export concern
An ordinary package client only needs to resolve metadata references and fetch package bytes. A mirror tool may additionally insist that every package be stored below the view directory it was given. Those are different contracts.
SOW does not distort the canonical Repository to make every mirror tool happy. When an
operator needs a standalone RPM leaf, sow export rpm-leaf creates an external
compatibility artifact with regenerated leaf-local metadata. Copy is the default;
--hardlink is an explicit same-filesystem optimization for a controlled, read-only
boundary. The export is not a Generation, publication source, membership owner, or garbage
collection root.
Client and mirror-tool results remain in the
compatibility matrix. A successful structural
sow check proves path, digest, and closure invariants; it does
not silently upgrade an unrun client or provider cell to PASS.
Consequences
- Package identity, membership, Generation, retention, and local GC stay Repository-scoped.
- Publication attempts, checkpoints, remote inventory, grace, and deletion evidence stay target-prefix-scoped.
- Every canonical file maps one-to-one to a static object key.
- Payload deletion follows reference closure and exact identity, never inode link count.
- Copying only an RPM architecture leaf is unsupported; copy the Repository root or create an explicit leaf export.
The current mechanism is documented in Pool & Metadata Views, while System Model and Publication & Recovery define the surrounding ownership and lifecycle contracts.
2 - Release Notes
Published SOW release notes cover features, performance, correctness, packaging, and verification, with the newest release first.
2.1 - SOW v0.4.0
SOW 0.4.0 is an integrity and recovery release for Managed repositories. It makes the deep checker’s I/O contract explicit, prevents RPM trust from being assembled across unrelated keys, adds an audited way to correct mutable publication-target settings, and closes the remaining v0.3 migration and interrupted-publication gaps.
Plain repository behavior and the public pool/ + dists/ layout do not change.
Upgrade from 0.3
Stop all Workspace writers, back up the Workspace, install 0.4.0, and run
sow repo migrate REPOSITORY
once for each Repository before ordinary reads or writes. The migration is
explicit and one-way; do not reopen a migrated database with SOW 0.3.
Schema v11 repairs the v0.3 Dist lifecycle case that could leave a Repository marked clean while another Dist remained dirty. Repository status is now derived in the same transaction that changes its Dists. The migration also repairs publication and signer evidence without inventing a historical signing identity: an identity that v0.3 never recorded remains explicitly unverified and cannot become a retained trust assertion.
Schema v12 backfills an append-only publication-target binding ledger. Initial binds, migration backfills, and later operator-confirmed rebinds are separate immutable revisions.
One authenticity pass per physical payload
Every sow check now performs exactly one authoritative content hash for each unique
physical package payload, even when its cached fingerprint still matches. Evidence is
bound to device, inode, size, mtime, and ctime, and to the descriptor that was actually
read. Hard links to the same physical object can share the proof; a replacement or
concurrent identity change invalidates it and fails closed.
The same authenticated evidence is reused while validating retained Generations, walking
the final manifest, and producing sow changes. Retaining several Generations or adding
more Dists therefore does not multiply payload hashing. DEBs and unsigned RPMs require
one full payload stream; a signed RPM uses at most one additional main-header-to-EOF
signature stream, independent of the number of Dists or trust rings.
Package-facts reads are likewise scoped to the selected digest set in bounded, deterministic SQLite batches. A warm 64 MiB build performs no package-body read. Fingerprint drift and a missing facts row share one authoritative payload pass instead of triggering separate scans.
Independent RPM trust rings
Embedded RPM signatures are verified against each candidate trust ring independently. Every recognized signature packet must verify inside one ring, and at least one verified path must authenticate the payload. This preserves support for historical CentOS v3/v4 signatures and deliberately dual-signed packages while preventing a retained single-key claim from being assembled out of packet-by-key successes from different rings.
The current policy ring, each retained single-key ring, and the combined trusted ring all consume the same signature stream. Adding keys changes authorization decisions, not the number of package reads.
Safe target rebind and public verification
sow publish TARGET --rebind is the explicit operator-confirmed path for correcting a
target’s mutable configuration while preserving its storage identity and recovery state.
May change with --rebind |
Requires a new target |
|---|---|
| target name | Repository identity |
public_endpoint |
provider or storage endpoint |
max_cache_ttl |
region, bucket, or prefix |
An ordinary publish reports the mismatch and points to --rebind; it never silently
adopts the new values. Rebind uses the same exclusive locks as publish, appends an audit
revision, and can resume an active commit-intent attempt forward. TTL changes are refused
while target maintenance is pending, and a filesystem public endpoint cannot change
during conditional-delete maintenance. Historical grace deadlines are never shortened.
Filesystem and R2 HTTP(S) public endpoints now share one hardened verifier. Response
headers and body idle progress have separate deadlines, so a large body may stream for
longer than two minutes as long as it continues making progress. Ordinary canonical GETs
remain authoritative; no-cache requests only prompt revalidation. Stale content and 404s
honor max_cache_ttl, 408/425/429/5xx responses get a short bounded retry window, and an
oversized response fails closed. Filesystem absence requires canonical 404/410 visibility.
R2 remote deletion remains deliberately disabled and report-only.
Prospective filesystem target paths are also checked for aliases before a durable bind, including case aliases on case-insensitive volumes. A failed preflight leaves no binding row and creates no target prefix.
Recovery, CLI, and implementation cleanup
- Incremental publication recovery accepts only the exact old checkpoint or target Generation bytes at each pointer. Unknown or third identities still fail closed.
- Routine publication verifies the exact changed object set rather than downloading the full public Generation. No-op and applied-checkpoint recovery reuse complete private inventory evidence, while target GC scans only the protocol pointers it needs.
- R2 operations use phase-specific header/idle deadlines, bounded conditional multipart upload, retryable reads and writes, and a whole-object SHA-256 check before multipart completion.
- Generation signer rows are now an exact manifest side table across partial builds, Dist changes, migration, retention, and local GC.
- Repeating a default
sow addconverges a selected Dist left dirty by an earlier--skipor configuration change, even when the package object itself is reused. sow rm --checkshares the mutating configuration guard and returns an exact no-write preview; its result no longer depends on the scratch filesystem device.- Every Managed command now has a human-readable renderer.
--jsonretains the stablesow.cli/v1envelope, including committed partial results and diagnostic check/preview results on failure. - The retired APT v1 builder, external sort, empty-Dist implementation, and Git-tracked by-hash ledger have been removed. Current Plain and Managed APT parsing/rendering remain covered.
Verification and artifacts
The release gates cover the full Go test suite, vet, staticcheck, dead-code reachability,
vulnerability and RPM-fork provenance checks, race tests, Linux amd64/arm64 builds, and
the deterministic payload/facts I/O contracts above. The release binaries were built
with Go 1.27.0 from source tag v0.4.0. Building from source now requires Go 1.27.0 or
newer, up from 1.26.5 in 0.3. The AWS SDK, SQLite, compression, and cryptography
dependencies were refreshed alongside it, and the quality gates pin staticcheck v0.8.1,
deadcode v0.49.0, and govulncheck v1.7.0.
The release set contains four Linux/macOS archives, two RPMs, two DEBs, and
SHA256SUMS. Every archive contains sow, README.md, CHANGELOG.md, and LICENSE;
native Linux packages install the Apache-2.0 license with the binary.
Get the release
Use the download page for platform-specific commands and verified digests,
or inspect the GitHub release.
After installation, run sow version, migrate each v0.3 Repository, and finish with
sow check before publication.
The maintained contracts are documented in Managed Workspaces, Signing Model, and Publication & Recovery.
2.2 - SOW v0.3.0
SOW 0.3.0 is a performance, durability, and product-focus release. Plain repository generation completes with one package-content pass. Managed repositories avoid per-object membership queries, reuse authenticated package facts, and promote payloads in bounded group commits. The shipping binary and its release pipeline are also consolidated around the repository workflows that SOW supports.
Plain: one package-content pass
The default unsigned sow create path hashes and parses each RPM or DEB once, with
parallelism controlled by --jobs. Metadata is rendered from the retained inspection
result. Before publication, SOW performs one final package-set and file-stat snapshot
check, catching concurrent input changes without reading and hashing every package again.
Plain metadata is rebuildable derived state. The implementation does not create an
operation journal, recovery trash, rollback pre-images, or repeated package hashes. If a
run is interrupted, rerun sow create and SOW reconstructs the metadata from the package
directory.
Pigsty preprocessing retains RPMs whose headers report i386, i486, i586, or
i686, so those packages remain in repository metadata and the repo_complete checksum
manifest. The intended DEB i386 and exact Patroni 3.0.4 filters still apply.
Managed: scale without relaxing integrity
Membership tables have reverse indexes by package SHA-256, and Desired and Built Membership are expanded with one ordered bulk projection instead of one query per object. In the project benchmark, listing a 5,000-object Dist fell from about 4.1 seconds to 33 milliseconds. A 50,000-object Dist completes in roughly 300 milliseconds instead of exceeding ten minutes.
A rebuildable package-facts cache is keyed by immutable package SHA-256. Ingest authenticates and parses each new RPM or DEB once and retains the view-independent facts needed to render metadata. Builds bulk-load those facts, match them in memory, and lazily rebuild missing or corrupt rows from authenticated package bytes.
On an unchanged Pool, warm builds validate payloads with device, inode, size, mtime, and
ctime fingerprints and avoid package-body reads. Fingerprint drift triggers one
authoritative SHA-256 pass and repairs the cache path. sow check remains the explicit
full cryptographic audit. RPM metadata artifacts and DEB architecture indexes use
bounded --jobs concurrency, Generation manifest and changeset rows are inserted in
batches, and final normalization reuses its descriptor snapshot instead of scanning the
Pool again.
Bounded commits and observable builds
Managed payload promotion is a bounded, single-writer group commit. Each batch handles at most 512 objects or 1 GiB: SOW creates the public Pool links, persists the distinct target directories, removes the pending names, and then persists the shared pending directory. A crash therefore leaves a recoverable pending-only, exact dual-link, or Pool-only state; durable loss of both names remains an integrity failure.
Pending payloads use their final 0644 mode inside the private 0700 pending
directory, making promotion a namespace operation. The pending-source guard records
object identity without holding one descriptor per package for the whole build, keeping
descriptor use bounded. Publication also persists each target directory entry before
unlinking its source name.
Long builds append structured build_progress events for rendering, payload promotion,
Dist publication, normalization, and finalization. These events are visible in
sow log and do not add a database checkpoint to each phase, so progress reporting
cannot turn a successful build into a failed one.
Focused publication and runtime
The R2 publication transport is limited to the storage primitives SOW uses:
list, head, get, and conditional put. Remote garbage collection remains report-only and
does not delete objects. Unused cloud-control, CDN, Edge-worker, migration-program, and
alternate runtime paths have been removed from the active tree, leaving one repository
core behind the production CLI. The default go test ./... run therefore covers the
complete active implementation.
Correctness and release quality
- Local GC safely removes a unique case-folded Pool alias recorded by a Generation, which matters on case-insensitive filesystems. Path, size, and digest must still identify the same immutable object; ambiguous or drifted aliases fail closed.
- Every archive includes
LICENSE. RPM and DEB packages declare Apache-2.0 and install the license at/usr/share/licenses/sow/LICENSEand/usr/share/doc/sow/copyright. - CI enforces formatting, module tidiness, vet, static analysis, dead-code checks, performance-test compilation, the full test suite, race tests, clean-delivery checks, and package snapshots.
- Integration gates exercise the production binary in a clean-room mixed RPM/DEB workflow, an exact unsigned Plain APT install on Ubuntu 22.04, DNF signature-transition probes on AlmaLinux 8/9/10, and conditional S3 operations against pinned MinIO.
- The published release contains macOS and Linux archives for amd64 and arm64, RPM and
DEB packages for both Linux architectures, and
SHA256SUMS.
Get the release
Use the download page for platform-specific commands, or inspect every
asset on the GitHub release. After
installation, run sow version to verify the selected binary.
The operating contracts are documented in Plain Mode, Managed Mode, and Platforms & Integrations.
2.3 - SOW v0.2.0
SOW 0.2.0 is a self-contained RPM and DEB repository manager from Pigsty. Release artifacts target Linux and macOS as single Go executables.
Two operating modes
Plain mode indexes RPM and DEB files already present at one directory’s top level:
It writes repodata/, Packages, and Packages.gz in place. Plain mode has no
Workspace, state database, generations, or DEB Release signing.
Managed mode owns package membership and lifecycle:
Each accepted package body is stored once beneath pool/. RPM and APT client views live
beneath dists/ and are materialized as immutable Generations.
Lifecycle controls
Managed repositories include:
- strict
sow/v3configuration and explicit membership policy; - RPM metadata, APT metadata, and optional RPM package signing;
- cheap
statusplus nine-layercheckfor publication gating; - recoverable filesystem and R2 publication attempts;
- explicit retained Generations and reachability-based local GC;
- standalone
rpm-leafexport for consumers that reject parent-relative rpm-md paths.
The canonical Managed tree must be delivered as a complete repository. Use sow publish
for configured targets or stage a whole-root copy offline before an atomic switch. Do not
update a live repository file by file.
Compatibility evidence
The active test suite proves clean-room current-CLI builds for both formats, a Plain APT consumer on Ubuntu 22.04, RPM detached-signature behavior in AlmaLinux 8/9/10 probes, and an isolated S3-compatible provider fixture. Those probes do not by themselves establish a complete current Managed DNF/APT or R2 CLI acceptance gate.
See Compatibility for the exact claim boundary and Quick Start for a fresh installation path.