Composer architecture and managed-state contract¶

Scope¶

The deterministic Composer uses the public lifecycle:

inspect -> plan -> apply -> validate

The operation mode is explicit for state-changing composition boundaries:

plan/apply --mode initial
plan/apply --mode update
plan/apply --mode upgrade

Omitting --mode remains equivalent to initial. Managed-state update and upgrade are never inferred from the presence of a lock.

Lock schema version 2 provides the consumer-intent and provenance foundation. Read-only reconciliation, crash-recoverable update apply, and explicit upgrade are implemented. Update and upgrade share the same ownership rules and transaction protocol; upgrade changes only which compatibility boundaries the planner may cross.

Source authority¶

The Composer runs from a clean composition source checkout. It binds every successful plan/apply to the exact full Git commit returned by git rev-parse HEAD and refuses tracked source modifications.

The production catalog is loaded closed: catalog IDs must exactly match components/ and recipes/, descriptors/recipes must validate against their schemas, dependencies must exist and be acyclic, and generic capability/lifecycle/topology/workspace components must not depend on artifact authorities.

For update and upgrade, the old lock source revision must exist in the local source history and must be an ancestor of, or identical to, the current source revision. This permits forward reconciliation while rejecting downgrade or unrelated-history reconciliation without consulting the network or a mutable branch.

Interrupted recovery requires the source checkout to be exactly the transaction's recorded target revision. Recovery never silently re-plans against a different source revision.

Resolution and consumer intent¶

One production recipe determines the artifact and selectable component surface. Resolution starts from:

artifact
+ recipe.required_components
+ recipe.default_components not explicitly excluded
+ configuration.components.include

Then all requires dependencies are added transitively.

The resolver fails closed for include/exclude overlap, unexposed selections, exclusion of required/transitive dependencies, active conflicts, or parameters targeting components absent from the resolved closure.

The consumer configuration remains schema version 1. Lock schema version 2 stores a normalized semantic snapshot of that configuration as intent:

{
  "recipe": "skill",
  "components": {
    "include": [],
    "exclude": []
  },
  "parameters": {}
}

Normalization sorts include/exclude component IDs lexically and recursively sorts object keys inside parameters while preserving array order.

update reconstructs resolver input only from this lock intent and rejects --config. upgrade requires an explicit configuration for a new transaction and records its normalized intent as the new lock authority.

configuration_sha256 is retained as provenance for the exact bytes of the most recent explicitly supplied configuration. Update carries that digest forward because it does not consume new configuration bytes. Upgrade replaces it with the digest of the exact supplied upgrade configuration. Interrupted upgrade recovery does not need the original raw configuration bytes: the transaction already binds normalized target intent, exact configuration digest, deterministic target lock, and exact target source revision.

Generated materials¶

A generated material names a bounded declarative generator ID; descriptors never contain executable hooks. The initial allowlist contains only contract-manifest-v1.

It collects contract_registrations from the resolved closure, rejects duplicate identities/paths, sorts by contract ID, and emits deterministic UTF-8 JSON. Unknown generator IDs fail before target writes.

Generated bytes are fully computed during planning. Managed-state operations therefore treat generated destinations like managed destinations for local-modification protection, while desired bytes are deterministically regenerated from the target resolved state.

Initial plan and apply¶

Initial plan is read-only. It computes every copied/generated material byte before classifying the target as create, adopt-identical, or conflict.

Initial composition never overwrites different existing bytes. Identical unmanaged files may be adopted because the resulting lock can truthfully bind their exact bytes. Portable case collisions, file/directory conflicts, symbolic-link boundaries, unsupported generated-material handlers, dependency conflicts, and existing managed-state metadata fail closed.

Created files are written to a temporary file in the destination directory and installed with a no-overwrite hard-link operation. The lock is written last. If the process stops before that point, the repository remains unmanaged and a later initial apply may adopt only exact previously materialized bytes.

Lock schema version 2¶

The canonical lock path remains .template-composition/lock.json. Schema version 2 contains no timestamp, random value, branch name, or network-derived value. It binds:

• canonical source repository identity;
• exact nonzero lowercase 40-hex source revision;
• normalized consumer intent (recipe, include/exclude selection, and parameters);
• SHA-256 of the exact recipe bytes used for resolution (recipe_sha256);
• SHA-256 of the exact most recently supplied configuration bytes (configuration_sha256);
• lexically ordered resolved component IDs, positive integer versions, and exact descriptor-byte SHA-256 values; and
• lexically ordered materialized destinations, owners, ownership modes, and materialized-byte SHA-256 values.

The former top-level recipe field is removed because intent.recipe is the canonical consumer selection. Lock schema v1 is intentionally not accepted; this repository is pre-production and no backward-compatibility migration is required.

The recipe digest closes a v1 audit gap: recipe bytes participate in resolution and therefore must be identifiable from the lock just as component descriptor bytes are.

For seed materials, the recorded digest identifies the bytes first supplied by Composition. Consumer-time validation permits later digest drift because content ownership has transferred to the consumer. Update and upgrade preserve that old recorded digest for a seed that remains in the composition rather than replacing it with the digest of newer source-side seed bytes that were never written to the consumer repository.

Reserved managed-state metadata¶

Component material must not claim paths that collide, case-insensitively or structurally, with Composer-owned metadata:

.template-composition/lock.json
.template-composition/transaction.json
.template-composition/staging/**

transaction.json and staging/** are reserved for the managed-state recovery protocol. transaction.json is implemented as the durable roll-forward marker. staging/** remains reserved so a later storage strategy can stage larger material sets without changing component destination authority.

Other files below .template-composition/, including the self-contained validator and schemas materialized by lifecycle.composition-state, remain valid component destinations.

Consumer-time independence¶

Every artifact requires lifecycle.composition-state transitively. It materializes a stdlib-only validator and lock schema under .template-composition/.

The consumer validator does not read the source catalog. It checks lock-v2 shape, source identity, normalized selection constraints, portable/symlink boundaries, and current material files:

• managed — must exist and match the lock digest;
• generated — must exist and match the lock digest;
• seed — must exist while actively locked, but digest drift is allowed after ownership transfer.

If .template-composition/transaction.json exists, the repository is explicitly reported as interrupted managed state rather than valid steady state. Recovery is a source-side Composer operation.

Extra consumer-owned files are allowed. In particular, a removed seed disappears from the new lock and remains as an ordinary consumer-owned extra file.

Update versus upgrade contract¶

The managed-state operations are intentionally distinct:

• update preserves the normalized lock intent and reconciles it against a descendant Composition source revision;
• upgrade accepts explicit new intent and permits declared compatibility-boundary changes such as recipe/include/exclude/parameter changes and component-version changes.

Composition is not a general-purpose merge engine. The baseline ownership rules apply to both operations:

• managed -> managed: replace/delete only when current bytes match the old lock digest;
• generated -> generated: regenerate/delete only when current bytes match the old lock digest;
• seed -> seed: preserve current consumer bytes unconditionally;
• new managed/generated: create only at a safe unoccupied destination;
• new seed: create only at a safe unoccupied destination, then immediately treat it as consumer-owned for later operations;
• removed seed: preserve it as a consumer-owned extra file;
• component-owner or ownership-mode transitions at an existing destination: never inferred, including during explicit upgrade.

A component version change requires upgrade. Update reports it as an upgrade-required conflict. Upgrade records it in components.changed and may continue. If a component remains at the same positive integer version but its descriptor digest changes, the source has changed a compatibility-bearing descriptor without changing its version. That is a source invariant violation and is rejected by both update and upgrade.

Source material bytes may change without a descriptor change; that is the normal managed/generated replacement or generated regeneration case. Seed source-byte changes do not overwrite a previously materialized seed.

A recipe's bytes may change across a source update while intent.recipe remains the same. Update reports recipe.from_sha256, recipe.to_sha256, and recipe.changed; any resulting component add/remove is reconciled explicitly. Changing the consumer's recipe ID itself is only accepted as explicit upgrade intent, and may still conflict if the resulting material graph would require owner/ownership migration at an existing destination.

Read-only update reconciliation¶

Run an update plan without a configuration file:

python scripts/compose.py plan --mode update --target /path/to/repository

The planner performs no filesystem mutation. It first validates the v2 lock shape and deterministic ordering constraints, verifies source identity/revision ancestry, reconstructs the old normalized intent, resolves the current catalog, and materializes every desired copied/generated byte in memory.

The machine-readable plan contains:

{
  "operation": "update",
  "from_revision": "...",
  "to_revision": "...",
  "intent": {},
  "recipe": {
    "id": "...",
    "from_sha256": "...",
    "to_sha256": "...",
    "changed": false
  },
  "components": {
    "added": [],
    "removed": [],
    "changed": [],
    "unchanged": []
  },
  "files": {
    "create": [],
    "replace": [],
    "remove": [],
    "preserve": [],
    "unchanged": [],
    "conflict": []
  },
  "conflicts": [],
  "lock_preview": {}
}

The top-level intent is the old normalized lock intent used to resolve the update. lock_preview.intent is the newly emitted normalized snapshot; for update they are semantically identical by construction.

For destinations present in both old and new state:

• a managed/generated destination is replace only after its current digest matches the old lock; equal old/new desired digests are unchanged;
• a seed is always preserve after verifying that the path still exists as a safe regular file;
• a different component owner or ownership mode is an upgrade-required conflict.

For newly selected destinations, any existing file, directory, symbolic link, portable case collision, or file/directory prefix collision is a conflict. The update planner does not adopt an existing file for a newly selected material, even when bytes happen to match. A new seed is a create action because no consumer-owned instance exists before first materialization; after that create succeeds, later operations preserve it as consumer-owned seed content.

For removed destinations, clean managed/generated files become remove candidates. Modified managed/generated files conflict. Removed seed files are preserve entries and disappear from the new lock preview, becoming ordinary consumer-owned extra files.

A missing old material is treated as invalid old managed state rather than as an implicit deletion. This prevents update from normalizing away unexplained consumer-side loss.

The lock preview records current source/recipe/component state and carries forward configuration_sha256 plus the old recorded digest for every preserved seed.

Explicit upgrade reconciliation¶

Run upgrade planning only with an explicit target configuration:

python scripts/compose.py plan --mode upgrade --config composition.json --target /path/to/repository

The plan is fully read-only and has the same file action classes as update. It additionally reports:

• intent.from and normalized intent.to;
• old/new exact configuration SHA-256 values;
• old/new recipe IDs and recipe digests; and
• component version changes as explicit components.changed compatibility boundaries.

Upgrade uses the same current-byte checks as update. It does not weaken managed/generated local-modification protection and does not overwrite existing seed bytes. Include/exclude or recipe changes may add/remove components and their files, but all destinations still pass the same safe-path, collision, and ownership reconciliation.

An ownership-mode transition or component-owner transition at the same destination is reported as *_TRANSITION_NOT_SUPPORTED. Explicit upgrade means the consumer has chosen a compatibility-boundary change; it does not provide enough information to infer how arbitrary existing content should be migrated between authorities.

If the new normalized intent changes only parameters, the file action set may be empty while the lock still changes. The new lock records the exact supplied configuration digest and normalized parameter intent.

Safe managed-state apply¶

Run mutation only after reviewing the corresponding read-only plan:

python scripts/compose.py apply --mode update --target /path/to/repository
python scripts/compose.py apply --mode upgrade --config composition.json --target /path/to/repository

apply reconstructs the complete plan before the first mutation. If conflicts exist, it performs no write. It also verifies that the exact lock bytes have not changed while planning.

For a non-no-op managed operation, the first durable managed-state mutation is creation of .template-composition/transaction.json. The transaction document contains:

• operation (update or upgrade) and exact target source revision;
• the complete old and new lock objects;
• SHA-256 identity for the exact old lock file bytes and deterministic new lock file bytes; and
• the lexically ordered create, replace, and remove actions derived from the read-only plan.

The transaction schema does not itself duplicate the full lock schema; source-side recovery validates both embedded locks against composition-lock.schema.json and checks every action against the corresponding old/new file inventory.

Each mutation is preconditioned:

• create accepts an absent destination or, during recovery, a regular file already matching the recorded new digest;
• replace accepts only the recorded old digest or the already-applied new digest;
• remove accepts only the recorded old digest or an already-absent destination;
• symbolic links, non-regular files, unsafe parent paths, or any third digest stop recovery.

create may include a newly selected seed. replace and remove never operate on seed ownership.

Replacement data are written to a same-directory temporary file, fsynced, rechecked against the expected digest, and installed with os.replace. Directory metadata is fsynced where supported. Creation retains no-overwrite installation semantics. Deletion is followed by a directory fsync where supported.

After all material actions, the lock is atomically replaced only when its exact file digest still matches the recorded old lock; an already-installed new lock is accepted during recovery. The source-side self-contained validator functions then validate the new lock and material state while the transaction marker is still present. Only after successful validation and verification that the marker itself has not changed is the marker deleted.

Interrupted update and upgrade recovery¶

The protocol is deterministic roll-forward rather than rollback. Any crash after transaction-marker creation leaves an explicit interrupted state. inspect/consumer validation report recovery required.

Re-run the operation recorded by the marker:

python scripts/compose.py apply --mode update --target /path/to/repository
python scripts/compose.py apply --mode upgrade --target /path/to/repository

Recovery loads the existing transaction instead of constructing a new plan. It requires the current source checkout to equal the transaction target revision and reconstructs deterministic target material/lock state from the transaction-bound intent and source.

Upgrade recovery intentionally takes no --config: the marker already binds the chosen target intent and exact new-lock configuration digest. Supplying a config while recovering an upgrade is rejected rather than allowing a second intent to compete with the interrupted transaction.

For each action, old state means "perform this action" and new state means "this action already completed". Any other state is a conflict and remains untouched. Therefore a consumer change made before an action is attempted, or after an interrupted marker was written, is not silently overwritten by retry.

A crash after the new lock is installed but before marker deletion is also recoverable: all actions and the lock are recognized as already applied, the new state is validated again, and only then is the marker removed.

A same-revision, same-intent no-op update writes no transaction marker and does not rewrite the lock. An upgrade whose exact target lock and mutation set are already identical is likewise write-free.

Determinism and execution boundary¶

For the same immutable source revision, selected intent, and valid old managed state, reconciliation order, managed/generated output bytes, plan JSON, transaction JSON, and the resulting lock preview are deterministic. Seed contents after ownership transfer are preserved rather than merged.

The Composer does not consult mutable branches, wall-clock time, random values, network-discovered defaults, arbitrary hooks, consumer code, package managers, or product build/test/deploy commands when deriving composition state.

As with ordinary filesystem tools, uncoordinated concurrent writers can race a filesystem syscall; Composer nevertheless rechecks digest/path preconditions immediately before each replacement/removal and never treats an unexpected digest as mergeable state.

Managed-state work decomposition¶

The managed-state implementation was delivered in four reviewable layers:

1. lock-v2 and update/upgrade contract;
2. read-only update reconciliation planning;
3. safe update mutation plus interrupted-update recovery; and
4. explicit upgrade semantics using the same ownership and transaction protocol.

Together these layers provide lock-contained intent recovery, deterministic read-only planning, digest-guarded managed/generated mutation, seed preservation, explicit compatibility-boundary changes, and deterministic roll-forward recovery without turning Composition into a general-purpose merge engine.