video-shader-toys/docs/PHASE_5_LIVE_STATE_LAYERING_DESIGN.md

# Phase 5 Design: Live State Layering And Composition

This document expands Phase 5 of [ARCHITECTURE_RESILIENCE_REVIEW.md](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/docs/ARCHITECTURE_RESILIENCE_REVIEW.md) into a concrete design target.

Phase 1 named the subsystems. Phase 2 added the typed event substrate. Phase 3 made render-facing live state explicit through `RuntimeLiveState`, `RenderStateComposer`, `RenderFrameInput`, `RenderFrameState`, `RenderFrameStateResolver`, and `RuntimeServiceLiveBridge`. Phase 4 made one render thread the owner of normal runtime GL work. Phase 5 should now make the live parameter model itself explicit: persisted truth, operator/session truth, and transient automation should be separate layers with one predictable composition rule.

## Status

- Phase 5 design package: proposed.
- Phase 5 implementation: Step 2 started.
- Current alignment: Phase 3 introduced the first pure composition boundary and transient OSC overlay owner. Phase 5 now has a small `RuntimeStateLayerModel` inventory that names the current state categories, and `RenderStateComposer` consumes a `LayeredRenderStateInput` whose fields make base persisted, committed live, and transient automation inputs explicit. Committed runtime values are still physically stored through `RuntimeStore`/`LayerStackStore`, and transient OSC overlay state is still applied through `RuntimeLiveState`.

Current live-state footholds:

- `RuntimeStore` owns persisted layer stack, parameter values, presets, config, and render snapshot read models.
- `RuntimeCoordinator` owns mutation validation, classification, accepted/rejected event publication, snapshot/reload follow-ups, and the policy switch between committed states and live snapshots.
- `RuntimeSnapshotProvider` publishes render-facing snapshots from committed runtime state.
- `RuntimeLiveState` owns transient OSC overlay bookkeeping, smoothing, generation tracking, and commit-settlement policy.
- `RenderStateComposer` consumes `LayeredRenderStateInput`, chooses committed-live layer states over base-persisted layer states when both are supplied, applies transient automation on top, and returns final per-frame layer states plus settled commit requests.
- `RuntimeServiceLiveBridge` drains OSC ingress/completion queues and applies them to render live state during frame preparation.
- `RuntimeStateLayerModel` names the Phase 5 state categories and classifies current fields as base persisted, committed live, transient automation, render-local, or health/config state.

## Why Phase 5 Exists

The resilience review identifies live OSC overlay and persisted state as separate concepts that still do not have a fully formal model. The app now has better boundaries, but several policies are still implicit:

- whether a value is durable, committed for the current session, or transient automation
- whether an OSC value should merely influence the current frame or eventually commit
- what reload, preset load, layer removal, shader change, and reset should do to transient values
- which layer wins when UI/operator changes race with OSC automation
- which state changes should publish snapshots, request persistence, or only affect render frames

Without a formal layering model, these rules can leak across `RuntimeStore`, `RuntimeCoordinator`, `RuntimeLiveState`, `RenderStateComposer`, and service bridges. Phase 5 should make those rules boring and testable.

## Goals

Phase 5 should establish:

- explicit state layers for persisted, committed/session, and transient automation values
- one named composition contract for final render values
- clear ownership for layer-specific mutation policy
- explicit reset/reload/preset behavior for transient and committed state
- a clean path for OSC automation to remain high-rate without becoming durable state by accident
- tests for layer precedence, lifecycle, invalidation, and commit policy without GL or DeckLink
- documentation that distinguishes render-local temporal/feedback state from parameter/live-state overlays

## Non-Goals

Phase 5 should not require:

- a background persistence writer implementation
- a DeckLink producer/consumer playout queue
- a full cue/timeline/preset performance system
- a new UI state-management framework
- replacing every synchronous coordinator API
- moving temporal history or shader feedback into the runtime state model

Those are later phases or separate feature work. Phase 5 is about parameter and live-value layering.

## Target State Model

Phase 5 should formalize three layers:

| Layer | Owner | Lifetime | Persistence | Render role |
| --- | --- | --- | --- | --- |
| Base persisted state | `RuntimeStore` / `LayerStackStore` | survives restart | written to disk | default layer stack, shader selections, saved parameter values |
| Committed live state | `RuntimeCoordinator` or a new live-session collaborator | current running session | may request persistence depending on mutation type | operator/UI/current truth until changed again |
| Transient automation overlay | `RuntimeLiveState` or a new automation overlay collaborator | high-rate/short-lived | not persisted directly | temporary OSC/automation target applied over committed truth |

The target composition rule is:

```text
final render state = base persisted state + committed live state + transient automation overlay
```

The actual implementation may continue using render snapshots as the base transport. The important part is that each layer has named ownership, documented lifetime, and tested precedence.

## Current Composition Shape

Today, final frame state is prepared through this path:

1. `OpenGLComposite::renderEffect()` processes runtime work.
2. `OpenGLComposite` builds `RenderFrameInput`.
3. `RuntimeServiceLiveBridge` drains OSC updates and completed commits.
4. `RenderEngine` updates `RuntimeLiveState`.
5. `RenderFrameStateResolver` chooses committed states or live snapshot states.
6. `RenderStateComposer` applies transient overlay values.
7. `RenderEngine::RenderPreparedFrame(...)` consumes `RenderFrameState`.

That is a good Phase 3/4 foundation. Phase 5 should make the hidden assumptions in steps 5 and 6 explicit enough that reset/reload/preset and future UI automation behavior are not scattered across those collaborators.

## Proposed Collaborators

### `RuntimeStateLayerModel`

Optional pure model that names the layers and composition metadata.

Responsibilities:

- represent base, committed, and transient layer state inputs
- define precedence and invalidation categories
- expose a pure composition function or input object
- keep GL, services, persistence, and device callbacks out of the model

Non-responsibilities:

- disk IO
- OSC socket handling
- render-thread scheduling
- shader compilation

This may be a small set of structs rather than a large class. The value is in naming the contract.

### `CommittedLiveState`

Optional runtime/session collaborator if committed session state needs to move out of `RuntimeStore`.

Responsibilities:

- hold operator/UI committed values that are true for the current session
- distinguish persistence-required commits from session-only commits
- expose a read model for snapshot publication
- provide reset/load behavior separate from durable storage

Non-responsibilities:

- transient OSC smoothing
- disk writes
- GL resources

Phase 5 can defer this physical split if the policy is documented and covered by tests. The key is that committed-live state becomes a distinct concept even if it still lives inside existing storage temporarily.

### `AutomationOverlayState`

Possible evolution of `RuntimeLiveState`.

Responsibilities:

- hold transient automation values keyed by route/layer/parameter identity
- track generation, commit-in-flight, and completion
- apply smoothing and settle policy
- decide whether an overlay is render-only, commit-requesting, stale, or invalidated

Non-responsibilities:

- owning committed truth
- persistent state mutation
- snapshot publication

This can start by renaming or narrowing current `RuntimeLiveState` responsibilities rather than replacing it outright.

### `LayeredStateComposer`

Possible evolution of `RenderStateComposer`.

Responsibilities:

- apply the target precedence rule
- produce final `RuntimeRenderState` values for a frame
- return commit requests or overlay observations when policy says a transient value settled
- keep value composition testable without OpenGL

Non-responsibilities:

- frame rendering
- service queue draining
- storage mutation

## Layering Rules

### Precedence

Default precedence should be:

1. base persisted/snapshot value
2. committed live/session value
3. transient automation overlay

The topmost valid layer wins for discrete values. Numeric/vector values may be smoothed by overlay policy before they win.

### Identity

Layering should use stable render-facing identity:

- layer id for persisted structural identity
- layer key/control key for OSC-facing identity
- parameter id for shader-defined identity
- parameter control key for external-control identity

Phase 5 should document which identity is authoritative when layer id and control key disagree or when a shader changes.

### Invalidations

The following should have explicit behavior:

- layer removed: clear committed and transient state for that layer
- layer shader changed: clear or remap parameter overlays according to compatible control keys
- preset loaded: replace base/committed state and clear incompatible transient overlays
- shader reload with same controls: preserve compatible transient overlays where safe
- manual reset parameters: clear committed overrides and transient overlays for that layer
- no input/source changes: should not affect parameter layers

### Commit Policy

Transient automation may:

- remain render-only
- settle and request a committed mutation
- commit without persistence
- commit with persistence only when the control path explicitly requests it

The policy should be explicit per ingress path or parameter category. Phase 5 does not need a full UI for it, but the default behavior should be documented and tested.

## Event And Snapshot Contract

Phase 5 should clarify which changes publish which effects:

| Change | Snapshot publication | Persistence request | Render reset | Runtime event |
| --- | --- | --- | --- | --- |
| persisted layer stack mutation | yes | yes | maybe | accepted mutation + persistence requested |
| operator live parameter change | yes | maybe | no, unless structural | accepted mutation |
| transient OSC overlay update | no committed snapshot by default | no | no | optional overlay observation |
| overlay settled commit | yes if accepted | usually no for OSC | no | accepted mutation or overlay-settled observation |
| preset load | yes | maybe | temporal/feedback policy dependent | accepted mutation + reload/reset observations |
| shader change/reload | yes after build | maybe | temporal/feedback policy dependent | shader build/reload events |

This table should evolve with implementation, but Phase 5 should prevent transient overlay updates from masquerading as durable committed state.

## Migration Plan

### Step 1. Inventory Current State Layers

Document and/or encode where each current state category lives:

- persisted layer stack and parameter values
- committed current-session parameter values
- runtime compile/reload flags
- transient OSC overlays
- render-local temporal history and feedback state

Initial target:

- [x] identify which fields are durable, committed-live, transient automation, render-local, or health/config
- [x] update subsystem docs where the current ownership is misleading
- [x] add small tests for classification if a pure helper exists

### Step 2. Name The Layered Composition Input

Introduce a named composition input model around the previous `RenderStateCompositionInput`.

Initial target:

- [x] make base/committed/transient inputs visible in type names or field names
- [x] keep `RenderStateComposer` behavior unchanged at first
- [x] add tests that assert precedence with no GL

Possible outcomes:

- [x] add a new `LayeredRenderStateInput`
- [ ] add a thin adapter if a later migration needs compatibility with the previous input shape

### Step 3. Make Reset And Reload Policy Explicit

Move reset/reload transient-state decisions into one policy point.

Initial target:

- layer removal clears matching transient overlays
- shader change clears incompatible overlays
- preset load clears incompatible overlays
- shader reload can preserve compatible overlays when requested
- temporal/feedback resets stay render-local and separate from parameter overlays

This is where Phase 5 should prevent "clear everything" and "preserve everything" from being scattered through unrelated code.

### Step 4. Clarify OSC Commit Semantics

Make the transient-to-committed path explicit.

Initial target:

- document and test whether settled OSC commits persist
- ensure stale generation completions are ignored
- ensure one settled route does not clear unrelated overlay state
- publish or preserve useful events for accepted overlay commits

Current Phase 3 behavior is a good base; Phase 5 should make the policy easier to reason about from the code.

### Step 5. Separate Committed-Live Concept From Durable Storage

Decide whether to physically split committed-live state now or introduce a read/model boundary first.

Conservative option:

- leave storage physically in `RuntimeStore`
- add a named committed-live read model
- keep persistence decisions in `RuntimeCoordinator`

Stronger option:

- introduce `CommittedLiveState`
- make `RuntimeSnapshotProvider` consume committed live state through a read model
- leave durable writes in `RuntimeStore`

Phase 5 does not need a flag-day split. It needs the concept to stop being implicit.

### Step 6. Update Docs And Exit Criteria

Before calling Phase 5 complete, update:

- architecture review checklist
- `RuntimeCoordinator`, `RuntimeStore`, `RuntimeSnapshotProvider`, `RenderEngine`, and `ControlServices` subsystem docs
- Phase 6 assumptions about persistence inputs
- Phase 7 assumptions about what render/backend state is not part of live parameter layering

## Testing Strategy

Phase 5 tests should avoid GL, DeckLink, sockets, and filesystem writes where possible.

Recommended tests:

- base value is used when no committed or transient value exists
- committed value overrides base value
- transient overlay overrides committed value
- numeric smoothing applies only to transient overlay values
- trigger/bool/discrete overlay behavior is explicit
- layer removal clears matching transient state
- shader change preserves only compatible overlays if policy allows
- preset load clears or replaces committed/transient state according to policy
- settled OSC overlay creates the expected commit request
- settled OSC commit does not request persistence unless policy says so
- stale commit completion does not clear a newer overlay
- render-local temporal/feedback resets do not mutate parameter layers

Existing useful homes:

- `RuntimeLiveStateTests` for overlay generation, smoothing, settle, and invalidation behavior
- `RuntimeSubsystemTests` for coordinator mutation, persistence request, and reset/reload policy
- `RuntimeEventTypeTests` for any new observations or accepted mutation events
- a possible new `RuntimeStateLayeringTests` target if the composition model gets a pure helper

## Risks

### Over-Abstraction Risk

It would be easy to introduce too many state containers. Phase 5 should add names where they clarify behavior, not create an elaborate framework.

### Persistence Confusion Risk

Committed live state and persisted state are related but not identical. If Phase 5 blurs them, Phase 6's background persistence writer will inherit ambiguous inputs.

### Automation Surprise Risk

OSC automation can be high-rate and transient, but users may expect settled values to become "real." The commit policy needs to be explicit enough that UI, OSC, presets, and reloads behave predictably.

### Identity/Compatibility Risk

Shader changes and preset loads can invalidate layer/parameter identities. Phase 5 should prefer conservative clearing over accidental application of an old automation value to the wrong control.

### Render Coupling Risk

Render-local resources such as temporal history, feedback buffers, readback caches, and playout queues are not parameter layers. Keeping them out of this model avoids turning Phase 5 into a render-resource refactor.

## Phase 5 Exit Criteria

Phase 5 can be considered complete once the project can say:

- [x] persisted, committed-live, and transient automation layers are named in code or clear read models
- [x] final render-value precedence is explicit and covered by tests
- [x] `RenderStateComposer` or its replacement consumes a layered input contract
- [ ] reset/reload/preset behavior for transient overlays is centralized or clearly delegated
- [ ] OSC overlay settle/commit behavior is explicit, including persistence policy
- [ ] `RuntimeStore` remains durable-state focused and does not absorb transient automation policy
- [ ] render-local temporal/feedback state remains separate from live parameter layering
- [ ] subsystem docs and the architecture review reflect the final ownership model

## Open Questions

- Should committed live state remain physically in `RuntimeStore` for now, or move to a `CommittedLiveState` collaborator?
- Should transient OSC overlay updates become app-level typed events, or stay source-local through `RuntimeServiceLiveBridge`?
- Should overlay commit persistence be global, ingress-specific, or parameter-definition-driven?
- What compatibility rule should apply when shader reload preserves a control key but changes parameter shape?
- Should preset load clear all transient automation, or only automation that no longer maps to the loaded stack?
- Should UI slider drags use the committed-live layer directly, or a short-lived transient layer that commits on release?

## Short Version

Phase 5 should make live values boring and explicit.

Persisted state is durable truth. Committed live state is current-session/operator truth. Transient automation is high-rate overlay truth. Render consumes the composed result, and each layer has clear ownership, lifetime, persistence behavior, and reset/reload rules.