video-shader-toys/docs/PHASE_5_LIVE_STATE_LAYERING_DESIGN.md

Phase 5 Design: Live State Layering And Composition

This document expands Phase 5 of ARCHITECTURE_RESILIENCE_REVIEW.md into a concrete design target.

Phase 1 named the subsystems. Phase 2 added the typed event substrate. Phase 3 made render-facing live state explicit through RuntimeLiveState, RenderStateComposer, RenderFrameInput, RenderFrameState, RenderFrameStateResolver, and RuntimeServiceLiveBridge. Phase 4 made one render thread the owner of normal runtime GL work. Phase 5 should now make the live parameter model itself explicit: persisted truth, operator/session truth, and transient automation should be separate layers with one predictable composition rule.

Status

• Phase 5 design package: proposed.
• Phase 5 implementation: not started.
• Current alignment: Phase 3 introduced the first pure composition boundary and transient OSC overlay owner, but committed runtime values are still physically stored through RuntimeStore/LayerStackStore, and transient OSC overlay state is applied through render-facing helpers rather than through a first-class layered state model.

Current live-state footholds:

• RuntimeStore owns persisted layer stack, parameter values, presets, config, and render snapshot read models.
• RuntimeCoordinator owns mutation validation, classification, accepted/rejected event publication, snapshot/reload follow-ups, and the policy switch between committed states and live snapshots.
• RuntimeSnapshotProvider publishes render-facing snapshots from committed runtime state.
• RuntimeLiveState owns transient OSC overlay bookkeeping, smoothing, generation tracking, and commit-settlement policy.
• RenderStateComposer combines base render states with live overlay state and returns final per-frame layer states plus settled commit requests.
• RuntimeServiceLiveBridge drains OSC ingress/completion queues and applies them to render live state during frame preparation.

Why Phase 5 Exists

The resilience review identifies live OSC overlay and persisted state as separate concepts that still do not have a fully formal model. The app now has better boundaries, but several policies are still implicit:

• whether a value is durable, committed for the current session, or transient automation
• whether an OSC value should merely influence the current frame or eventually commit
• what reload, preset load, layer removal, shader change, and reset should do to transient values
• which layer wins when UI/operator changes race with OSC automation
• which state changes should publish snapshots, request persistence, or only affect render frames

Without a formal layering model, these rules can leak across RuntimeStore, RuntimeCoordinator, RuntimeLiveState, RenderStateComposer, and service bridges. Phase 5 should make those rules boring and testable.

Goals

Phase 5 should establish:

• explicit state layers for persisted, committed/session, and transient automation values
• one named composition contract for final render values
• clear ownership for layer-specific mutation policy
• explicit reset/reload/preset behavior for transient and committed state
• a clean path for OSC automation to remain high-rate without becoming durable state by accident
• tests for layer precedence, lifecycle, invalidation, and commit policy without GL or DeckLink
• documentation that distinguishes render-local temporal/feedback state from parameter/live-state overlays

Non-Goals

Phase 5 should not require:

• a background persistence writer implementation
• a DeckLink producer/consumer playout queue
• a full cue/timeline/preset performance system
• a new UI state-management framework
• replacing every synchronous coordinator API
• moving temporal history or shader feedback into the runtime state model

Those are later phases or separate feature work. Phase 5 is about parameter and live-value layering.

Target State Model

Phase 5 should formalize three layers:

Layer	Owner	Lifetime	Persistence	Render role
Base persisted state	RuntimeStore / LayerStackStore	survives restart	written to disk	default layer stack, shader selections, saved parameter values
Committed live state	RuntimeCoordinator or a new live-session collaborator	current running session	may request persistence depending on mutation type	operator/UI/current truth until changed again
Transient automation overlay	RuntimeLiveState or a new automation overlay collaborator	high-rate/short-lived	not persisted directly	temporary OSC/automation target applied over committed truth

The target composition rule is:

final render state = base persisted state + committed live state + transient automation overlay

The actual implementation may continue using render snapshots as the base transport. The important part is that each layer has named ownership, documented lifetime, and tested precedence.

Current Composition Shape

Today, final frame state is prepared through this path:

1. OpenGLComposite::renderEffect() processes runtime work.
2. OpenGLComposite builds RenderFrameInput.
3. RuntimeServiceLiveBridge drains OSC updates and completed commits.
4. RenderEngine updates RuntimeLiveState.
5. RenderFrameStateResolver chooses committed states or live snapshot states.
6. RenderStateComposer applies transient overlay values.
7. RenderEngine::RenderPreparedFrame(...) consumes RenderFrameState.

That is a good Phase 3/4 foundation. Phase 5 should make the hidden assumptions in steps 5 and 6 explicit enough that reset/reload/preset and future UI automation behavior are not scattered across those collaborators.

Proposed Collaborators

RuntimeStateLayerModel

Optional pure model that names the layers and composition metadata.

Responsibilities:

• represent base, committed, and transient layer state inputs
• define precedence and invalidation categories
• expose a pure composition function or input object
• keep GL, services, persistence, and device callbacks out of the model

Non-responsibilities:

• disk IO
• OSC socket handling
• render-thread scheduling
• shader compilation

This may be a small set of structs rather than a large class. The value is in naming the contract.

CommittedLiveState

Optional runtime/session collaborator if committed session state needs to move out of RuntimeStore.

Responsibilities:

• hold operator/UI committed values that are true for the current session
• distinguish persistence-required commits from session-only commits
• expose a read model for snapshot publication
• provide reset/load behavior separate from durable storage

Non-responsibilities:

• transient OSC smoothing
• disk writes
• GL resources

Phase 5 can defer this physical split if the policy is documented and covered by tests. The key is that committed-live state becomes a distinct concept even if it still lives inside existing storage temporarily.

AutomationOverlayState

Possible evolution of RuntimeLiveState.

Responsibilities:

• hold transient automation values keyed by route/layer/parameter identity
• track generation, commit-in-flight, and completion
• apply smoothing and settle policy
• decide whether an overlay is render-only, commit-requesting, stale, or invalidated

Non-responsibilities:

• owning committed truth
• persistent state mutation
• snapshot publication

This can start by renaming or narrowing current RuntimeLiveState responsibilities rather than replacing it outright.

LayeredStateComposer

Possible evolution of RenderStateComposer.

Responsibilities:

• apply the target precedence rule
• produce final RuntimeRenderState values for a frame
• return commit requests or overlay observations when policy says a transient value settled
• keep value composition testable without OpenGL

Non-responsibilities:

• frame rendering
• service queue draining
• storage mutation

Layering Rules

Precedence

Default precedence should be:

1. base persisted/snapshot value
2. committed live/session value
3. transient automation overlay

The topmost valid layer wins for discrete values. Numeric/vector values may be smoothed by overlay policy before they win.

Identity

Layering should use stable render-facing identity:

• layer id for persisted structural identity
• layer key/control key for OSC-facing identity
• parameter id for shader-defined identity
• parameter control key for external-control identity

Phase 5 should document which identity is authoritative when layer id and control key disagree or when a shader changes.

Invalidations

The following should have explicit behavior:

• layer removed: clear committed and transient state for that layer
• layer shader changed: clear or remap parameter overlays according to compatible control keys
• preset loaded: replace base/committed state and clear incompatible transient overlays
• shader reload with same controls: preserve compatible transient overlays where safe
• manual reset parameters: clear committed overrides and transient overlays for that layer
• no input/source changes: should not affect parameter layers

Commit Policy

Transient automation may:

• remain render-only
• settle and request a committed mutation
• commit without persistence
• commit with persistence only when the control path explicitly requests it

The policy should be explicit per ingress path or parameter category. Phase 5 does not need a full UI for it, but the default behavior should be documented and tested.

Event And Snapshot Contract

Phase 5 should clarify which changes publish which effects:

Change	Snapshot publication	Persistence request	Render reset	Runtime event
persisted layer stack mutation	yes	yes	maybe	accepted mutation + persistence requested
operator live parameter change	yes	maybe	no, unless structural	accepted mutation
transient OSC overlay update	no committed snapshot by default	no	no	optional overlay observation
overlay settled commit	yes if accepted	usually no for OSC	no	accepted mutation or overlay-settled observation
preset load	yes	maybe	temporal/feedback policy dependent	accepted mutation + reload/reset observations
shader change/reload	yes after build	maybe	temporal/feedback policy dependent	shader build/reload events

This table should evolve with implementation, but Phase 5 should prevent transient overlay updates from masquerading as durable committed state.

Migration Plan

Step 1. Inventory Current State Layers

Document and/or encode where each current state category lives:

• persisted layer stack and parameter values
• committed current-session parameter values
• runtime compile/reload flags
• transient OSC overlays
• render-local temporal history and feedback state

Initial target:

• identify which fields are durable, committed-live, transient automation, render-local, or health/config
• update subsystem docs where the current ownership is misleading
• add small tests for classification if a pure helper exists

Step 2. Name The Layered Composition Input

Introduce a named composition input model around the current RenderStateCompositionInput.

Initial target:

• make base/committed/transient inputs visible in type names or field names
• keep RenderStateComposer behavior unchanged at first
• add tests that assert precedence with no GL

Possible outcomes:

• evolve RenderStateCompositionInput
• add a new LayeredRenderStateInput
• add a thin adapter that feeds existing RenderStateComposer

Step 3. Make Reset And Reload Policy Explicit

Move reset/reload transient-state decisions into one policy point.

Initial target:

• layer removal clears matching transient overlays
• shader change clears incompatible overlays
• preset load clears incompatible overlays
• shader reload can preserve compatible overlays when requested
• temporal/feedback resets stay render-local and separate from parameter overlays

This is where Phase 5 should prevent "clear everything" and "preserve everything" from being scattered through unrelated code.

Step 4. Clarify OSC Commit Semantics

Make the transient-to-committed path explicit.

Initial target:

• document and test whether settled OSC commits persist
• ensure stale generation completions are ignored
• ensure one settled route does not clear unrelated overlay state
• publish or preserve useful events for accepted overlay commits

Current Phase 3 behavior is a good base; Phase 5 should make the policy easier to reason about from the code.

Step 5. Separate Committed-Live Concept From Durable Storage

Decide whether to physically split committed-live state now or introduce a read/model boundary first.

Conservative option:

• leave storage physically in RuntimeStore
• add a named committed-live read model
• keep persistence decisions in RuntimeCoordinator

Stronger option:

• introduce CommittedLiveState
• make RuntimeSnapshotProvider consume committed live state through a read model
• leave durable writes in RuntimeStore

Phase 5 does not need a flag-day split. It needs the concept to stop being implicit.

Step 6. Update Docs And Exit Criteria

Before calling Phase 5 complete, update:

• architecture review checklist
• RuntimeCoordinator, RuntimeStore, RuntimeSnapshotProvider, RenderEngine, and ControlServices subsystem docs
• Phase 6 assumptions about persistence inputs
• Phase 7 assumptions about what render/backend state is not part of live parameter layering

Testing Strategy

Phase 5 tests should avoid GL, DeckLink, sockets, and filesystem writes where possible.

Recommended tests:

• base value is used when no committed or transient value exists
• committed value overrides base value
• transient overlay overrides committed value
• numeric smoothing applies only to transient overlay values
• trigger/bool/discrete overlay behavior is explicit
• layer removal clears matching transient state
• shader change preserves only compatible overlays if policy allows
• preset load clears or replaces committed/transient state according to policy
• settled OSC overlay creates the expected commit request
• settled OSC commit does not request persistence unless policy says so
• stale commit completion does not clear a newer overlay
• render-local temporal/feedback resets do not mutate parameter layers

Existing useful homes:

• RuntimeLiveStateTests for overlay generation, smoothing, settle, and invalidation behavior
• RuntimeSubsystemTests for coordinator mutation, persistence request, and reset/reload policy
• RuntimeEventTypeTests for any new observations or accepted mutation events
• a possible new RuntimeStateLayeringTests target if the composition model gets a pure helper

Risks

Over-Abstraction Risk

It would be easy to introduce too many state containers. Phase 5 should add names where they clarify behavior, not create an elaborate framework.

Persistence Confusion Risk

Committed live state and persisted state are related but not identical. If Phase 5 blurs them, Phase 6's background persistence writer will inherit ambiguous inputs.

Automation Surprise Risk

OSC automation can be high-rate and transient, but users may expect settled values to become "real." The commit policy needs to be explicit enough that UI, OSC, presets, and reloads behave predictably.

Identity/Compatibility Risk

Shader changes and preset loads can invalidate layer/parameter identities. Phase 5 should prefer conservative clearing over accidental application of an old automation value to the wrong control.

Render Coupling Risk

Render-local resources such as temporal history, feedback buffers, readback caches, and playout queues are not parameter layers. Keeping them out of this model avoids turning Phase 5 into a render-resource refactor.

Phase 5 Exit Criteria

Phase 5 can be considered complete once the project can say:

• persisted, committed-live, and transient automation layers are named in code or clear read models
• final render-value precedence is explicit and covered by tests
• RenderStateComposer or its replacement consumes a layered input contract
• reset/reload/preset behavior for transient overlays is centralized or clearly delegated
• OSC overlay settle/commit behavior is explicit, including persistence policy
• RuntimeStore remains durable-state focused and does not absorb transient automation policy
• render-local temporal/feedback state remains separate from live parameter layering
• subsystem docs and the architecture review reflect the final ownership model

Open Questions

• Should committed live state remain physically in RuntimeStore for now, or move to a CommittedLiveState collaborator?
• Should transient OSC overlay updates become app-level typed events, or stay source-local through RuntimeServiceLiveBridge?
• Should overlay commit persistence be global, ingress-specific, or parameter-definition-driven?
• What compatibility rule should apply when shader reload preserves a control key but changes parameter shape?
• Should preset load clear all transient automation, or only automation that no longer maps to the loaded stack?
• Should UI slider drags use the committed-live layer directly, or a short-lived transient layer that commits on release?

Short Version

Phase 5 should make live values boring and explicit.

Persisted state is durable truth. Committed live state is current-session/operator truth. Transient automation is high-rate overlay truth. Render consumes the composed result, and each layer has clear ownership, lifetime, persistence behavior, and reset/reload rules.