docs update
Some checks failed
CI / React UI Build (push) Successful in 11s
CI / Native Windows Build And Tests (push) Successful in 2m40s
CI / Windows Release Package (push) Has been cancelled

This commit is contained in:
Aiden
2026-05-11 18:45:03 +10:00
parent 761df3b2d0
commit 99fd903144
4 changed files with 402 additions and 8 deletions

View File

@@ -538,6 +538,10 @@ Expected benefits:
Once rendering and snapshots are isolated, formalize how final parameter values are derived. Once rendering and snapshots are isolated, formalize how final parameter values are derived.
Dedicated design note:
- [PHASE_5_LIVE_STATE_LAYERING_DESIGN.md](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/docs/PHASE_5_LIVE_STATE_LAYERING_DESIGN.md)
Recommended layers: Recommended layers:
- base persisted state - base persisted state

View File

@@ -24,7 +24,7 @@ Current footholds:
- `RuntimeServiceLiveBridge` translates service OSC queues into render live-state updates and queues settled overlay commit requests. - `RuntimeServiceLiveBridge` translates service OSC queues into render live-state updates and queues settled overlay commit requests.
- `RuntimeEventDispatcher` now routes accepted mutations, reloads, snapshots, shader build events, backend observations, and health observations. - `RuntimeEventDispatcher` now routes accepted mutations, reloads, snapshots, shader build events, backend observations, and health observations.
The current architecture is much better than the original `RuntimeHost` shape. The remaining render risk is now mostly Phase 4 work: GL calls are still reached from callback and UI paths through a shared context lock rather than through one render-thread owner. The current architecture is much better than the original `RuntimeHost` shape. Phase 4 has since moved normal runtime GL work onto the `RenderEngine` render thread, so the remaining render-facing risk is no longer shared context ownership; it is the later producer/consumer playout work needed to keep DeckLink callbacks from synchronously waiting on output production.
## Why Phase 3 Exists ## Why Phase 3 Exists
@@ -314,7 +314,7 @@ Before calling Phase 3 complete, update:
- architecture review checklist - architecture review checklist
- Phase 4 assumptions about render thread input state - Phase 4 assumptions about render thread input state
Status: complete. The Phase 4 design note starts from the `RenderFrameInput` / `RenderFrameState` contract and the remaining shared-GL ownership paths. Status: complete. The Phase 4 design note started from the `RenderFrameInput` / `RenderFrameState` contract and has now completed the shared-GL ownership migration.
## Testing Strategy ## Testing Strategy

View File

@@ -0,0 +1,390 @@
# Phase 5 Design: Live State Layering And Composition
This document expands Phase 5 of [ARCHITECTURE_RESILIENCE_REVIEW.md](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/docs/ARCHITECTURE_RESILIENCE_REVIEW.md) into a concrete design target.
Phase 1 named the subsystems. Phase 2 added the typed event substrate. Phase 3 made render-facing live state explicit through `RuntimeLiveState`, `RenderStateComposer`, `RenderFrameInput`, `RenderFrameState`, `RenderFrameStateResolver`, and `RuntimeServiceLiveBridge`. Phase 4 made one render thread the owner of normal runtime GL work. Phase 5 should now make the live parameter model itself explicit: persisted truth, operator/session truth, and transient automation should be separate layers with one predictable composition rule.
## Status
- Phase 5 design package: proposed.
- Phase 5 implementation: not started.
- Current alignment: Phase 3 introduced the first pure composition boundary and transient OSC overlay owner, but committed runtime values are still physically stored through `RuntimeStore`/`LayerStackStore`, and transient OSC overlay state is applied through render-facing helpers rather than through a first-class layered state model.
Current live-state footholds:
- `RuntimeStore` owns persisted layer stack, parameter values, presets, config, and render snapshot read models.
- `RuntimeCoordinator` owns mutation validation, classification, accepted/rejected event publication, snapshot/reload follow-ups, and the policy switch between committed states and live snapshots.
- `RuntimeSnapshotProvider` publishes render-facing snapshots from committed runtime state.
- `RuntimeLiveState` owns transient OSC overlay bookkeeping, smoothing, generation tracking, and commit-settlement policy.
- `RenderStateComposer` combines base render states with live overlay state and returns final per-frame layer states plus settled commit requests.
- `RuntimeServiceLiveBridge` drains OSC ingress/completion queues and applies them to render live state during frame preparation.
## Why Phase 5 Exists
The resilience review identifies live OSC overlay and persisted state as separate concepts that still do not have a fully formal model. The app now has better boundaries, but several policies are still implicit:
- whether a value is durable, committed for the current session, or transient automation
- whether an OSC value should merely influence the current frame or eventually commit
- what reload, preset load, layer removal, shader change, and reset should do to transient values
- which layer wins when UI/operator changes race with OSC automation
- which state changes should publish snapshots, request persistence, or only affect render frames
Without a formal layering model, these rules can leak across `RuntimeStore`, `RuntimeCoordinator`, `RuntimeLiveState`, `RenderStateComposer`, and service bridges. Phase 5 should make those rules boring and testable.
## Goals
Phase 5 should establish:
- explicit state layers for persisted, committed/session, and transient automation values
- one named composition contract for final render values
- clear ownership for layer-specific mutation policy
- explicit reset/reload/preset behavior for transient and committed state
- a clean path for OSC automation to remain high-rate without becoming durable state by accident
- tests for layer precedence, lifecycle, invalidation, and commit policy without GL or DeckLink
- documentation that distinguishes render-local temporal/feedback state from parameter/live-state overlays
## Non-Goals
Phase 5 should not require:
- a background persistence writer implementation
- a DeckLink producer/consumer playout queue
- a full cue/timeline/preset performance system
- a new UI state-management framework
- replacing every synchronous coordinator API
- moving temporal history or shader feedback into the runtime state model
Those are later phases or separate feature work. Phase 5 is about parameter and live-value layering.
## Target State Model
Phase 5 should formalize three layers:
| Layer | Owner | Lifetime | Persistence | Render role |
| --- | --- | --- | --- | --- |
| Base persisted state | `RuntimeStore` / `LayerStackStore` | survives restart | written to disk | default layer stack, shader selections, saved parameter values |
| Committed live state | `RuntimeCoordinator` or a new live-session collaborator | current running session | may request persistence depending on mutation type | operator/UI/current truth until changed again |
| Transient automation overlay | `RuntimeLiveState` or a new automation overlay collaborator | high-rate/short-lived | not persisted directly | temporary OSC/automation target applied over committed truth |
The target composition rule is:
```text
final render state = base persisted state + committed live state + transient automation overlay
```
The actual implementation may continue using render snapshots as the base transport. The important part is that each layer has named ownership, documented lifetime, and tested precedence.
## Current Composition Shape
Today, final frame state is prepared through this path:
1. `OpenGLComposite::renderEffect()` processes runtime work.
2. `OpenGLComposite` builds `RenderFrameInput`.
3. `RuntimeServiceLiveBridge` drains OSC updates and completed commits.
4. `RenderEngine` updates `RuntimeLiveState`.
5. `RenderFrameStateResolver` chooses committed states or live snapshot states.
6. `RenderStateComposer` applies transient overlay values.
7. `RenderEngine::RenderPreparedFrame(...)` consumes `RenderFrameState`.
That is a good Phase 3/4 foundation. Phase 5 should make the hidden assumptions in steps 5 and 6 explicit enough that reset/reload/preset and future UI automation behavior are not scattered across those collaborators.
## Proposed Collaborators
### `RuntimeStateLayerModel`
Optional pure model that names the layers and composition metadata.
Responsibilities:
- represent base, committed, and transient layer state inputs
- define precedence and invalidation categories
- expose a pure composition function or input object
- keep GL, services, persistence, and device callbacks out of the model
Non-responsibilities:
- disk IO
- OSC socket handling
- render-thread scheduling
- shader compilation
This may be a small set of structs rather than a large class. The value is in naming the contract.
### `CommittedLiveState`
Optional runtime/session collaborator if committed session state needs to move out of `RuntimeStore`.
Responsibilities:
- hold operator/UI committed values that are true for the current session
- distinguish persistence-required commits from session-only commits
- expose a read model for snapshot publication
- provide reset/load behavior separate from durable storage
Non-responsibilities:
- transient OSC smoothing
- disk writes
- GL resources
Phase 5 can defer this physical split if the policy is documented and covered by tests. The key is that committed-live state becomes a distinct concept even if it still lives inside existing storage temporarily.
### `AutomationOverlayState`
Possible evolution of `RuntimeLiveState`.
Responsibilities:
- hold transient automation values keyed by route/layer/parameter identity
- track generation, commit-in-flight, and completion
- apply smoothing and settle policy
- decide whether an overlay is render-only, commit-requesting, stale, or invalidated
Non-responsibilities:
- owning committed truth
- persistent state mutation
- snapshot publication
This can start by renaming or narrowing current `RuntimeLiveState` responsibilities rather than replacing it outright.
### `LayeredStateComposer`
Possible evolution of `RenderStateComposer`.
Responsibilities:
- apply the target precedence rule
- produce final `RuntimeRenderState` values for a frame
- return commit requests or overlay observations when policy says a transient value settled
- keep value composition testable without OpenGL
Non-responsibilities:
- frame rendering
- service queue draining
- storage mutation
## Layering Rules
### Precedence
Default precedence should be:
1. base persisted/snapshot value
2. committed live/session value
3. transient automation overlay
The topmost valid layer wins for discrete values. Numeric/vector values may be smoothed by overlay policy before they win.
### Identity
Layering should use stable render-facing identity:
- layer id for persisted structural identity
- layer key/control key for OSC-facing identity
- parameter id for shader-defined identity
- parameter control key for external-control identity
Phase 5 should document which identity is authoritative when layer id and control key disagree or when a shader changes.
### Invalidations
The following should have explicit behavior:
- layer removed: clear committed and transient state for that layer
- layer shader changed: clear or remap parameter overlays according to compatible control keys
- preset loaded: replace base/committed state and clear incompatible transient overlays
- shader reload with same controls: preserve compatible transient overlays where safe
- manual reset parameters: clear committed overrides and transient overlays for that layer
- no input/source changes: should not affect parameter layers
### Commit Policy
Transient automation may:
- remain render-only
- settle and request a committed mutation
- commit without persistence
- commit with persistence only when the control path explicitly requests it
The policy should be explicit per ingress path or parameter category. Phase 5 does not need a full UI for it, but the default behavior should be documented and tested.
## Event And Snapshot Contract
Phase 5 should clarify which changes publish which effects:
| Change | Snapshot publication | Persistence request | Render reset | Runtime event |
| --- | --- | --- | --- | --- |
| persisted layer stack mutation | yes | yes | maybe | accepted mutation + persistence requested |
| operator live parameter change | yes | maybe | no, unless structural | accepted mutation |
| transient OSC overlay update | no committed snapshot by default | no | no | optional overlay observation |
| overlay settled commit | yes if accepted | usually no for OSC | no | accepted mutation or overlay-settled observation |
| preset load | yes | maybe | temporal/feedback policy dependent | accepted mutation + reload/reset observations |
| shader change/reload | yes after build | maybe | temporal/feedback policy dependent | shader build/reload events |
This table should evolve with implementation, but Phase 5 should prevent transient overlay updates from masquerading as durable committed state.
## Migration Plan
### Step 1. Inventory Current State Layers
Document and/or encode where each current state category lives:
- persisted layer stack and parameter values
- committed current-session parameter values
- runtime compile/reload flags
- transient OSC overlays
- render-local temporal history and feedback state
Initial target:
- identify which fields are durable, committed-live, transient automation, render-local, or health/config
- update subsystem docs where the current ownership is misleading
- add small tests for classification if a pure helper exists
### Step 2. Name The Layered Composition Input
Introduce a named composition input model around the current `RenderStateCompositionInput`.
Initial target:
- make base/committed/transient inputs visible in type names or field names
- keep `RenderStateComposer` behavior unchanged at first
- add tests that assert precedence with no GL
Possible outcomes:
- evolve `RenderStateCompositionInput`
- add a new `LayeredRenderStateInput`
- add a thin adapter that feeds existing `RenderStateComposer`
### Step 3. Make Reset And Reload Policy Explicit
Move reset/reload transient-state decisions into one policy point.
Initial target:
- layer removal clears matching transient overlays
- shader change clears incompatible overlays
- preset load clears incompatible overlays
- shader reload can preserve compatible overlays when requested
- temporal/feedback resets stay render-local and separate from parameter overlays
This is where Phase 5 should prevent "clear everything" and "preserve everything" from being scattered through unrelated code.
### Step 4. Clarify OSC Commit Semantics
Make the transient-to-committed path explicit.
Initial target:
- document and test whether settled OSC commits persist
- ensure stale generation completions are ignored
- ensure one settled route does not clear unrelated overlay state
- publish or preserve useful events for accepted overlay commits
Current Phase 3 behavior is a good base; Phase 5 should make the policy easier to reason about from the code.
### Step 5. Separate Committed-Live Concept From Durable Storage
Decide whether to physically split committed-live state now or introduce a read/model boundary first.
Conservative option:
- leave storage physically in `RuntimeStore`
- add a named committed-live read model
- keep persistence decisions in `RuntimeCoordinator`
Stronger option:
- introduce `CommittedLiveState`
- make `RuntimeSnapshotProvider` consume committed live state through a read model
- leave durable writes in `RuntimeStore`
Phase 5 does not need a flag-day split. It needs the concept to stop being implicit.
### Step 6. Update Docs And Exit Criteria
Before calling Phase 5 complete, update:
- architecture review checklist
- `RuntimeCoordinator`, `RuntimeStore`, `RuntimeSnapshotProvider`, `RenderEngine`, and `ControlServices` subsystem docs
- Phase 6 assumptions about persistence inputs
- Phase 7 assumptions about what render/backend state is not part of live parameter layering
## Testing Strategy
Phase 5 tests should avoid GL, DeckLink, sockets, and filesystem writes where possible.
Recommended tests:
- base value is used when no committed or transient value exists
- committed value overrides base value
- transient overlay overrides committed value
- numeric smoothing applies only to transient overlay values
- trigger/bool/discrete overlay behavior is explicit
- layer removal clears matching transient state
- shader change preserves only compatible overlays if policy allows
- preset load clears or replaces committed/transient state according to policy
- settled OSC overlay creates the expected commit request
- settled OSC commit does not request persistence unless policy says so
- stale commit completion does not clear a newer overlay
- render-local temporal/feedback resets do not mutate parameter layers
Existing useful homes:
- `RuntimeLiveStateTests` for overlay generation, smoothing, settle, and invalidation behavior
- `RuntimeSubsystemTests` for coordinator mutation, persistence request, and reset/reload policy
- `RuntimeEventTypeTests` for any new observations or accepted mutation events
- a possible new `RuntimeStateLayeringTests` target if the composition model gets a pure helper
## Risks
### Over-Abstraction Risk
It would be easy to introduce too many state containers. Phase 5 should add names where they clarify behavior, not create an elaborate framework.
### Persistence Confusion Risk
Committed live state and persisted state are related but not identical. If Phase 5 blurs them, Phase 6's background persistence writer will inherit ambiguous inputs.
### Automation Surprise Risk
OSC automation can be high-rate and transient, but users may expect settled values to become "real." The commit policy needs to be explicit enough that UI, OSC, presets, and reloads behave predictably.
### Identity/Compatibility Risk
Shader changes and preset loads can invalidate layer/parameter identities. Phase 5 should prefer conservative clearing over accidental application of an old automation value to the wrong control.
### Render Coupling Risk
Render-local resources such as temporal history, feedback buffers, readback caches, and playout queues are not parameter layers. Keeping them out of this model avoids turning Phase 5 into a render-resource refactor.
## Phase 5 Exit Criteria
Phase 5 can be considered complete once the project can say:
- [ ] persisted, committed-live, and transient automation layers are named in code or clear read models
- [ ] final render-value precedence is explicit and covered by tests
- [ ] `RenderStateComposer` or its replacement consumes a layered input contract
- [ ] reset/reload/preset behavior for transient overlays is centralized or clearly delegated
- [ ] OSC overlay settle/commit behavior is explicit, including persistence policy
- [ ] `RuntimeStore` remains durable-state focused and does not absorb transient automation policy
- [ ] render-local temporal/feedback state remains separate from live parameter layering
- [ ] subsystem docs and the architecture review reflect the final ownership model
## Open Questions
- Should committed live state remain physically in `RuntimeStore` for now, or move to a `CommittedLiveState` collaborator?
- Should transient OSC overlay updates become app-level typed events, or stay source-local through `RuntimeServiceLiveBridge`?
- Should overlay commit persistence be global, ingress-specific, or parameter-definition-driven?
- What compatibility rule should apply when shader reload preserves a control key but changes parameter shape?
- Should preset load clear all transient automation, or only automation that no longer maps to the loaded stack?
- Should UI slider drags use the committed-live layer directly, or a short-lived transient layer that commits on release?
## Short Version
Phase 5 should make live values boring and explicit.
Persisted state is durable truth. Committed live state is current-session/operator truth. Transient automation is high-rate overlay truth. Render consumes the composed result, and each layer has clear ownership, lifetime, persistence behavior, and reset/reload rules.

View File

@@ -300,16 +300,16 @@ Today the callback path effectively does this:
1. DeckLink signals completion. 1. DeckLink signals completion.
2. The callback path asks for a new output buffer. 2. The callback path asks for a new output buffer.
3. The callback path enters the shared GL section. 3. The callback path requests render-thread output production.
4. The callback path renders the next frame. 4. The render thread renders the next frame.
5. The callback path reads it back. 5. The render thread reads it back into the output buffer.
6. The callback path schedules the next hardware frame. 6. The callback path schedules the next hardware frame.
That path is visible in: That path is visible in:
- [OpenGLVideoIOBridge::RenderScheduledFrame](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/gl/pipeline/OpenGLVideoIOBridge.cpp:18) - [OpenGLVideoIOBridge::RenderScheduledFrame](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/gl/pipeline/OpenGLVideoIOBridge.cpp:18)
This couples output timing directly to render work. This no longer borrows the GL context from the callback thread, but it still couples output timing directly to render-thread work.
### Target Model ### Target Model
@@ -352,11 +352,11 @@ Recommended default policy for live playout:
- prefer recency over completeness - prefer recency over completeness
- drop stale capture frames instead of blocking render or output - drop stale capture frames instead of blocking render or output
The current "skip upload if the GL bridge is busy" behavior is directionally correct for live timing: The current latest-input mailbox behavior is directionally correct for live timing:
- [OpenGLVideoIOBridge::UploadInputFrame](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/gl/pipeline/OpenGLVideoIOBridge.cpp:11) - [OpenGLVideoIOBridge::UploadInputFrame](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/gl/pipeline/OpenGLVideoIOBridge.cpp:11)
But in the target architecture that decision should move out of GL lock acquisition and into an explicit backend-to-render handoff queue policy. The next improvement is to make the backend-to-render handoff policy more explicit in telemetry and playout scheduling, rather than treating it as only a render command mailbox detail.
Suggested input metrics: Suggested input metrics: