# RuntimeStore Subsystem Design This document expands the `RuntimeStore` portion of [PHASE_1_SUBSYSTEM_BOUNDARIES_DESIGN.md](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/docs/PHASE_1_SUBSYSTEM_BOUNDARIES_DESIGN.md) into a subsystem-specific design note. The purpose of `RuntimeStore` is to give the Phase 1 target architecture one clear home for durable runtime data. In the current codebase, that responsibility is spread through `RuntimeHost`, where persistence, mutation entrypoints, render-state building, shader metadata access, and status reporting all share the same object and lock domain. `RuntimeStore` is the design boundary that separates "what the app knows and saves" from "how the app decides to mutate it" and "how rendering consumes it." ## Role In The Phase 1 Architecture Within the Phase 1 subsystem model, `RuntimeStore` is the durable data authority. It exists to answer questions like: - what runtime configuration is currently loaded - what the saved layer stack structure is - what the saved parameter values are - what stack presets exist and what they contain - what package and manifest metadata is available for validation and snapshot building It should not answer questions like: - should this control mutation be allowed - should this OSC value be treated as transient or persisted - how should the render thread consume state - when should output frames be scheduled - what warnings should be shown to the operator That policy belongs elsewhere: - mutation policy: `RuntimeCoordinator` - render-facing publication: `RuntimeSnapshotProvider` - hardware timing: `VideoBackend` - operational visibility: `HealthTelemetry` ## Design Goals `RuntimeStore` should optimize for: - explicit ownership of durable runtime data - predictable disk-backed load and save behavior - minimal knowledge of GL, callbacks, or live playout timing - stable read models for validation and snapshot building - a clean seam for introducing debounced or asynchronous persistence later - testability without GPU or DeckLink dependencies ## Responsibilities `RuntimeStore` owns persisted and operator-authored state. Primary responsibilities: - load runtime host configuration from disk - load saved runtime state from disk - save runtime state snapshots to disk - own the stored layer stack model - own persisted parameter values and bypass flags - own stack preset serialization and deserialization - own package/manifest metadata needed across renders and reloads - expose query/read APIs over stored state - expose write APIs for coordinator-approved durable mutations - normalize or repair stored data at load boundaries when necessary Secondary responsibilities that still fit here: - path resolution for runtime state and preset files - preset name normalization/file-stem safety - compatibility handling for older saved-state schemas - default seeding of initial persistent state when no saved runtime exists ## Non-Responsibilities `RuntimeStore` must not become a general convenience layer again. It does not own: - render-thread timing - GL objects or resource lifetime - shader compilation orchestration - render-local transient state such as temporal history, feedback buffers, preview caches, or playout queues - OSC smoothing, coalescing, or overlay application - websocket broadcast policy - REST or OSC ingress handling - device callbacks, queue-depth policy, or preroll policy - app-wide health aggregation It also should not directly decide: - whether a mutation is valid in policy terms - whether a change should persist immediately, eventually, or not at all - when a new render snapshot should be published - whether a reload should be treated as config-only, package-only, or render-affecting Those are coordinator concerns, not store concerns. ## State Ownership `RuntimeStore` should own the following state categories. ### Runtime Configuration Examples: - server/control ports - OSC bind address - OSC smoothing defaults - runtime paths and directory configuration - any host-side configuration loaded from `config/runtime-host.json` This data is durable, file-backed, and not inherently render-local. ### Persistent Layer Stack State Examples: - ordered layer list - stable layer ids - selected shader id per layer - bypass state - persisted parameter values This is the stored "official" layer model, not a render-thread working copy. ### Stack Presets Examples: - preset names - serialized saved layer stacks under `runtime/stack_presets` Preset files are durable artifacts and should remain in the store domain even if later phases add async writing. ### Shader/Package Metadata Needed As Durable Reference Data Examples: - discovered shader package manifests - parameter definitions used for validation/default restoration - manifest-level capability metadata such as temporal history and feedback declarations - package ordering that should survive across reloads Important distinction: - manifest and package metadata belongs here - render-ready compiled programs and GPU resources do not ### Load-Time Compatibility/Repair State Examples: - schema version adaptation - default value filling for missing parameters - removal or migration of layers that reference missing packages - preset compatibility cleanup This should be treated as store hygiene during ingest, not runtime mutation policy. ## Data Model Boundaries `RuntimeStore` should present data in durable-model terms rather than live-render terms. Core model groupings: - `RuntimeConfigModel` - `PersistentLayerStackModel` - `LayerStoredState` - `StoredParameterValue` - `StackPresetModel` - `ShaderPackageCatalog` or equivalent durable package registry view The exact C++ types may differ from these names, but the boundary should hold: - store models describe durable intent - snapshot models describe render consumption That means `RuntimeStore` should not expose render-optimized structures such as `RuntimeRenderState` directly as its primary interface. ## Interface Shape The Phase 1 architecture doc already sketches the high-level interface. This section expands it. ### Load / Save Interface Expected responsibilities: - `LoadConfig()` - `LoadPersistentState()` - `SavePersistentStateSnapshot(...)` - `LoadStackPreset(...)` - `SaveStackPreset(...)` - `GetStackPresetNames()` Design notes: - `Load*` operations should parse and normalize external file content into durable in-memory models. - `Save*` operations should serialize durable models without needing render or control subsystem context. - later debounce/background writing should wrap these operations, not redefine their ownership ### Read Interface Expected responsibilities: - `GetRuntimeConfig()` - `GetStoredLayerStack()` - `FindStoredLayer(...)` - `GetShaderPackageCatalog()` - `GetStackPresetNames()` - `BuildPersistenceSnapshot()` or equivalent stable serialization input Design notes: - read APIs should support coordinator validation and snapshot building - read APIs should avoid exposing raw mutable internals across subsystem boundaries - stable read snapshots from the store are fine; render snapshots are still the snapshot provider's job ### Write Interface Expected responsibilities: - `SetStoredLayerStack(...)` - `ReplaceStoredLayer(...)` - `SetStoredParameterValue(...)` - `SetStoredBypassState(...)` - `SetStoredShaderSelection(...)` - `ReplaceShaderPackageCatalog(...)` Design notes: - writes should assume the coordinator already decided the mutation is allowed - store APIs may still enforce structural invariants and shape correctness - writes should not contain ingress-specific policy like OSC smoothing or UI throttling ### Normalization / Validation-Support Interface Expected responsibilities: - `NormalizeLoadedState(...)` - `EnsureStoredDefaults(...)` - `MakeSafePresetFileStem(...)` - package lookup helpers for parameter-definition queries Design notes: - lightweight structure and schema validation belongs here - policy validation belongs in the coordinator - render compatibility translation belongs in the snapshot provider ## Concurrency Expectations `RuntimeStore` should be designed as a shared data authority, but not as the app's global lock for everything. Phase 1 design expectations: - coordinator-driven writes may still be synchronized internally - read APIs should be safe for coordinator and snapshot-provider use - render should not directly take a large mutable store lock in the target architecture This implies: - `RuntimeStore` may keep an internal mutex during migration - that mutex should protect durable models only - render-facing consumers should eventually read via `RuntimeSnapshotProvider`, not by reaching into the store One of the main goals here is reducing the current situation where `RuntimeHost` lock scope effectively mixes: - persistent state - status reporting - render-state caches - timing stats - reload flags `RuntimeStore` should sharply narrow that scope. ## Dependency Rules Per the Phase 1 subsystem design, `RuntimeStore` should sit low in the dependency graph. Allowed inbound dependencies: - `RuntimeCoordinator -> RuntimeStore` - `RenderSnapshotBuilder -> RuntimeStore` - temporary migration shims from `ControlServices` only where explicitly tolerated Allowed outbound dependencies: - file/serialization helpers - package manifest parsing helpers - pure utility types Not allowed: - `RuntimeStore -> RenderEngine` - `RuntimeStore -> VideoBackend` - `RuntimeStore -> ControlServices` - `RuntimeStore -> HealthTelemetry` for behavior control The store may emit errors or return result objects, but it should not coordinate the rest of the system directly. ## Current Code Mapping Today, `RuntimeHost` contains most of the responsibilities that should move into `RuntimeStore`. Key current code paths: - config load: - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/legacy/RuntimeHost.cpp:1651) - persistent state load: - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/legacy/RuntimeHost.cpp:1748) - persistent state save: - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/legacy/RuntimeHost.cpp:1842) - preset save/load: - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/legacy/RuntimeHost.cpp:1286) - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/legacy/RuntimeHost.cpp:1304) - state serialization helpers: - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/legacy/RuntimeHost.cpp:2061) - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/legacy/RuntimeHost.cpp:2172) - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/legacy/RuntimeHost.cpp:2268) - path and file helpers: - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/legacy/RuntimeHost.cpp:1988) - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/legacy/RuntimeHost.cpp:2002) - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/legacy/RuntimeHost.cpp:2034) Durable-state mutation entrypoints that currently live on `RuntimeHost` but conceptually split between coordinator and store: - layer stack edits: - `AddLayer` - `RemoveLayer` - `MoveLayer` - `MoveLayerToIndex` - committed-state edits: - `SetLayerBypass` - `SetLayerShader` - `UpdateLayerParameter` - `ResetLayerParameters` The target split should be: - validation/policy/orchestration -> `RuntimeCoordinator` - durable state write application -> `RuntimeStore` Methods that should not move into `RuntimeStore` even though they currently live on `RuntimeHost`: - render-state building and caching: - `GetLayerRenderStates` - `TryRefreshCachedLayerStates` - `BuildLayerRenderStatesLocked` - status/timing reporting: - `SetSignalStatus` - `SetPerformanceStats` - `SetFramePacingStats` - `AdvanceFrame` - live reload flags/polling shell: - `PollFileChanges` - `ManualReloadRequested` - `ClearReloadRequest` Those belong under other target subsystems. ## Proposed Internal Subcomponents `RuntimeStore` does not need to be one monolithic class forever. A practical internal shape would be: - `RuntimeConfigStore` - runtime host config load and resolved paths The current codebase has begun this split: `RuntimeConfigStore` owns config parsing, path resolution, configured ports/formats, runtime roots, and shader compiler paths, while `RuntimeStore` keeps compatibility delegates for existing callers. - `LayerStackStore` - durable layer stack and parameter values - layer CRUD/reorder and shader selection - stack preset value serialization/load - `RuntimeStatePresenter` / `RuntimeStateJson` - runtime-state JSON assembly - layer-stack presentation serialization - `RenderSnapshotBuilder` - render-state assembly and parameter refresh - dynamic frame-field refresh and render snapshot version counters - `ShaderPackageCatalog` - durable manifest/package metadata - shader package scanning, status/order/lookup, and asset/source change comparison - `PersistenceWriter` helper - synchronous at first, async/debounced later The current codebase has begun the layer split: `LayerStackStore` owns durable layer state, layer CRUD/reorder, parameter persistence, and stack preset value serialization/load. `RuntimeStore` still owns locking, file IO, and compatibility delegates for existing callers. The current codebase has begun the render snapshot split: `RenderSnapshotBuilder` owns render-state assembly, cached parameter refresh, dynamic frame-field refresh, and render snapshot versions. `RuntimeSnapshotProvider` depends on this builder rather than on `RuntimeStore` friendship. The current codebase has also begun the presentation split: `RuntimeStatePresenter` owns top-level runtime-state JSON assembly, while `RuntimeStateJson` owns the layer-stack and parameter presentation shape used by runtime state clients. The current codebase has also begun the package split: `ShaderPackageCatalog` owns package scanning and registry comparison, while `RuntimeStore` uses it to keep layer state valid and to build compatibility read models. These can still be presented through one subsystem façade during migration. ## Persistence Model The store should treat persistence as durable snapshot management, not incremental side-effect spraying. Target behavior: - in-memory durable models are updated first - serialization snapshots are built from those models - save requests persist a coherent snapshot This matters because the current code still calls `SavePersistentState()` directly from many mutation paths. That is one of the architectural pressure points already called out in [ARCHITECTURE_RESILIENCE_REVIEW.md](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/docs/ARCHITECTURE_RESILIENCE_REVIEW.md). The Phase 1 design for `RuntimeStore` should therefore assume: - store ownership of serialization remains - immediate save-after-mutate is a migration detail, not the final behavioral contract By Phase 6, a background snapshot writer may sit underneath or beside this subsystem, but the durable model still belongs here. ## Migration Plan From Current Code The safest migration path is to extract responsibilities by interface, not by big-bang rename. ### Step 1: Introduce The `RuntimeStore` Name And Facade Create a façade interface that wraps the durable-data parts of `RuntimeHost`. Initial likely contents: - config load/save access - persistent layer-stack get/set access - preset load/save access - package catalog read access This stage is now past the initial compatibility point: `RuntimeStore` owns its durable/session backing fields directly rather than wrapping a `RuntimeHost` object. ### Step 2: Move Pure Persistence Helpers First Low-risk extractions: - path resolution helpers - file read/write helpers - preset enumeration and serialization helpers - persistent-state serialization/deserialization helpers These have relatively low coupling to GL and backend timing. ### Step 3: Split Durable Models From Render Cache/Status Fields Move out or conceptually separate: - `mPersistentState` - runtime config fields - preset roots and runtime roots - package catalog/order metadata From fields that should stay elsewhere: - render-state dirty flags and caches - status/timing counters - reload flags This is one of the most important separations in the whole program. ### Step 4: Route Durable Mutations Through Coordinator-Owned Policy Once the coordinator exists, `RuntimeStore` write calls should become lower-level and less policy-rich. Examples: - `SetStoredParameterValue(...)` rather than `ApplyOscTargetByControlKey(...)` - `ReplaceStoredLayerStack(...)` rather than `LoadStackPreset(...)` directly mutating every downstream concern ### Step 5: Keep Render Off The Store As `RuntimeSnapshotProvider` arrives, render should stop reading store internals directly. That is the moment where `RuntimeStore` becomes a proper durable authority instead of a shared mutable app center. ## Risks ### 1. Recreating `RuntimeHost` Under A New Name The biggest risk is calling something `RuntimeStore` while leaving policy, status, and render-cache behavior attached. Guardrail: - only durable data and store hygiene belong here ### 2. Letting Validation Drift Into Persistence Store-level shape validation is appropriate. High-level mutation policy is not. Risk examples: - store decides whether OSC should persist - store decides whether a layer reorder should trigger snapshot publication - store decides whether a reload is render-only or package-affecting Those are coordinator decisions. ### 3. Overexposing Mutable Internals If callers keep direct mutable access to the underlying vectors/maps, the subsystem boundary will exist only on paper. Guardrail: - prefer controlled write methods and stable read models ### 4. Coupling Package Metadata Too Tightly To Compile Outputs Package manifest and parameter-definition metadata belongs here. Compiled program state does not. Guardrail: - keep compile products and GPU artifacts out of the store ### 5. Using The Store Lock As A Global Synchronization Shortcut This would recreate timing and contention issues in a new form. Guardrail: - store locking protects durable models only - render synchronization must happen through snapshots, not by sharing the store lock ## Open Questions ### 1. How Much Shader Package Data Should Live Here? Clear yes: - manifest metadata - parameter definitions - package discovery/order information Still open: - whether compile-ready transformed sources belong here or in a later build-focused subsystem Current recommendation: - keep only durable reference/package metadata here ### 2. Should Committed Live State Be Co-Located With Persisted State? The Phase 1 parent doc leaves open whether committed live state stays in the store or is split with a live companion model owned by the coordinator. For `RuntimeStore`, the important rule is: - if a piece of state is part of the durable truth model, the store should own it - if it is transient or session-only, it should not be forced into the store just for convenience ### 3. Should Preset Application Be A Store Operation Or A Coordinator Operation? The file load and preset parse clearly belong here. The policy question of how a loaded preset affects live state, snapshot publication, overlays, and notifications belongs in the coordinator. Current recommendation: - `RuntimeStore` loads preset content - `RuntimeCoordinator` decides how to apply it ### 4. How Early Should Async Persistence Land? Phase 1 does not require it, but the store design should not block it. Current recommendation: - keep synchronous save semantics initially if needed - shape the interfaces so a background writer can be introduced without changing subsystem ownership ## Success Criteria For This Subsystem `RuntimeStore` can be considered well-defined once the codebase can say, without ambiguity: - all durable runtime config and saved layer data has one authoritative home - stack presets are owned by that same durable-data subsystem - render does not depend on store internals directly - timing/status/reporting state is no longer mixed into the same subsystem - persistence ownership is clear even before async persistence is introduced ## Short Version `RuntimeStore` is the subsystem that should answer: - what durable runtime data exists - what saved layer stack and parameters exist - what presets and package metadata exist - how that durable data is loaded and serialized It should not answer: - whether a mutation should happen - how rendering should consume state - how hardware pacing should work - what health warnings should be emitted If this boundary holds, later phases can safely split `RuntimeHost` without just recreating the same coupling under a different class name.