# RuntimeStore Subsystem Design This document expands the `RuntimeStore` portion of [PHASE_1_SUBSYSTEM_BOUNDARIES_DESIGN.md](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/docs/PHASE_1_SUBSYSTEM_BOUNDARIES_DESIGN.md) into a subsystem-specific design note. The purpose of `RuntimeStore` is to give the Phase 1 target architecture one clear home for durable runtime data. Before the Phase 1 runtime split, that responsibility was spread through `RuntimeHost`, where persistence, mutation entrypoints, render-state building, shader metadata access, and status reporting all shared the same object and lock domain. `RuntimeStore` is the design boundary that separates "what the app knows and saves" from "how the app decides to mutate it" and "how rendering consumes it." ## Role In The Phase 1 Architecture Within the Phase 1 subsystem model, `RuntimeStore` is the durable data authority. It exists to answer questions like: - what runtime configuration is currently loaded - what the saved layer stack structure is - what the saved parameter values are - what stack presets exist and what they contain - what package and manifest metadata is available for validation and snapshot building It should not answer questions like: - should this control mutation be allowed - should this OSC value be treated as transient or persisted - how should the render thread consume state - when should output frames be scheduled - what warnings should be shown to the operator That policy belongs elsewhere: - mutation policy: `RuntimeCoordinator` - render-facing publication: `RuntimeSnapshotProvider` - hardware timing: `VideoBackend` - operational visibility: `HealthTelemetry` ## Design Goals `RuntimeStore` should optimize for: - explicit ownership of durable runtime data - predictable disk-backed load and save behavior - minimal knowledge of GL, callbacks, or live playout timing - stable read models for validation and snapshot building - a clean seam for introducing debounced or asynchronous persistence later - testability without GPU or DeckLink dependencies ## Responsibilities `RuntimeStore` owns persisted and operator-authored state. Primary responsibilities: - load runtime host configuration from disk - load saved runtime state from disk - save runtime state snapshots to disk - own the stored layer stack model - own persisted parameter values and bypass flags - own stack preset serialization and deserialization - own package/manifest metadata needed across renders and reloads - expose query/read APIs over stored state - expose write APIs for coordinator-approved durable mutations - normalize or repair stored data at load boundaries when necessary Secondary responsibilities that still fit here: - path resolution for runtime state and preset files - preset name normalization/file-stem safety - compatibility handling for older saved-state schemas - default seeding of initial persistent state when no saved runtime exists ## Non-Responsibilities `RuntimeStore` must not become a general convenience layer again. It does not own: - render-thread timing - GL objects or resource lifetime - shader compilation orchestration - render-local transient state such as temporal history, feedback buffers, preview caches, or playout queues - OSC smoothing, coalescing, or overlay application - websocket broadcast policy - REST or OSC ingress handling - device callbacks, queue-depth policy, or preroll policy - app-wide health aggregation It also should not directly decide: - whether a mutation is valid in policy terms - whether a change should persist immediately, eventually, or not at all - when a new render snapshot should be published - whether a reload should be treated as config-only, package-only, or render-affecting Those are coordinator concerns, not store concerns. ## State Ownership `RuntimeStore` should own the following state categories. Phase 5 names this boundary in code through `RuntimeStateLayerModel`: persisted layer stack data, saved parameter values, and stack presets are classified as base persisted state. Operator/session values are owned by `CommittedLiveState`; their mutation policy is committed-live policy owned by the coordinator, not durable-store policy by default. Phase 5 also adds `CommittedLiveState` as the physical owner of current session/operator layer state and `CommittedLiveStateReadModel` as the named read boundary for render snapshot publication. `RuntimeStore` still owns file IO, config, package metadata, preset persistence, and persistence requests, but it delegates current-session layer mutations to `CommittedLiveState`. ### Runtime Configuration Examples: - server/control ports - OSC bind address - OSC smoothing defaults - runtime paths and directory configuration - any host-side configuration loaded from `config/runtime-host.json` This data is durable, file-backed, and not inherently render-local. ### Persistent Layer Stack State Examples: - ordered layer list - stable layer ids - selected shader id per layer - bypass state - persisted parameter values This is the stored "official" layer model, not a render-thread working copy. ### Stack Presets Examples: - preset names - serialized saved layer stacks under `runtime/stack_presets` Preset files are durable artifacts and should remain in the store domain even if later phases add async writing. ### Shader/Package Metadata Needed As Durable Reference Data Examples: - discovered shader package manifests - parameter definitions used for validation/default restoration - manifest-level capability metadata such as temporal history and feedback declarations - package ordering that should survive across reloads Important distinction: - manifest and package metadata belongs here - render-ready compiled programs and GPU resources do not ### Load-Time Compatibility/Repair State Examples: - schema version adaptation - default value filling for missing parameters - removal or migration of layers that reference missing packages - preset compatibility cleanup This should be treated as store hygiene during ingest, not runtime mutation policy. ## Data Model Boundaries `RuntimeStore` should present data in durable-model terms rather than live-render terms. Core model groupings: - `RuntimeConfigModel` - `PersistentLayerStackModel` - `LayerStoredState` - `StoredParameterValue` - `StackPresetModel` - `ShaderPackageCatalog` or equivalent durable package registry view The exact C++ types may differ from these names, but the boundary should hold: - store models describe durable intent - snapshot models describe render consumption That means `RuntimeStore` should not expose render-optimized structures such as `RuntimeRenderState` directly as its primary interface. ## Interface Shape The Phase 1 architecture doc already sketches the high-level interface. This section expands it. ### Load / Save Interface Expected responsibilities: - `LoadConfig()` - `LoadPersistentState()` - `BuildPersistentStateSnapshot(...)` - `RequestPersistence(...)` - `LoadStackPreset(...)` - `SaveStackPreset(...)` - `GetStackPresetNames()` Design notes: - `Load*` operations should parse and normalize external file content into durable in-memory models. - `Save*` operations should serialize durable models without needing render or control subsystem context. - debounce/background writing wraps these operations rather than redefining store ownership ### Read Interface Expected responsibilities: - `GetRuntimeConfig()` - `GetStoredLayerStack()` - `FindStoredLayer(...)` - `GetShaderPackageCatalog()` - `GetStackPresetNames()` - `BuildPersistenceSnapshot()` or equivalent stable serialization input Design notes: - read APIs should support coordinator validation and snapshot building - read APIs should avoid exposing raw mutable internals across subsystem boundaries - stable read snapshots from the store are fine; render snapshots are still the snapshot provider's job ### Write Interface Expected responsibilities: - `SetStoredLayerStack(...)` - `ReplaceStoredLayer(...)` - `SetStoredParameterValue(...)` - `SetStoredBypassState(...)` - `SetStoredShaderSelection(...)` - `ReplaceShaderPackageCatalog(...)` Design notes: - writes should assume the coordinator already decided the mutation is allowed - store APIs may still enforce structural invariants and shape correctness - writes should not contain ingress-specific policy like OSC smoothing or UI throttling ### Normalization / Validation-Support Interface Expected responsibilities: - `NormalizeLoadedState(...)` - `EnsureStoredDefaults(...)` - `MakeSafePresetFileStem(...)` - package lookup helpers for parameter-definition queries Design notes: - lightweight structure and schema validation belongs here - policy validation belongs in the coordinator - render compatibility translation belongs in the snapshot provider ## Concurrency Expectations `RuntimeStore` should be designed as a shared data authority, but not as the app's global lock for everything. Phase 1 design expectations: - coordinator-driven writes may still be synchronized internally - read APIs should be safe for coordinator and snapshot-provider use - render should not directly take a large mutable store lock in the target architecture This implies: - `RuntimeStore` may keep an internal mutex during migration - that mutex should protect durable models only - render-facing consumers should eventually read via `RuntimeSnapshotProvider`, not by reaching into the store One of the main goals here is avoiding the old situation where runtime lock scope effectively mixed: - persistent state - status reporting - render-state caches - timing stats - reload flags `RuntimeStore` should sharply narrow that scope. ## Dependency Rules Per the Phase 1 subsystem design, `RuntimeStore` should sit low in the dependency graph. Allowed inbound dependencies: - `RuntimeCoordinator -> RuntimeStore` - `RenderSnapshotBuilder -> RuntimeStore` - temporary migration shims from `ControlServices` only where explicitly tolerated Allowed outbound dependencies: - file/serialization helpers - package manifest parsing helpers - pure utility types Not allowed: - `RuntimeStore -> RenderEngine` - `RuntimeStore -> VideoBackend` - `RuntimeStore -> ControlServices` - `RuntimeStore -> HealthTelemetry` for behavior control The store may emit errors or return result objects, but it should not coordinate the rest of the system directly. ## Current Code Mapping Before the Phase 1 runtime split, `RuntimeHost` contained many responsibilities that needed to move into `RuntimeStore` or adjacent runtime collaborators. Previous code paths: - config load: - `RuntimeHost.cpp` - persistent state load: - `RuntimeHost.cpp` - persistent state save: - `RuntimeHost.cpp` - preset save/load: - `RuntimeHost.cpp` - `RuntimeHost.cpp` - state serialization helpers: - `RuntimeHost.cpp` - `RuntimeHost.cpp` - `RuntimeHost.cpp` - path and file helpers: - `RuntimeHost.cpp` - `RuntimeHost.cpp` - `RuntimeHost.cpp` Durable-state mutation entrypoints that previously lived on `RuntimeHost` but conceptually split between coordinator and store: - layer stack edits: - `AddLayer` - `RemoveLayer` - `MoveLayer` - `MoveLayerToIndex` - committed-state edits: - `SetLayerBypass` - `SetLayerShader` - `UpdateLayerParameter` - `ResetLayerParameters` The target split should be: - validation/policy/orchestration -> `RuntimeCoordinator` - durable state write application -> `RuntimeStore` Methods that were intentionally not moved into `RuntimeStore` because they belong under other runtime subsystems: - render-state building and caching: - `GetLayerRenderStates` - `TryRefreshCachedLayerStates` - `BuildLayerRenderStatesLocked` - status/timing reporting: - `SetSignalStatus` - `SetPerformanceStats` - `SetFramePacingStats` - `AdvanceFrame` - live reload flags/polling shell: - `PollFileChanges` - `ManualReloadRequested` - `ClearReloadRequest` Those belong under other target subsystems. ## Proposed Internal Subcomponents `RuntimeStore` does not need to be one monolithic class forever. A practical internal shape would be: - `RuntimeConfigStore` - runtime host config load and resolved paths The current codebase has completed this part of the split: `RuntimeConfigStore` owns config parsing, path resolution, configured ports/formats, runtime roots, and shader compiler paths, while `RuntimeStore` exposes compatibility-shaped delegates for existing callers. - `CommittedLiveState` - current committed/session layer stack and parameter values - layer CRUD/reorder and shader selection for the running session - committed-live read model for snapshot publication - `LayerStackStore` - backing layer stack mechanics used by committed-live state - layer CRUD/reorder and shader selection helpers - stack preset value serialization/load helpers - `RuntimeStatePresenter` / `RuntimeStateJson` - runtime-state JSON assembly - layer-stack presentation serialization - `RenderSnapshotBuilder` - render-state assembly and parameter refresh - dynamic frame-field refresh and render snapshot version counters - `ShaderPackageCatalog` - durable manifest/package metadata - shader package scanning, status/order/lookup, and asset/source change comparison - `PersistenceWriter` helper - synchronous at first, async/debounced later The current codebase has completed the committed-live split: `CommittedLiveState` owns current committed/session layer state using `LayerStackStore` backing mechanics. `RuntimeStore` keeps file IO, package metadata, persistence serialization, persistence requests, preset file access, and facade methods for existing callers. The current codebase has completed the render snapshot split: `RenderSnapshotBuilder` owns render-state assembly, cached parameter refresh, dynamic frame-field refresh, and render snapshot versions. `RuntimeSnapshotProvider` depends on this builder rather than on `RuntimeStore` friendship. The current codebase has also completed the presentation split: `RuntimeStatePresenter` owns top-level runtime-state JSON assembly, while `RuntimeStateJson` owns the layer-stack and parameter presentation shape used by runtime state clients. The current codebase has also completed the package split: `ShaderPackageCatalog` owns package scanning and registry comparison, while `RuntimeStore` uses it to keep layer state valid and to build compatibility read models. These can still be presented through one subsystem façade during migration. ## Persistence Model The store should treat persistence as durable snapshot management, not incremental side-effect spraying. Target behavior: - in-memory durable models are updated first - serialization snapshots are built from those models - save requests persist a coherent snapshot This matters because earlier code called persistent-state saves directly from mutation paths. Phase 6 removed that pressure point: accepted durable mutations now publish persistence requests, and `RuntimeStore::RequestPersistence(...)` builds a coherent snapshot for the background writer. The Phase 1 design for `RuntimeStore` should therefore assume: - store ownership of serialization remains - persistence requests, not mutation methods, are the durable write boundary Phase 6 added that background snapshot writer underneath this subsystem, while keeping the durable model here. ## Migration Plan From Current Code The safest migration path is to extract responsibilities by interface, not by big-bang rename. ### Step 1: Introduce The `RuntimeStore` Name And Facade Create a facade interface for the durable-data parts that used to live in `RuntimeHost`. Initial likely contents: - config load/save access - persistent layer-stack get/set access - preset load/save access - package catalog read access This stage is complete: `RuntimeStore` owns its durable/session backing fields directly rather than wrapping a `RuntimeHost` object. ### Step 2: Move Pure Persistence Helpers First Low-risk extractions: - path resolution helpers - file read/write helpers - preset enumeration and serialization helpers - persistent-state serialization/deserialization helpers These have relatively low coupling to GL and backend timing. ### Step 3: Split Durable Models From Render Cache/Status Fields Move out or conceptually separate: - `mPersistentState` - runtime config fields - preset roots and runtime roots - package catalog/order metadata From fields that should stay elsewhere: - render-state dirty flags and caches - status/timing counters - reload flags This is one of the most important separations in the whole program. ### Step 4: Route Durable Mutations Through Coordinator-Owned Policy Once the coordinator exists, `RuntimeStore` write calls should become lower-level and less policy-rich. Examples: - `SetStoredParameterValue(...)` rather than `ApplyOscTargetByControlKey(...)` - `ReplaceStoredLayerStack(...)` rather than `LoadStackPreset(...)` directly mutating every downstream concern ### Step 5: Keep Render Off The Store As `RuntimeSnapshotProvider` arrives, render should stop reading store internals directly. That is the moment where `RuntimeStore` becomes a proper durable authority instead of a shared mutable app center. ## Risks ### 1. Recreating `RuntimeHost` Under A New Name The biggest risk is calling something `RuntimeStore` while leaving policy, status, and render-cache behavior attached. Guardrail: - only durable data and store hygiene belong here ### 2. Letting Validation Drift Into Persistence Store-level shape validation is appropriate. High-level mutation policy is not. Risk examples: - store decides whether OSC should persist - store decides whether a layer reorder should trigger snapshot publication - store decides whether a reload is render-only or package-affecting Those are coordinator decisions. ### 3. Overexposing Mutable Internals If callers keep direct mutable access to the underlying vectors/maps, the subsystem boundary will exist only on paper. Guardrail: - prefer controlled write methods and stable read models ### 4. Coupling Package Metadata Too Tightly To Compile Outputs Package manifest and parameter-definition metadata belongs here. Compiled program state does not. Guardrail: - keep compile products and GPU artifacts out of the store ### 5. Using The Store Lock As A Global Synchronization Shortcut This would recreate timing and contention issues in a new form. Guardrail: - store locking protects durable models only - render synchronization must happen through snapshots, not by sharing the store lock ## Open Questions ### 1. How Much Shader Package Data Should Live Here? Clear yes: - manifest metadata - parameter definitions - package discovery/order information Still open: - whether compile-ready transformed sources belong here or in a later build-focused subsystem Current recommendation: - keep only durable reference/package metadata here ### 2. Should Preset Application Be A Store Operation Or A Coordinator Operation? The file load and preset parse clearly belong here. The policy question of how a loaded preset affects live state, snapshot publication, overlays, and notifications belongs in the coordinator. Current recommendation: - `RuntimeStore` loads preset content - `RuntimeCoordinator` decides how to apply it ### 3. How Early Should Async Persistence Land? Phase 1 does not require it, but the store design should not block it. Current recommendation: - keep synchronous save semantics initially if needed - shape the interfaces so a background writer can be introduced without changing subsystem ownership ## Success Criteria For This Subsystem `RuntimeStore` can be considered well-defined once the codebase can say, without ambiguity: - all durable runtime config and saved layer data has one authoritative home - stack presets are owned by that same durable-data subsystem - render does not depend on store internals directly - timing/status/reporting state is no longer mixed into the same subsystem - persistence ownership is clear even before async persistence is introduced ## Short Version `RuntimeStore` is the subsystem that should answer: - what durable runtime data exists - what saved layer stack and parameters exist - what presets and package metadata exist - how that durable data is loaded and serialized It should not answer: - whether a mutation should happen - how rendering should consume state - how hardware pacing should work - what health warnings should be emitted If this boundary holds, later phases can continue without recreating the old coupling under a different class name.