video-shader-toys/docs/subsystems/RuntimeStore.md

# RuntimeStore Subsystem Design

This document expands the `RuntimeStore` portion of [PHASE_1_SUBSYSTEM_BOUNDARIES_DESIGN.md](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/docs/PHASE_1_SUBSYSTEM_BOUNDARIES_DESIGN.md) into a subsystem-specific design note.

The purpose of `RuntimeStore` is to give the Phase 1 target architecture one clear home for durable runtime data. In the current codebase, that responsibility is spread through `RuntimeHost`, where persistence, mutation entrypoints, render-state building, shader metadata access, and status reporting all share the same object and lock domain. `RuntimeStore` is the design boundary that separates "what the app knows and saves" from "how the app decides to mutate it" and "how rendering consumes it."

## Role In The Phase 1 Architecture

Within the Phase 1 subsystem model, `RuntimeStore` is the durable data authority.

It exists to answer questions like:

- what runtime configuration is currently loaded
- what the saved layer stack structure is
- what the saved parameter values are
- what stack presets exist and what they contain
- what package and manifest metadata is available for validation and snapshot building

It should not answer questions like:

- should this control mutation be allowed
- should this OSC value be treated as transient or persisted
- how should the render thread consume state
- when should output frames be scheduled
- what warnings should be shown to the operator

That policy belongs elsewhere:

- mutation policy: `RuntimeCoordinator`
- render-facing publication: `RuntimeSnapshotProvider`
- hardware timing: `VideoBackend`
- operational visibility: `HealthTelemetry`

## Design Goals

`RuntimeStore` should optimize for:

- explicit ownership of durable runtime data
- predictable disk-backed load and save behavior
- minimal knowledge of GL, callbacks, or live playout timing
- stable read models for validation and snapshot building
- a clean seam for introducing debounced or asynchronous persistence later
- testability without GPU or DeckLink dependencies

## Responsibilities

`RuntimeStore` owns persisted and operator-authored state.

Primary responsibilities:

- load runtime host configuration from disk
- load saved runtime state from disk
- save runtime state snapshots to disk
- own the stored layer stack model
- own persisted parameter values and bypass flags
- own stack preset serialization and deserialization
- own package/manifest metadata needed across renders and reloads
- expose query/read APIs over stored state
- expose write APIs for coordinator-approved durable mutations
- normalize or repair stored data at load boundaries when necessary

Secondary responsibilities that still fit here:

- path resolution for runtime state and preset files
- preset name normalization/file-stem safety
- compatibility handling for older saved-state schemas
- default seeding of initial persistent state when no saved runtime exists

## Non-Responsibilities

`RuntimeStore` must not become a general convenience layer again.

It does not own:

- render-thread timing
- GL objects or resource lifetime
- shader compilation orchestration
- render-local transient state such as temporal history, feedback buffers, preview caches, or playout queues
- OSC smoothing, coalescing, or overlay application
- websocket broadcast policy
- REST or OSC ingress handling
- device callbacks, queue-depth policy, or preroll policy
- app-wide health aggregation

It also should not directly decide:

- whether a mutation is valid in policy terms
- whether a change should persist immediately, eventually, or not at all
- when a new render snapshot should be published
- whether a reload should be treated as config-only, package-only, or render-affecting

Those are coordinator concerns, not store concerns.

## State Ownership

`RuntimeStore` should own the following state categories.

### Runtime Configuration

Examples:

- server/control ports
- OSC bind address
- OSC smoothing defaults
- runtime paths and directory configuration
- any host-side configuration loaded from `config/runtime-host.json`

This data is durable, file-backed, and not inherently render-local.

### Persistent Layer Stack State

Examples:

- ordered layer list
- stable layer ids
- selected shader id per layer
- bypass state
- persisted parameter values

This is the stored "official" layer model, not a render-thread working copy.

### Stack Presets

Examples:

- preset names
- serialized saved layer stacks under `runtime/stack_presets`

Preset files are durable artifacts and should remain in the store domain even if later phases add async writing.

### Shader/Package Metadata Needed As Durable Reference Data

Examples:

- discovered shader package manifests
- parameter definitions used for validation/default restoration
- manifest-level capability metadata such as temporal history and feedback declarations
- package ordering that should survive across reloads

Important distinction:

- manifest and package metadata belongs here
- render-ready compiled programs and GPU resources do not

### Load-Time Compatibility/Repair State

Examples:

- schema version adaptation
- default value filling for missing parameters
- removal or migration of layers that reference missing packages
- preset compatibility cleanup

This should be treated as store hygiene during ingest, not runtime mutation policy.

## Data Model Boundaries

`RuntimeStore` should present data in durable-model terms rather than live-render terms.

Core model groupings:

- `RuntimeConfigModel`
- `PersistentLayerStackModel`
- `LayerStoredState`
- `StoredParameterValue`
- `StackPresetModel`
- `ShaderPackageCatalog` or equivalent durable package registry view

The exact C++ types may differ from these names, but the boundary should hold:

- store models describe durable intent
- snapshot models describe render consumption

That means `RuntimeStore` should not expose render-optimized structures such as `RuntimeRenderState` directly as its primary interface.

## Interface Shape

The Phase 1 architecture doc already sketches the high-level interface. This section expands it.

### Load / Save Interface

Expected responsibilities:

- `LoadConfig()`
- `LoadPersistentState()`
- `SavePersistentStateSnapshot(...)`
- `LoadStackPreset(...)`
- `SaveStackPreset(...)`
- `GetStackPresetNames()`

Design notes:

- `Load*` operations should parse and normalize external file content into durable in-memory models.
- `Save*` operations should serialize durable models without needing render or control subsystem context.
- later debounce/background writing should wrap these operations, not redefine their ownership

### Read Interface

Expected responsibilities:

- `GetRuntimeConfig()`
- `GetStoredLayerStack()`
- `FindStoredLayer(...)`
- `GetShaderPackageCatalog()`
- `GetStackPresetNames()`
- `BuildPersistenceSnapshot()` or equivalent stable serialization input

Design notes:

- read APIs should support coordinator validation and snapshot building
- read APIs should avoid exposing raw mutable internals across subsystem boundaries
- stable read snapshots from the store are fine; render snapshots are still the snapshot provider's job

### Write Interface

Expected responsibilities:

- `SetStoredLayerStack(...)`
- `ReplaceStoredLayer(...)`
- `SetStoredParameterValue(...)`
- `SetStoredBypassState(...)`
- `SetStoredShaderSelection(...)`
- `ReplaceShaderPackageCatalog(...)`

Design notes:

- writes should assume the coordinator already decided the mutation is allowed
- store APIs may still enforce structural invariants and shape correctness
- writes should not contain ingress-specific policy like OSC smoothing or UI throttling

### Normalization / Validation-Support Interface

Expected responsibilities:

- `NormalizeLoadedState(...)`
- `EnsureStoredDefaults(...)`
- `MakeSafePresetFileStem(...)`
- package lookup helpers for parameter-definition queries

Design notes:

- lightweight structure and schema validation belongs here
- policy validation belongs in the coordinator
- render compatibility translation belongs in the snapshot provider

## Concurrency Expectations

`RuntimeStore` should be designed as a shared data authority, but not as the app's global lock for everything.

Phase 1 design expectations:

- coordinator-driven writes may still be synchronized internally
- read APIs should be safe for coordinator and snapshot-provider use
- render should not directly take a large mutable store lock in the target architecture

This implies:

- `RuntimeStore` may keep an internal mutex during migration
- that mutex should protect durable models only
- render-facing consumers should eventually read via `RuntimeSnapshotProvider`, not by reaching into the store

One of the main goals here is reducing the current situation where `RuntimeHost` lock scope effectively mixes:

- persistent state
- status reporting
- render-state caches
- timing stats
- reload flags

`RuntimeStore` should sharply narrow that scope.

## Dependency Rules

Per the Phase 1 subsystem design, `RuntimeStore` should sit low in the dependency graph.

Allowed inbound dependencies:

- `RuntimeCoordinator -> RuntimeStore`
- `RuntimeSnapshotProvider -> RuntimeStore`
- temporary migration shims from `ControlServices` only where explicitly tolerated

Allowed outbound dependencies:

- file/serialization helpers
- package manifest parsing helpers
- pure utility types

Not allowed:

- `RuntimeStore -> RenderEngine`
- `RuntimeStore -> VideoBackend`
- `RuntimeStore -> ControlServices`
- `RuntimeStore -> HealthTelemetry` for behavior control

The store may emit errors or return result objects, but it should not coordinate the rest of the system directly.

## Current Code Mapping

Today, `RuntimeHost` contains most of the responsibilities that should move into `RuntimeStore`.

Key current code paths:

- config load:
  - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/RuntimeHost.cpp:1651)
- persistent state load:
  - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/RuntimeHost.cpp:1748)
- persistent state save:
  - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/RuntimeHost.cpp:1842)
- preset save/load:
  - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/RuntimeHost.cpp:1286)
  - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/RuntimeHost.cpp:1304)
- state serialization helpers:
  - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/RuntimeHost.cpp:2061)
  - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/RuntimeHost.cpp:2172)
  - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/RuntimeHost.cpp:2268)
- path and file helpers:
  - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/RuntimeHost.cpp:1988)
  - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/RuntimeHost.cpp:2002)
  - [RuntimeHost.cpp](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/apps/LoopThroughWithOpenGLCompositing/runtime/RuntimeHost.cpp:2034)

Durable-state mutation entrypoints that currently live on `RuntimeHost` but conceptually split between coordinator and store:

- layer stack edits:
  - `AddLayer`
  - `RemoveLayer`
  - `MoveLayer`
  - `MoveLayerToIndex`
- committed-state edits:
  - `SetLayerBypass`
  - `SetLayerShader`
  - `UpdateLayerParameter`
  - `ResetLayerParameters`

The target split should be:

- validation/policy/orchestration -> `RuntimeCoordinator`
- durable state write application -> `RuntimeStore`

Methods that should not move into `RuntimeStore` even though they currently live on `RuntimeHost`:

- render-state building and caching:
  - `GetLayerRenderStates`
  - `TryRefreshCachedLayerStates`
  - `BuildLayerRenderStatesLocked`
- status/timing reporting:
  - `SetSignalStatus`
  - `SetPerformanceStats`
  - `SetFramePacingStats`
  - `AdvanceFrame`
- live reload flags/polling shell:
  - `PollFileChanges`
  - `ManualReloadRequested`
  - `ClearReloadRequest`

Those belong under other target subsystems.

## Proposed Internal Subcomponents

`RuntimeStore` does not need to be one monolithic class forever. A practical internal shape would be:

- `RuntimeConfigStore`
  - runtime host config load/save and resolved paths
- `PersistentLayerStore`
  - durable layer stack and parameter values
- `StackPresetStore`
  - preset enumeration/load/save
- `ShaderPackageCatalogStore`
  - durable manifest/package metadata
- `PersistenceWriter` helper
  - synchronous at first, async/debounced later

These can still be presented through one subsystem façade during migration.

## Persistence Model

The store should treat persistence as durable snapshot management, not incremental side-effect spraying.

Target behavior:

- in-memory durable models are updated first
- serialization snapshots are built from those models
- save requests persist a coherent snapshot

This matters because the current code still calls `SavePersistentState()` directly from many mutation paths. That is one of the architectural pressure points already called out in [ARCHITECTURE_RESILIENCE_REVIEW.md](/c:/Users/Aiden/Documents/GitHub/video-shader-toys/docs/ARCHITECTURE_RESILIENCE_REVIEW.md).

The Phase 1 design for `RuntimeStore` should therefore assume:

- store ownership of serialization remains
- immediate save-after-mutate is a migration detail, not the final behavioral contract

By Phase 6, a background snapshot writer may sit underneath or beside this subsystem, but the durable model still belongs here.

## Migration Plan From Current Code

The safest migration path is to extract responsibilities by interface, not by big-bang rename.

### Step 1: Introduce The `RuntimeStore` Name And Facade

Create a façade interface that wraps the durable-data parts of `RuntimeHost`.

Initial likely contents:

- config load/save access
- persistent layer-stack get/set access
- preset load/save access
- package catalog read access

This stage is now past the initial compatibility point: `RuntimeStore` owns its durable/session backing fields directly rather than wrapping a `RuntimeHost` object.

### Step 2: Move Pure Persistence Helpers First

Low-risk extractions:

- path resolution helpers
- file read/write helpers
- preset enumeration and serialization helpers
- persistent-state serialization/deserialization helpers

These have relatively low coupling to GL and backend timing.

### Step 3: Split Durable Models From Render Cache/Status Fields

Move out or conceptually separate:

- `mPersistentState`
- runtime config fields
- preset roots and runtime roots
- package catalog/order metadata

From fields that should stay elsewhere:

- render-state dirty flags and caches
- status/timing counters
- reload flags

This is one of the most important separations in the whole program.

### Step 4: Route Durable Mutations Through Coordinator-Owned Policy

Once the coordinator exists, `RuntimeStore` write calls should become lower-level and less policy-rich.

Examples:

- `SetStoredParameterValue(...)` rather than `ApplyOscTargetByControlKey(...)`
- `ReplaceStoredLayerStack(...)` rather than `LoadStackPreset(...)` directly mutating every downstream concern

### Step 5: Keep Render Off The Store

As `RuntimeSnapshotProvider` arrives, render should stop reading store internals directly.

That is the moment where `RuntimeStore` becomes a proper durable authority instead of a shared mutable app center.

## Risks

### 1. Recreating `RuntimeHost` Under A New Name

The biggest risk is calling something `RuntimeStore` while leaving policy, status, and render-cache behavior attached.

Guardrail:

- only durable data and store hygiene belong here

### 2. Letting Validation Drift Into Persistence

Store-level shape validation is appropriate. High-level mutation policy is not.

Risk examples:

- store decides whether OSC should persist
- store decides whether a layer reorder should trigger snapshot publication
- store decides whether a reload is render-only or package-affecting

Those are coordinator decisions.

### 3. Overexposing Mutable Internals

If callers keep direct mutable access to the underlying vectors/maps, the subsystem boundary will exist only on paper.

Guardrail:

- prefer controlled write methods and stable read models

### 4. Coupling Package Metadata Too Tightly To Compile Outputs

Package manifest and parameter-definition metadata belongs here. Compiled program state does not.

Guardrail:

- keep compile products and GPU artifacts out of the store

### 5. Using The Store Lock As A Global Synchronization Shortcut

This would recreate timing and contention issues in a new form.

Guardrail:

- store locking protects durable models only
- render synchronization must happen through snapshots, not by sharing the store lock

## Open Questions

### 1. How Much Shader Package Data Should Live Here?

Clear yes:

- manifest metadata
- parameter definitions
- package discovery/order information

Still open:

- whether compile-ready transformed sources belong here or in a later build-focused subsystem

Current recommendation:

- keep only durable reference/package metadata here

### 2. Should Committed Live State Be Co-Located With Persisted State?

The Phase 1 parent doc leaves open whether committed live state stays in the store or is split with a live companion model owned by the coordinator.

For `RuntimeStore`, the important rule is:

- if a piece of state is part of the durable truth model, the store should own it
- if it is transient or session-only, it should not be forced into the store just for convenience

### 3. Should Preset Application Be A Store Operation Or A Coordinator Operation?

The file load and preset parse clearly belong here.

The policy question of how a loaded preset affects live state, snapshot publication, overlays, and notifications belongs in the coordinator.

Current recommendation:

- `RuntimeStore` loads preset content
- `RuntimeCoordinator` decides how to apply it

### 4. How Early Should Async Persistence Land?

Phase 1 does not require it, but the store design should not block it.

Current recommendation:

- keep synchronous save semantics initially if needed
- shape the interfaces so a background writer can be introduced without changing subsystem ownership

## Success Criteria For This Subsystem

`RuntimeStore` can be considered well-defined once the codebase can say, without ambiguity:

- all durable runtime config and saved layer data has one authoritative home
- stack presets are owned by that same durable-data subsystem
- render does not depend on store internals directly
- timing/status/reporting state is no longer mixed into the same subsystem
- persistence ownership is clear even before async persistence is introduced

## Short Version

`RuntimeStore` is the subsystem that should answer:

- what durable runtime data exists
- what saved layer stack and parameters exist
- what presets and package metadata exist
- how that durable data is loaded and serialized

It should not answer:

- whether a mutation should happen
- how rendering should consume state
- how hardware pacing should work
- what health warnings should be emitted

If this boundary holds, later phases can safely split `RuntimeHost` without just recreating the same coupling under a different class name.