video-shader-toys/docs/CURRENT_SYSTEM_ARCHITECTURE.md

Current System Architecture

This document describes how the current RenderCadenceCompositor app works.

The implementation is cadence-first: the render/output path owns frame timing, while shader compilation, HTTP control, preview, persistence, and video I/O edges stay outside the render cadence loop wherever possible. The guardrails in Render Cadence Golden Rules are the operating contract.

Application Shape

The app is a native C++ OpenGL compositor with:

• optional DeckLink input
• optional DeckLink scheduled output
• a render-thread-owned OpenGL context
• runtime Slang shader packages from shaders/
• a configurable active layer stack
• a local HTTP/WebSocket control server
• optional Win32/GDI preview from system-memory output frames
• background runtime-state persistence
• cadence telemetry and log output

Primary source areas:

• src/app: startup/shutdown orchestration, config loading, runtime layer controller
• src/render: cadence clock, input texture upload, simple renderer, readback, and runtime GL support
• src/render/thread: render thread lifecycle, cadence loop, metrics, and runtime shader commit mailbox
• src/render/runtime: render-thread-owned runtime shader scene, renderer, text texture upload cache, and shared-context shader prepare worker
• src/frames: system-memory frame exchange
• src/video/core: generic video IO edge contracts, formats, and output scheduling thread
• src/video/decklink: current DeckLink input/output backend
• src/video/playout: backend-adjacent playout policy, queues, frame pools, and scheduling helpers
• src/video/legacy: older backend pipeline pieces kept separate from the current app path
• src/runtime/catalog: supported shader catalog and package filtering
• src/runtime/layers: app-side runtime layer model, restore, reload, and render snapshot construction
• src/runtime/shader: background Slang build bridge and prepared shader artifact types
• src/runtime/state: runtime JSON helpers, parameter normalization, and debounced runtime-state persistence
• src/runtime/text: MSDF/MTSDF font atlas build and CPU-side prepared text texture composition
• src/control: HTTP routing, command parsing, OpenAPI state JSON
• src/preview: optional non-consuming preview window
• src/telemetry and src/logging: runtime observation and logging

Startup

Startup broadly proceeds as:

1. Load config/runtime-host.json through AppConfigProvider, then apply CLI overrides.
2. Load the supported shader catalog from the configured shaderLibrary.
3. Start the runtime-state persistence writer.
4. Try to restore runtime/runtime_state.json.
5. If restore fails or no usable state exists, create one layer from the configured default shader.
6. Start the render thread.
7. Queue background Slang builds for every pending active layer.
8. Build a small completed-frame reserve.
9. Start optional preview, optional video output, telemetry, and HTTP control.

The runtime-state restore is intentionally app/control side work. The render thread does not read JSON, inspect the shader library, or decide what to compile.

Runtime Layer State

RuntimeLayerController owns the app-side layer model and coordinates:

• supported shader catalog loading
• layer add/remove/reorder/bypass/shader assignment
• parameter update/reset
• startup restore
• reload reconciliation
• background Slang builds
• render-layer publication
• runtime-state persistence requests

RuntimeLayerModel owns the in-memory active stack:

• layer id
• shader id and display name
• build state and message
• bypass state
• manifest parameter definitions
• current parameter values
• render-ready artifacts

The current durable runtime state is stored in runtime/runtime_state.json. It contains the active stack order, shader ids, bypass flags, and parameter values. On startup, valid saved layers are restored in order. Missing shader packages are skipped, invalid saved parameter values fall back to manifest defaults, and a missing or unusable file falls back to the configured default shader.

Manual stack preset routes are present in the UI/OpenAPI surface but are not implemented in the current native command path yet. runtime_state.json is the supported latest-working-state mechanism.

Persistence

RuntimeStatePersistenceWriter performs debounced background writes to runtime/runtime_state.json.

Durable UI/API mutations request persistence after they are accepted:

• add/remove layer
• reorder layer
• set bypass
• set shader
• update parameter
• reset parameters
• reload compatibility refresh

The mutation path snapshots the current layer model and hands serialized state to the writer. File IO happens on the persistence worker, not on the render thread or cadence path. Shutdown flushes the latest pending snapshot.

OSC-driven changes are intentionally not part of this autosave path yet.

Shader Reload

POST /api/reload and the control UI reload button:

• rescan shaders/
• re-read manifests
• rebuild the supported shader catalog
• update active layer metadata and parameter definitions from changed manifests
• preserve compatible parameter values
• default new or incompatible parameter values
• queue recompilation for every catalog-valid layer in the active stack

Reload does not compile every package in the shader library. A package is compiled when it is part of the active layer stack. If an active layer references a shader that no longer exists, that layer is marked failed and skipped. Existing render output remains active where possible until replacement builds are ready.

autoReload is still exposed in config/state for compatibility, but automatic file watching is not currently wired.

Render Ownership

The render thread owns the app OpenGL context during normal operation.

The render path consumes published render-layer snapshots. It does not:

• parse JSON
• scan files
• launch Slang
• run font atlas generation
• perform persistence
• handle HTTP or OSC
• call DeckLink discovery/setup APIs

When a runtime shader build completes, the app publishes a render-layer artifact. The render thread-owned runtime scene diffs the snapshot and queues changed pass programs to the shared-context prepare worker. The render thread swaps in an already-prepared render plan at a frame boundary.

Video And Preview

Video input and output are optional edges. DeckLink is the current concrete backend.

The input edge writes CPU frames into InputFrameMailbox. The current DeckLink backend captures BGRA8 directly where possible, or raw UYVY8 for render-thread GPU decode. The input edge does not call GL, render, preview, screenshot, shader, or output scheduling code.

The output edge consumes completed system-memory frames from SystemFrameExchange. The current DeckLink backend schedules those frames to DeckLink. If video output is unavailable, the app continues running render cadence, control, preview, telemetry, and logging.

PreviewWindowThread is optional and uses a non-consuming system-memory tap. It paints with Win32/GDI on its own thread and skips preview ticks instead of blocking the frame exchange.

Screenshot routes are present in the UI/OpenAPI surface but are not implemented in the current native command path yet.

Control Surface

The HTTP server runs on its own thread. It serves:

• UI assets
• OpenAPI/Swagger docs
• GET /api/state
• /ws state updates
• layer mutation POST routes
• /api/reload

Known but not implemented in the current native command path:

• /api/layers/move
• /api/stack-presets/save
• /api/stack-presets/load
• /api/screenshot

Unsupported routes return an action response with ok: false.

Tests

Native tests cover the main non-GL contracts:

• JSON parsing/serialization
• runtime parameter normalization
• shader package registry and Slang validation
• supported shader catalog
• runtime layer restore/reload behavior
• runtime-state persistence writer
• HTTP command parsing
• frame exchange and input mailbox behavior
• video format and scheduling helpers

Run:

cmake --build --preset build-debug --target RUN_TESTS --parallel