199 lines
11 KiB
Markdown
199 lines
11 KiB
Markdown
# Current System Architecture
|
|
|
|
This document describes how the current `RenderCadenceCompositor` app works.
|
|
|
|
The implementation is cadence-first: the render/output path owns frame timing, while shader compilation, HTTP control, preview, persistence, and video I/O edges stay outside the render cadence loop wherever possible. The guardrails in [Render Cadence Golden Rules](RENDER_CADENCE_GOLDEN_RULES.md) are the operating contract.
|
|
|
|
## Application Shape
|
|
|
|
The app is a native C++ OpenGL compositor with:
|
|
|
|
- optional DeckLink input
|
|
- optional DeckLink scheduled output
|
|
- a render-thread-owned OpenGL context
|
|
- a pluggable app-side runtime-content controller
|
|
- the default runtime Slang shader package stack from `shaders/`
|
|
- a local HTTP/WebSocket control server
|
|
- optional Win32/GDI preview from system-memory output frames
|
|
- background runtime-state persistence
|
|
- cadence telemetry and log output
|
|
|
|
Primary source areas:
|
|
|
|
- `src/app`: startup/shutdown orchestration, config loading, video backend factory, runtime-content controller boundary, and the default shader runtime-content controller
|
|
- `src/render`: cadence clock, input texture upload, render-content boundary, readback, and runtime GL support
|
|
- `src/render/thread`: render thread lifecycle, cadence loop, metrics, and runtime shader commit mailbox
|
|
- `src/render/runtime`: render-thread-owned runtime shader scene, renderer, text texture upload cache, and shared-context shader prepare worker
|
|
- `src/frames`: system-memory frame exchange
|
|
- `src/video/core`: generic video IO edge contracts, mode descriptions, formats, and output scheduling thread
|
|
- `src/video/decklink`: current DeckLink input/output backend
|
|
- `src/video/playout`: backend-adjacent playout policy, queues, frame pools, and scheduling helpers
|
|
- `src/video/legacy`: older backend pipeline pieces kept separate from the current app path
|
|
- `src/runtime/catalog`: supported shader catalog and package filtering
|
|
- `src/runtime/layers`: app-side runtime layer model, restore, reload, and render snapshot construction
|
|
- `src/runtime/shader`: background Slang build bridge and prepared shader artifact types
|
|
- `src/runtime/state`: runtime JSON helpers, parameter normalization, and debounced runtime-state persistence
|
|
- `src/runtime/text`: MSDF/MTSDF font atlas build and CPU-side prepared text texture composition
|
|
- `src/control`: command parsing, HTTP/WebSocket transport helpers, OSC status stub, OpenAPI state JSON
|
|
- `src/app/RenderCadenceHttpRoutes.*`: this app's current HTTP endpoint map
|
|
- `src/preview`: optional non-consuming preview window
|
|
- `src/telemetry` and `src/logging`: runtime observation and logging
|
|
|
|
## Startup
|
|
|
|
Startup broadly proceeds as:
|
|
|
|
1. Load `config/runtime-host.json` through `AppConfigProvider`, then apply CLI overrides.
|
|
2. Initialize the active `IRuntimeContentController`. The checked-in app uses `ShaderRuntimeContentController`, which loads the supported shader catalog, starts runtime-state persistence, and tries to restore `runtime/runtime_state.json`.
|
|
3. If restore fails or no usable state exists, the shader controller falls back to the optional configured startup shader. The checked-in config leaves `runtimeShaderId` empty, so a fresh host keeps the simple fallback renderer.
|
|
4. Start the render thread.
|
|
5. Start the active runtime-content controller. The shader controller queues background Slang builds for every pending active layer.
|
|
6. Build a small completed-frame reserve.
|
|
7. Start optional preview, optional video output, telemetry, and HTTP control.
|
|
|
|
The runtime-state restore is intentionally app/control side work. The render thread does not read JSON, inspect the shader library, or decide what to compile.
|
|
|
|
## Runtime Content And Shader Layer State
|
|
|
|
`RenderCadenceApp` owns startup, video output, preview, telemetry, OSC status, and HTTP server lifetime. It does not own the Slang shader stack directly. Runtime content plugs in through `IRuntimeContentController`.
|
|
|
|
The checked-in implementation is `ShaderRuntimeContentController`. It wraps `RuntimeLayerController`, exposes shader catalog/layer JSON for `/api/state`, handles shader layer POST commands, and publishes render-ready shader layer snapshots to the render thread.
|
|
|
|
`RuntimeLayerController` owns the app-side layer model and coordinates:
|
|
|
|
- supported shader catalog loading
|
|
- layer add/remove/reorder/bypass/shader assignment
|
|
- parameter update/reset
|
|
- startup restore
|
|
- reload reconciliation
|
|
- background Slang builds
|
|
- render-layer publication
|
|
- runtime-state persistence requests
|
|
|
|
`RuntimeLayerModel` owns the in-memory active stack:
|
|
|
|
- layer id
|
|
- shader id and display name
|
|
- build state and message
|
|
- bypass state
|
|
- manifest parameter definitions
|
|
- optional shader-declared custom UI metadata
|
|
- current parameter values
|
|
- render-ready artifacts
|
|
|
|
The current durable runtime state is stored in `runtime/runtime_state.json`. It contains the active stack order, shader ids, bypass flags, and parameter values. On startup, valid saved layers are restored in order. Missing shader packages are skipped, invalid saved parameter values fall back to manifest defaults, and a missing or unusable file falls back to the optional configured startup shader.
|
|
|
|
Manual stack preset routes are present in the UI/OpenAPI surface but are not implemented in the current native command path yet. `runtime_state.json` is the supported latest-working-state mechanism.
|
|
|
|
## Persistence
|
|
|
|
`RuntimeStatePersistenceWriter` performs debounced background writes to `runtime/runtime_state.json`.
|
|
|
|
Durable UI/API mutations request persistence after they are accepted:
|
|
|
|
- add/remove layer
|
|
- reorder layer
|
|
- set bypass
|
|
- set shader
|
|
- update parameter
|
|
- reset parameters
|
|
- reload compatibility refresh
|
|
|
|
The mutation path snapshots the current layer model and hands serialized state to the writer. File IO happens on the persistence worker, not on the render thread or cadence path. Shutdown flushes the latest pending snapshot.
|
|
|
|
OSC-driven changes are intentionally not part of this autosave path yet.
|
|
|
|
The host configuration editor is separate from runtime layer persistence. The UI reads active and saved startup config through `/api/config`, saves `config/runtime-host.json` through `/api/config/save`, and requests a native host restart through `/api/app/restart`. Render cadence, video input/output selection, resolution, frame rate, output pixel format, HTTP port, and preview settings are still startup-owned; they are not hot-swapped inside the cadence path.
|
|
|
|
## Shader Reload
|
|
|
|
`POST /api/reload` and the control UI reload button:
|
|
|
|
- rescan `shaders/`
|
|
- re-read manifests
|
|
- rebuild the supported shader catalog
|
|
- refresh optional shader custom UI metadata
|
|
- update active layer metadata and parameter definitions from changed manifests
|
|
- preserve compatible parameter values
|
|
- default new or incompatible parameter values
|
|
- queue recompilation for every catalog-valid layer in the active stack
|
|
|
|
Reload does not compile every package in the shader library. A package is compiled when it is part of the active layer stack. If an active layer references a shader that no longer exists, that layer is marked failed and skipped. Existing render output remains active where possible until replacement builds are ready.
|
|
|
|
`autoReload` is still exposed in config/state for compatibility, but automatic file watching is not currently wired.
|
|
|
|
## Render Ownership
|
|
|
|
The render thread owns the app OpenGL context during normal operation.
|
|
|
|
The render path consumes published render-layer snapshots. It does not:
|
|
|
|
- parse JSON
|
|
- scan files
|
|
- launch Slang
|
|
- run font atlas generation
|
|
- perform persistence
|
|
- handle HTTP or OSC
|
|
- call DeckLink discovery/setup APIs
|
|
|
|
When a runtime shader build completes, the default shader runtime-content controller publishes a render-layer artifact. The render thread forwards pending layer snapshots to the active render-content adapter. The default `RuntimeShaderRenderContent` owns the runtime scene, diffs the snapshot, and queues changed pass programs to the shared-context prepare worker. The render thread swaps in an already-prepared render plan at a frame boundary through that adapter.
|
|
|
|
## Video And Preview
|
|
|
|
Video input and output are optional edges. `input.backend` and `output.backend` select the concrete backend through the app-side backend factory. DeckLink and NDI are the current concrete backends, and `none` disables an edge. `input` and `output` also carry the device selector plus resolution/frame-rate settings. Configured video modes are represented in `src/video/core` and translated to backend-specific modes only inside the concrete edge.
|
|
|
|
The input edge writes CPU frames into `InputFrameMailbox`. The current DeckLink backend captures BGRA8 directly where possible, or raw UYVY8 for render-thread GPU decode. The input edge does not call GL, render, preview, screenshot, shader, or output scheduling code.
|
|
|
|
The output edge consumes completed system-memory frames from `SystemFrameExchange`. The render thread owns output pixel packing before readback: BGRA8 is read directly, and UYVY8 is packed on the GPU into a half-width RGBA8 target before async PBO readback. DeckLink and NDI output then schedule/send those completed CPU frames without invoking GL or converting pixels. If video output is unavailable, the app continues running render cadence, control, preview, telemetry, and logging.
|
|
|
|
Runtime state exposes backend-neutral output telemetry through `videoOutput`. Portable fields such as `enabled`, `backend`, and `scheduleFailures` stay at that level; backend-specific counters live under `videoOutput.backendMetrics`.
|
|
|
|
`PreviewWindowThread` is optional and uses a non-consuming system-memory tap. It paints BGRA8 directly, decodes UYVY8 only for preview presentation, and skips preview ticks instead of blocking the frame exchange.
|
|
|
|
Screenshot routes are present in the UI/OpenAPI surface but are not implemented in the current native command path yet.
|
|
|
|
## Control Surface
|
|
|
|
The HTTP server runs on its own thread. `HttpControlServer` owns socket lifetime, HTTP parsing, static asset helpers, OpenAPI/Swagger helper serving, and WebSocket state transport. `RenderCadenceHttpRoutes` owns this app's current endpoint map:
|
|
|
|
- UI assets
|
|
- shader package custom UI assets under `/shader-assets/{shaderId}/...`
|
|
- OpenAPI/Swagger docs
|
|
- `GET /api/state`
|
|
- `/ws` state updates
|
|
- layer mutation POST routes, dispatched to the active runtime-content controller
|
|
- `/api/reload`
|
|
|
|
Known but not implemented in the current native command path:
|
|
|
|
- `/api/layers/move`
|
|
- `/api/stack-presets/save`
|
|
- `/api/stack-presets/load`
|
|
- `/api/screenshot`
|
|
|
|
Unsupported routes return an action response with `ok: false`.
|
|
|
|
Forks can reuse the HTTP/WebSocket shell without keeping these endpoints by installing a different route callback.
|
|
|
|
`OscControlServer` is currently a lifecycle/status stub. It consumes startup OSC config, exposes configured/disabled/not-listening state through `/api/state`, and leaves UDP socket receive, OSC decode, and runtime dispatch unimplemented until that ingress boundary is built deliberately.
|
|
|
|
## Tests
|
|
|
|
Native tests cover the main non-GL contracts:
|
|
|
|
- JSON parsing/serialization
|
|
- runtime parameter normalization
|
|
- shader package registry and Slang validation
|
|
- supported shader catalog
|
|
- runtime layer restore/reload behavior
|
|
- runtime-state persistence writer
|
|
- HTTP transport and app-route dispatch
|
|
- frame exchange and input mailbox behavior
|
|
- video format and scheduling helpers
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
cmake --build --preset build-debug --target RUN_TESTS --parallel
|
|
```
|