video-shader-toys/README.md

# Video Shader

Native video shader host with an OpenGL/DeckLink render path, Slang shader packages, and a local React control UI.

The app loads shader packages from `shaders/`, compiles Slang to GLSL at runtime, renders a configurable layer stack, and exposes a browser-based control surface over a local HTTP/WebSocket server.

## Repository Layout

- `apps/LoopThroughWithOpenGLCompositing/`: native C++ host app.
- `shaders/`: shader packages, each with `shader.json` and `shader.slang`.
- `ui/`: Vite/React control UI.
- `config/runtime-host.json`: runtime configuration.
- `runtime/templates/`: tracked shader wrapper templates.
- `runtime/`: ignored generated runtime cache/state output. See `runtime/README.md`.
- `tests/`: focused native tests for pure runtime logic.
- `.gitea/workflows/ci.yml`: Gitea Actions CI for Windows native tests and Ubuntu UI build.

## Requirements

- Windows with Visual Studio 2022 C++ tooling.
- CMake 3.24 or newer.
- Node.js and npm for the control UI.
- Blackmagic DeckLink SDK 16.0 with the NVIDIA GPUDirect sample files available locally.
- Slang compiler available under the repo/tooling paths expected by the runtime, or otherwise discoverable by the existing app setup.

The Blackmagic/GPUDirect SDK should not be committed to this repository. `CMakeLists.txt` exposes `GPUDIRECT_DIR` as a cache path so local machines and CI runners can point at their installed SDK location.

Default expected SDK path:

```text
3rdParty/Blackmagic DeckLink SDK 16.0/Win/Samples/NVIDIA_GPUDirect
```

Override example:

```powershell
cmake --preset vs2022-x64-debug -DGPUDIRECT_DIR="D:/SDKs/Blackmagic DeckLink SDK 16.0/Win/Samples/NVIDIA_GPUDirect"
```

## Build

Configure and build the native app:

```powershell
cmake --preset vs2022-x64-debug
cmake --build --preset build-debug
```

Build the React control UI:

```powershell
cd ui
npm ci
npm run build
```

The native app serves `ui/dist` when it exists, otherwise it falls back to the source UI directory during development.

## Package

Build the UI, build the native Release target, then install into a portable runtime folder:

```powershell
cd ui
npm ci
npm run build
cd ..
cmake --preset vs2022-x64-release
cmake --build --preset build-release
cmake --install build/vs2022-x64-release --config Release --prefix dist/VideoShader
```

The package folder will contain:

```text
dist/VideoShader/
  LoopThroughWithOpenGLCompositing.exe
  dvp.dll
  config/
  shaders/
  ui/dist/
  runtime/templates/
```

You can run `LoopThroughWithOpenGLCompositing.exe` directly from that folder. In packaged mode, the app resolves `config/`, `shaders/`, `ui/dist/`, and `runtime/templates/` relative to the exe folder. In development mode, it still falls back to repo-root discovery.

Create a zip for distribution:

```powershell
Compress-Archive -Path dist/VideoShader/* -DestinationPath dist/VideoShader.zip -Force
```

## Tests

Run native tests:

```powershell
cmake --build --preset build-debug --target RUN_TESTS
```

Run the UI production build check:

```powershell
cd ui
npm run build
```

Current native test coverage includes:

- JSON parsing and serialization.
- Parameter normalization and preset filename safety.
- Shader manifest parsing and package registry scanning.

## Runtime Configuration

`config/runtime-host.json` controls host behavior:

```json
{
  "shaderLibrary": "shaders",
  "serverPort": 8080,
  "videoFormat": "1080p",
  "frameRate": "59.94",
  "autoReload": true,
  "maxTemporalHistoryFrames": 12,
  "enableExternalKeying": true
}
```

`videoFormat` and `frameRate` select the DeckLink capture/playout display mode. Common examples include `720p`/`50`, `720p`/`59.94`, `1080i`/`50`, `1080i`/`59.94`, `1080p`/`25`, `1080p`/`50`, `1080p`/`59.94`, and `2160p`/`59.94`, depending on card support.

The control UI is available at:

```text
http://127.0.0.1:<serverPort>
```

## Control API

The local REST control API is documented as an OpenAPI/Swagger spec:

```text
docs/openapi.yaml
```

When the control server is running, the same spec is also served at:

```text
http://127.0.0.1:<serverPort>/docs/openapi.yaml
http://127.0.0.1:<serverPort>/openapi.yaml
```

A Swagger UI page is available at:

```text
http://127.0.0.1:<serverPort>/docs
```

Use those docs to inspect the `/api/state`, layer control, stack preset, and reload endpoints. Live state updates are also sent over the `/ws` WebSocket.

## OSC Control

The native host also listens for local OSC parameter control on the configured `oscPort`:

```text
/VideoShaderToys/{LayerNameOrID}/{ParameterNameOrID}
```

For example, `/VideoShaderToys/VHS/intensity` updates the `intensity` parameter on the first matching `VHS` layer. The listener accepts float, integer, string, and boolean OSC values, and validates them through the same shader parameter path as the REST API. See `docs/OSC_CONTROL.md` for details.

## Shader Packages

Each shader package lives under:

```text
shaders/<id>/
  shader.json
  shader.slang
```

See `SHADER_CONTRACT.md` for the manifest schema, parameter types, texture assets, temporal history support, and the Slang entry point contract.

## Generated Files

Runtime-generated files are intentionally ignored:

- `runtime/shader_cache/active_shader_wrapper.slang`
- `runtime/shader_cache/active_shader.raw.frag`
- `runtime/shader_cache/active_shader.frag`
- `runtime/runtime_state.json`
- `runtime/stack_presets/*.json`

Only `runtime/templates/` and `runtime/README.md` are tracked.

## CI

The Gitea workflow expects two act runners:

- `windows-latest`: builds the native app and runs native tests.
- `ubuntu-latest`: installs UI dependencies and runs the Vite build.

If your Windows runner stores the Blackmagic SDK outside the repo, configure `GPUDIRECT_DIR` in the runner environment or adjust the workflow configure command to pass `-DGPUDIRECT_DIR=...`.