video-shader-toys/README.md

Video Shader

Native video shader host with an OpenGL/DeckLink render path, Slang shader packages, and a local React control UI.

The app loads shader packages from shaders/, compiles Slang to GLSL at runtime, renders a configurable layer stack, and exposes a browser-based control surface over a local HTTP/WebSocket server.

Repository Layout

• apps/LoopThroughWithOpenGLCompositing/: native C++ host app.
• shaders/: shader packages, each with shader.json and shader.slang.
• ui/: Vite/React control UI.
• config/runtime-host.json: runtime configuration.
• runtime/templates/: tracked shader wrapper templates.
• runtime/: ignored generated runtime cache/state output. See runtime/README.md.
• tests/: focused native tests for pure runtime logic.
• .gitea/workflows/ci.yml: Gitea Actions CI for Windows native tests and Ubuntu UI build.

Requirements

• Windows with Visual Studio 2022 C++ tooling.
• CMake 3.24 or newer.
• Node.js and npm for the control UI.
• Blackmagic DeckLink SDK 16.0 with the NVIDIA GPUDirect sample files available locally.
• Slang binary release with slangc.exe, slang-compiler.dll, slang-glslang.dll, and LICENSE.

The Blackmagic/GPUDirect SDK should not be committed to this repository. CMakeLists.txt exposes GPUDIRECT_DIR as a cache path so local machines and CI runners can point at their installed SDK location.

Default expected SDK path:

3rdParty/Blackmagic DeckLink SDK 16.0/Win/Samples/NVIDIA_GPUDirect

Override example:

cmake --preset vs2022-x64-debug -DGPUDIRECT_DIR="D:/SDKs/Blackmagic DeckLink SDK 16.0/Win/Samples/NVIDIA_GPUDirect"

Default expected Slang path:

3rdParty/slang-2026.8-windows-x86_64

Override example:

cmake --preset vs2022-x64-debug -DSLANG_ROOT="D:/SDKs/slang-2026.8-windows-x86_64"

Build

Configure and build the native app:

cmake --preset vs2022-x64-debug
cmake --build --preset build-debug

Build the React control UI:

cd ui
npm ci
npm run build

The native app serves ui/dist when it exists, otherwise it falls back to the source UI directory during development.

Package

Build the UI, build the native Release target, then install into a portable runtime folder:

cd ui
npm ci
npm run build
cd ..
cmake --preset vs2022-x64-release
cmake --build --preset build-release
cmake --install build/vs2022-x64-release --config Release --prefix dist/VideoShader

The package folder will contain:

dist/VideoShader/
  LoopThroughWithOpenGLCompositing.exe
  dvp.dll
  config/
  shaders/
  3rdParty/slang/bin/
  ui/dist/
  runtime/templates/
  third_party_notices/

You can run LoopThroughWithOpenGLCompositing.exe directly from that folder. In packaged mode, the app resolves config/, shaders/, 3rdParty/slang/bin/slangc.exe, ui/dist/, and runtime/templates/ relative to the exe folder. In development mode, it still falls back to repo-root discovery.

The install step copies only the Slang runtime files required by the shader compiler (slangc.exe, slang-compiler.dll, and slang-glslang.dll) plus third_party_notices/SLANG_LICENSE.txt. It does not copy the full Slang release folder.

Create a zip for distribution:

Compress-Archive -Path dist/VideoShader/* -DestinationPath dist/VideoShader.zip -Force

Tests

Run native tests:

cmake --build --preset build-debug --target RUN_TESTS

Run the UI production build check:

cd ui
npm run build

Current native test coverage includes:

• JSON parsing and serialization.
• Parameter normalization and preset filename safety.
• Shader manifest parsing, temporal manifest validation, and package registry scanning.

Runtime Configuration

config/runtime-host.json controls host behavior:

{
  "shaderLibrary": "shaders",
  "serverPort": 8080,
  "oscPort": 9000,
  "inputVideoFormat": "1080p",
  "inputFrameRate": "59.94",
  "outputVideoFormat": "1080p",
  "outputFrameRate": "59.94",
  "autoReload": true,
  "maxTemporalHistoryFrames": 12,
  "enableExternalKeying": true
}

inputVideoFormat/inputFrameRate select the DeckLink capture mode. outputVideoFormat/outputFrameRate select the playout mode. The shader stack runs at input resolution and the final rendered frame is scaled once into the configured output mode. Common examples include 720p/50, 720p/59.94, 1080i/50, 1080i/59.94, 1080p/25, 1080p/50, 1080p/59.94, and 2160p/59.94, depending on card support.

Legacy videoFormat and frameRate keys are still accepted and apply to both input and output unless the explicit input/output keys are present.

The control UI is available at:

http://127.0.0.1:<serverPort>

Runtime State And Presets

The current layer stack is autosaved to runtime/runtime_state.json whenever layers, shader assignments, bypass state, ordering, or parameter values change. On startup, the host reloads that file before compiling the stack, so the last working stack should come back automatically.

Manual stack presets are still available from the control UI and are saved under runtime/stack_presets/*.json. Presets are useful for named looks, while runtime_state.json is the latest working state for the local machine.

Control API

The local REST control API is documented as an OpenAPI/Swagger spec:

docs/openapi.yaml

When the control server is running, the same spec is also served at:

http://127.0.0.1:<serverPort>/docs/openapi.yaml
http://127.0.0.1:<serverPort>/openapi.yaml

A Swagger UI page is available at:

http://127.0.0.1:<serverPort>/docs

Use those docs to inspect the /api/state, layer control, stack preset, and reload endpoints. Live state updates are also sent over the /ws WebSocket.

OSC Control

The native host also listens for local OSC parameter control on the configured oscPort:

/VideoShaderToys/{LayerNameOrID}/{ParameterNameOrID}

For example, /VideoShaderToys/VHS/intensity updates the intensity parameter on the first matching VHS layer. The listener accepts float, integer, string, and boolean OSC values, and validates them through the same shader parameter path as the REST API. See docs/OSC_CONTROL.md for details.

Shader Packages

Each shader package lives under:

shaders/<id>/
  shader.json
  shader.slang
  optional-font-or-texture-assets

See SHADER_CONTRACT.md for the manifest schema, parameter types, texture assets, font/text assets, temporal history support, and the Slang entry point contract. shaders/text-overlay/ is the reference live text package and bundles Roboto Regular with its OFL license.

Generated Files

Runtime-generated files are intentionally ignored:

• runtime/shader_cache/active_shader_wrapper.slang
• runtime/shader_cache/active_shader.raw.frag
• runtime/shader_cache/active_shader.frag
• runtime/runtime_state.json autosaved latest stack and parameter state.
• runtime/stack_presets/*.json

Only runtime/templates/ and runtime/README.md are tracked.

CI

The Gitea workflow expects two act runners:

• windows-2022: builds the native app and runs native tests.
• ubuntu-latest: installs UI dependencies and runs the Vite build.

The Windows jobs validate native third-party dependencies before configuring CMake. Because 3rdParty/ is ignored, configure these paths on the runner or in Gitea repository variables:

• GPUDIRECT_DIR: path to Blackmagic DeckLink SDK 16.0/Win/Samples/NVIDIA_GPUDirect.
• SLANG_ROOT: path to the Slang binary release folder containing bin/slangc.exe.

The Windows runner also needs the Visual Studio ATL component installed. In Visual Studio Build Tools 2022, add C++ ATL for latest v143 build tools (x86 & x64), component ID Microsoft.VisualStudio.Component.VC.ATL.

Example runner paths:

D:\SDKs\Blackmagic DeckLink SDK 16.0\Win\Samples\NVIDIA_GPUDirect
D:\SDKs\slang-2026.8-windows-x86_64

If neither variable is set, the workflow falls back to the repo-local defaults under 3rdParty/.

Still Todo

• Audio.
• Improve text rendering.
• Genlock.
• Find a better UI library.
• Logs.
• Continue source cleanup/refactoring. Pass 1 done
• Display the control URL in the Windows app, ideally clickable, without rendering it on the video output.
• Support a separate sound shader .slang file in shader packages.