Video Shader

Native video shader host with an OpenGL render path, pluggable video I/O boundary, DeckLink backend, Slang shader packages, and a local React control UI.

The app loads shader packages from shaders/, compiles Slang to GLSL at runtime, renders a configurable layer stack, and exposes a browser-based control surface over a local HTTP/WebSocket server. Shader compilation is prepared off the frame path where possible, then committed on the render thread so editing shader files does not block video output for the whole compile.

Repository Layout

• apps/LoopThroughWithOpenGLCompositing/: native C++ host app.
• shaders/: shader packages, each with shader.json and shader.slang.
• ui/: Vite/React control UI.
• config/runtime-host.json: runtime configuration.
• runtime/templates/: tracked shader wrapper templates.
• runtime/: ignored generated runtime cache/state output. See runtime/README.md.
• tests/: focused native tests for pure runtime logic.
• .gitea/workflows/ci.yml: Gitea Actions CI for Windows native tests and Ubuntu UI build.

Native app internals are grouped by boundary:

• videoio/: backend-neutral video I/O contracts, formats, and playout timing.
• videoio/decklink/: DeckLink-specific device adapter, callbacks, and SDK bindings.
• gl/renderer/: low-level OpenGL resources and extension helpers.
• gl/pipeline/: frame pipeline, render passes, video I/O bridge, preview/readback, and screenshots.
• gl/shader/: shader compilation, texture/text assets, UBO packing, and shader program ownership.

Requirements

• Windows with Visual Studio 2022 C++ tooling.
• CMake 3.24 or newer.
• Node.js and npm for the control UI.
• Blackmagic Desktop Video drivers and a DeckLink device for the current production video I/O backend.
• Slang binary release with slangc.exe, slang-compiler.dll, slang-glslang.dll, and LICENSE.

Default expected Slang path:

3rdParty/slang-2026.8-windows-x86_64

Override example:

cmake --preset vs2022-x64-debug -DSLANG_ROOT="D:/SDKs/slang-2026.8-windows-x86_64"

Build

Configure and build the native app:

cmake --preset vs2022-x64-debug
cmake --build --preset build-debug --parallel

Build the React control UI:

cd ui
npm ci
npm run build

The native app serves ui/dist when it exists, otherwise it falls back to the source UI directory during development.

The control UI provides:

• A searchable shader library for adding layers.
• Compact parameter rows with inline descriptions and OSC copy controls.
• Stack save/recall presets.
• Manual shader reload.
• Screenshot capture from the final output render target.

Package

Build the UI, build the native Release target, then install into a portable runtime folder:

cd ui
npm ci
npm run build
cd ..
cmake --preset vs2022-x64-release
cmake --build --preset build-release --parallel
cmake --install build/vs2022-x64-release --config Release --prefix dist/VideoShader

The package folder will contain:

dist/VideoShader/
  LoopThroughWithOpenGLCompositing.exe
  config/
  shaders/
  3rdParty/slang/bin/
  ui/dist/
  docs/
  SHADER_CONTRACT.md
  runtime/templates/
  third_party_notices/

You can run LoopThroughWithOpenGLCompositing.exe directly from that folder. In packaged mode, the app resolves config/, shaders/, 3rdParty/slang/bin/slangc.exe, ui/dist/, and runtime/templates/ relative to the exe folder. In development mode, it still falls back to repo-root discovery.

The install step copies only the Slang runtime files required by the shader compiler (slangc.exe, slang-compiler.dll, and slang-glslang.dll) plus third_party_notices/SLANG_LICENSE.txt. It does not copy the full Slang release folder.

Create a zip for distribution:

Compress-Archive -Path dist/VideoShader/* -DestinationPath dist/VideoShader.zip -Force

Tests

Run native tests:

cmake --build --preset build-debug --target RUN_TESTS --parallel

Run the UI production build check:

cd ui
npm run build

Current native test coverage includes:

• JSON parsing and serialization.
• Parameter normalization and preset filename safety.
• Shader manifest parsing, temporal manifest validation, and package registry scanning.
• Video I/O format helpers, v210/Ay10 row-byte math, v210 pack/unpack math, playout scheduler timing, and fake backend contract coverage.
• OSC packet parsing.
• Slang validation for every available shader package.

Runtime Configuration

config/runtime-host.json controls host behavior:

{
  "shaderLibrary": "shaders",
  "serverPort": 8080,
  "oscBindAddress": "127.0.0.1",
  "oscPort": 9000,
  "oscSmoothing": 0.18,
  "inputVideoFormat": "1080p",
  "inputFrameRate": "59.94",
  "outputVideoFormat": "1080p",
  "outputFrameRate": "59.94",
  "autoReload": true,
  "maxTemporalHistoryFrames": 12,
  "enableExternalKeying": true
}

inputVideoFormat/inputFrameRate select the video capture mode. outputVideoFormat/outputFrameRate select the playout mode. With the current DeckLink backend, supported modes depend on the installed card and driver. The shader stack runs at input resolution and the final rendered frame is scaled once into the configured output mode. Common examples include 720p/50, 720p/59.94, 1080i/50, 1080i/59.94, 1080p/25, 1080p/50, 1080p/59.94, and 2160p/59.94.

Legacy videoFormat and frameRate keys are still accepted and apply to both input and output unless the explicit input/output keys are present.

The control UI is available at:

http://127.0.0.1:<serverPort>

Runtime State And Presets

The current layer stack is autosaved to runtime/runtime_state.json whenever layers, shader assignments, bypass state, ordering, or parameter values change. On startup, the host reloads that file before compiling the stack, so the last working stack should come back automatically.

Manual stack presets are still available from the control UI and are saved under runtime/stack_presets/*.json. Presets are useful for named looks, while runtime_state.json is the latest working state for the local machine.

Control API

The local REST control API is documented as an OpenAPI/Swagger spec:

docs/openapi.yaml

When the control server is running, the same spec is also served at:

http://127.0.0.1:<serverPort>/docs/openapi.yaml
http://127.0.0.1:<serverPort>/openapi.yaml

A Swagger UI page is available at:

http://127.0.0.1:<serverPort>/docs

Use those docs to inspect the /api/state, layer control, stack preset, and reload endpoints. Live state updates are also sent over the /ws WebSocket.

The control UI has a Reload shaders button. It rescans shaders/, re-reads manifests, queues shader compilation, refreshes shader availability/errors, and keeps the previous working shader stack running if a changed shader fails to compile.

Each parameter row also includes a small OSC button. Clicking it copies that parameter's OSC route to the clipboard.

The control UI also has a Screenshot button. It queues a capture of the final output render target and writes a PNG under:

runtime/screenshots/

OSC Control

The native host also listens for OSC parameter control on the configured oscBindAddress and oscPort:

/VideoShaderToys/{LayerNameOrID}/{ParameterNameOrID}

For example, /VideoShaderToys/VHS/intensity updates the intensity parameter on the first matching VHS layer. The listener accepts float, integer, string, and boolean OSC values, and validates them through the same shader parameter path as the REST API. OSC updates are coalesced and applied once per render tick, UI state broadcasts are throttled, and OSC-driven parameter changes are not autosaved to runtime/runtime_state.json. oscSmoothing adds a small per-frame easing amount for numeric OSC controls such as floats, vec2, and color, while booleans, enums, text, and triggers stay immediate. The default bind address is 127.0.0.1; set oscBindAddress to 0.0.0.0 to accept OSC on all IPv4 interfaces. See docs/OSC_CONTROL.md for details.

Shader Packages

Each shader package lives under:

shaders/<id>/
  shader.json
  shader.slang
  optional-extra-pass.slang
  optional-font-or-texture-assets

See SHADER_CONTRACT.md for the manifest schema, parameter types, texture assets, font/text assets, temporal history support, optional render-pass declarations, and the Slang entry point contract. shaders/text-overlay/ is the reference live text package and bundles Roboto Regular with its OFL license. Broken shader packages are shown as unavailable in the selector with their error text instead of preventing the app from launching.

Generated Files

Runtime-generated files are intentionally ignored:

• runtime/shader_cache/active_shader_wrapper.slang
• runtime/shader_cache/active_shader.raw.frag
• runtime/shader_cache/active_shader.frag
• runtime/runtime_state.json autosaved latest stack and parameter state.
• runtime/stack_presets/*.json
• runtime/screenshots/*.png screenshots captured from the final output render target.

Only runtime/templates/ and runtime/README.md are tracked.

CI

The Gitea workflow expects two act runners:

• windows-2022: builds the native app and runs native tests.
• ubuntu-latest: installs UI dependencies and runs the Vite build.

The Windows jobs validate native third-party dependencies before configuring CMake. Because 3rdParty/ is ignored, configure this path on the runner or in a Gitea repository variable:

• SLANG_ROOT: path to the Slang binary release folder containing bin/slangc.exe.

The Windows runner also needs the Visual Studio ATL component installed. In Visual Studio Build Tools 2022, add C++ ATL for latest v143 build tools (x86 & x64), component ID Microsoft.VisualStudio.Component.VC.ATL.

Example runner paths:

D:\SDKs\slang-2026.8-windows-x86_64

If SLANG_ROOT is not set, the workflow falls back to the repo-local default under 3rdParty/.

Still Todo

• Audio.
• Genlock.
• Logs.
• Add more video I/O backends now that the DeckLink path is behind videoio/.
• Support a separate sound shader .slang file in shader packages. (https://www.shadertoy.com/view/XsBXWt)
• Add WebView2 for an embedded native control surface.
• MSDF typography rasterisation
• More shader-library organisation and filtering as the built-in library grows.
• Optional linear-light compositing mode.
• compute shaders or a small 1x1 or nx1 RGBA16f render target for arbitrary data storage
• allow shaders to read other shaders data store based on name? or output over OSC
• Mipmapping for shader-declared textures
• Anotate included shaders
• allow 3 vector exposed controls
• add nearest sampling to the extra shader pass