256 lines
8.3 KiB
Markdown
256 lines
8.3 KiB
Markdown
# Video Shader
|
|
|
|
Native video shader host with an OpenGL/DeckLink render path, Slang shader packages, and a local React control UI.
|
|
|
|
The app loads shader packages from `shaders/`, compiles Slang to GLSL at runtime, renders a configurable layer stack, and exposes a browser-based control surface over a local HTTP/WebSocket server.
|
|
|
|
## Repository Layout
|
|
|
|
- `apps/LoopThroughWithOpenGLCompositing/`: native C++ host app.
|
|
- `shaders/`: shader packages, each with `shader.json` and `shader.slang`.
|
|
- `ui/`: Vite/React control UI.
|
|
- `config/runtime-host.json`: runtime configuration.
|
|
- `runtime/templates/`: tracked shader wrapper templates.
|
|
- `runtime/`: ignored generated runtime cache/state output. See `runtime/README.md`.
|
|
- `tests/`: focused native tests for pure runtime logic.
|
|
- `.gitea/workflows/ci.yml`: Gitea Actions CI for Windows native tests and Ubuntu UI build.
|
|
|
|
## Requirements
|
|
|
|
- Windows with Visual Studio 2022 C++ tooling.
|
|
- CMake 3.24 or newer.
|
|
- Node.js and npm for the control UI.
|
|
- Blackmagic DeckLink SDK 16.0 with the NVIDIA GPUDirect sample files available locally.
|
|
- Slang binary release with `slangc.exe`, `slang-compiler.dll`, `slang-glslang.dll`, and `LICENSE`.
|
|
|
|
The Blackmagic/GPUDirect SDK should not be committed to this repository. `CMakeLists.txt` exposes `GPUDIRECT_DIR` as a cache path so local machines and CI runners can point at their installed SDK location.
|
|
|
|
Default expected SDK path:
|
|
|
|
```text
|
|
3rdParty/Blackmagic DeckLink SDK 16.0/Win/Samples/NVIDIA_GPUDirect
|
|
```
|
|
|
|
Override example:
|
|
|
|
```powershell
|
|
cmake --preset vs2022-x64-debug -DGPUDIRECT_DIR="D:/SDKs/Blackmagic DeckLink SDK 16.0/Win/Samples/NVIDIA_GPUDirect"
|
|
```
|
|
|
|
Default expected Slang path:
|
|
|
|
```text
|
|
3rdParty/slang-2026.8-windows-x86_64
|
|
```
|
|
|
|
Override example:
|
|
|
|
```powershell
|
|
cmake --preset vs2022-x64-debug -DSLANG_ROOT="D:/SDKs/slang-2026.8-windows-x86_64"
|
|
```
|
|
|
|
## Build
|
|
|
|
Configure and build the native app:
|
|
|
|
```powershell
|
|
cmake --preset vs2022-x64-debug
|
|
cmake --build --preset build-debug
|
|
```
|
|
|
|
Build the React control UI:
|
|
|
|
```powershell
|
|
cd ui
|
|
npm ci
|
|
npm run build
|
|
```
|
|
|
|
The native app serves `ui/dist` when it exists, otherwise it falls back to the source UI directory during development.
|
|
|
|
## Package
|
|
|
|
Build the UI, build the native Release target, then install into a portable runtime folder:
|
|
|
|
```powershell
|
|
cd ui
|
|
npm ci
|
|
npm run build
|
|
cd ..
|
|
cmake --preset vs2022-x64-release
|
|
cmake --build --preset build-release
|
|
cmake --install build/vs2022-x64-release --config Release --prefix dist/VideoShader
|
|
```
|
|
|
|
The package folder will contain:
|
|
|
|
```text
|
|
dist/VideoShader/
|
|
LoopThroughWithOpenGLCompositing.exe
|
|
dvp.dll
|
|
config/
|
|
shaders/
|
|
3rdParty/slang/bin/
|
|
ui/dist/
|
|
runtime/templates/
|
|
third_party_notices/
|
|
```
|
|
|
|
You can run `LoopThroughWithOpenGLCompositing.exe` directly from that folder. In packaged mode, the app resolves `config/`, `shaders/`, `3rdParty/slang/bin/slangc.exe`, `ui/dist/`, and `runtime/templates/` relative to the exe folder. In development mode, it still falls back to repo-root discovery.
|
|
|
|
The install step copies only the Slang runtime files required by the shader compiler (`slangc.exe`, `slang-compiler.dll`, and `slang-glslang.dll`) plus `third_party_notices/SLANG_LICENSE.txt`. It does not copy the full Slang release folder.
|
|
|
|
Create a zip for distribution:
|
|
|
|
```powershell
|
|
Compress-Archive -Path dist/VideoShader/* -DestinationPath dist/VideoShader.zip -Force
|
|
```
|
|
|
|
## Tests
|
|
|
|
Run native tests:
|
|
|
|
```powershell
|
|
cmake --build --preset build-debug --target RUN_TESTS
|
|
```
|
|
|
|
Run the UI production build check:
|
|
|
|
```powershell
|
|
cd ui
|
|
npm run build
|
|
```
|
|
|
|
Current native test coverage includes:
|
|
|
|
- JSON parsing and serialization.
|
|
- Parameter normalization and preset filename safety.
|
|
- Shader manifest parsing, temporal manifest validation, and package registry scanning.
|
|
|
|
## Runtime Configuration
|
|
|
|
`config/runtime-host.json` controls host behavior:
|
|
|
|
```json
|
|
{
|
|
"shaderLibrary": "shaders",
|
|
"serverPort": 8080,
|
|
"oscPort": 9000,
|
|
"inputVideoFormat": "1080p",
|
|
"inputFrameRate": "59.94",
|
|
"outputVideoFormat": "1080p",
|
|
"outputFrameRate": "59.94",
|
|
"autoReload": true,
|
|
"maxTemporalHistoryFrames": 12,
|
|
"enableExternalKeying": true
|
|
}
|
|
```
|
|
|
|
`inputVideoFormat`/`inputFrameRate` select the DeckLink capture mode. `outputVideoFormat`/`outputFrameRate` select the playout mode. The shader stack runs at input resolution and the final rendered frame is scaled once into the configured output mode. Common examples include `720p`/`50`, `720p`/`59.94`, `1080i`/`50`, `1080i`/`59.94`, `1080p`/`25`, `1080p`/`50`, `1080p`/`59.94`, and `2160p`/`59.94`, depending on card support.
|
|
|
|
Legacy `videoFormat` and `frameRate` keys are still accepted and apply to both input and output unless the explicit input/output keys are present.
|
|
|
|
The control UI is available at:
|
|
|
|
```text
|
|
http://127.0.0.1:<serverPort>
|
|
```
|
|
|
|
## Runtime State And Presets
|
|
|
|
The current layer stack is autosaved to `runtime/runtime_state.json` whenever layers, shader assignments, bypass state, ordering, or parameter values change. On startup, the host reloads that file before compiling the stack, so the last working stack should come back automatically.
|
|
|
|
Manual stack presets are still available from the control UI and are saved under `runtime/stack_presets/*.json`. Presets are useful for named looks, while `runtime_state.json` is the latest working state for the local machine.
|
|
|
|
## Control API
|
|
|
|
The local REST control API is documented as an OpenAPI/Swagger spec:
|
|
|
|
```text
|
|
docs/openapi.yaml
|
|
```
|
|
|
|
When the control server is running, the same spec is also served at:
|
|
|
|
```text
|
|
http://127.0.0.1:<serverPort>/docs/openapi.yaml
|
|
http://127.0.0.1:<serverPort>/openapi.yaml
|
|
```
|
|
|
|
A Swagger UI page is available at:
|
|
|
|
```text
|
|
http://127.0.0.1:<serverPort>/docs
|
|
```
|
|
|
|
Use those docs to inspect the `/api/state`, layer control, stack preset, and reload endpoints. Live state updates are also sent over the `/ws` WebSocket.
|
|
|
|
## OSC Control
|
|
|
|
The native host also listens for local OSC parameter control on the configured `oscPort`:
|
|
|
|
```text
|
|
/VideoShaderToys/{LayerNameOrID}/{ParameterNameOrID}
|
|
```
|
|
|
|
For example, `/VideoShaderToys/VHS/intensity` updates the `intensity` parameter on the first matching `VHS` layer. The listener accepts float, integer, string, and boolean OSC values, and validates them through the same shader parameter path as the REST API. See `docs/OSC_CONTROL.md` for details.
|
|
|
|
## Shader Packages
|
|
|
|
Each shader package lives under:
|
|
|
|
```text
|
|
shaders/<id>/
|
|
shader.json
|
|
shader.slang
|
|
optional-font-or-texture-assets
|
|
```
|
|
|
|
See `SHADER_CONTRACT.md` for the manifest schema, parameter types, texture assets, font/text assets, temporal history support, and the Slang entry point contract. `shaders/text-overlay/` is the reference live text package and bundles Roboto Regular with its OFL license.
|
|
|
|
## Generated Files
|
|
|
|
Runtime-generated files are intentionally ignored:
|
|
|
|
- `runtime/shader_cache/active_shader_wrapper.slang`
|
|
- `runtime/shader_cache/active_shader.raw.frag`
|
|
- `runtime/shader_cache/active_shader.frag`
|
|
- `runtime/runtime_state.json` autosaved latest stack and parameter state.
|
|
- `runtime/stack_presets/*.json`
|
|
|
|
Only `runtime/templates/` and `runtime/README.md` are tracked.
|
|
|
|
## CI
|
|
|
|
The Gitea workflow expects two act runners:
|
|
|
|
- `windows-2022`: builds the native app and runs native tests.
|
|
- `ubuntu-latest`: installs UI dependencies and runs the Vite build.
|
|
|
|
The Windows jobs validate native third-party dependencies before configuring CMake. Because `3rdParty/` is ignored, configure these paths on the runner or in Gitea repository variables:
|
|
|
|
- `GPUDIRECT_DIR`: path to `Blackmagic DeckLink SDK 16.0/Win/Samples/NVIDIA_GPUDirect`.
|
|
- `SLANG_ROOT`: path to the Slang binary release folder containing `bin/slangc.exe`.
|
|
|
|
The Windows runner also needs the Visual Studio ATL component installed. In Visual Studio Build Tools 2022, add `C++ ATL for latest v143 build tools (x86 & x64)`, component ID `Microsoft.VisualStudio.Component.VC.ATL`.
|
|
|
|
Example runner paths:
|
|
|
|
```text
|
|
D:\SDKs\Blackmagic DeckLink SDK 16.0\Win\Samples\NVIDIA_GPUDirect
|
|
D:\SDKs\slang-2026.8-windows-x86_64
|
|
```
|
|
|
|
If neither variable is set, the workflow falls back to the repo-local defaults under `3rdParty/`.
|
|
|
|
|
|
## Still Todo
|
|
|
|
- Audio.
|
|
- Genlock.
|
|
- Find a better UI library for react.
|
|
- Logs.
|
|
- Continue source cleanup/refactoring. Pass 2 done
|
|
- Support a separate sound shader `.slang` file in shader packages. (https://www.shadertoy.com/view/XsBXWt)
|
|
- Add WebView2
|
|
- move to MSDF, typography rasterisation |