214 lines
6.4 KiB
Markdown
214 lines
6.4 KiB
Markdown
# Video Shader
|
|
|
|
Native video shader host with an OpenGL/DeckLink render path, Slang shader packages, and a local React control UI.
|
|
|
|
The app loads shader packages from `shaders/`, compiles Slang to GLSL at runtime, renders a configurable layer stack, and exposes a browser-based control surface over a local HTTP/WebSocket server.
|
|
|
|
## Repository Layout
|
|
|
|
- `apps/LoopThroughWithOpenGLCompositing/`: native C++ host app.
|
|
- `shaders/`: shader packages, each with `shader.json` and `shader.slang`.
|
|
- `ui/`: Vite/React control UI.
|
|
- `config/runtime-host.json`: runtime configuration.
|
|
- `runtime/templates/`: tracked shader wrapper templates.
|
|
- `runtime/`: ignored generated runtime cache/state output. See `runtime/README.md`.
|
|
- `tests/`: focused native tests for pure runtime logic.
|
|
- `.gitea/workflows/ci.yml`: Gitea Actions CI for Windows native tests and Ubuntu UI build.
|
|
|
|
## Requirements
|
|
|
|
- Windows with Visual Studio 2022 C++ tooling.
|
|
- CMake 3.24 or newer.
|
|
- Node.js and npm for the control UI.
|
|
- Blackmagic DeckLink SDK 16.0 with the NVIDIA GPUDirect sample files available locally.
|
|
- Slang compiler available under the repo/tooling paths expected by the runtime, or otherwise discoverable by the existing app setup.
|
|
|
|
The Blackmagic/GPUDirect SDK should not be committed to this repository. `CMakeLists.txt` exposes `GPUDIRECT_DIR` as a cache path so local machines and CI runners can point at their installed SDK location.
|
|
|
|
Default expected SDK path:
|
|
|
|
```text
|
|
3rdParty/Blackmagic DeckLink SDK 16.0/Win/Samples/NVIDIA_GPUDirect
|
|
```
|
|
|
|
Override example:
|
|
|
|
```powershell
|
|
cmake --preset vs2022-x64-debug -DGPUDIRECT_DIR="D:/SDKs/Blackmagic DeckLink SDK 16.0/Win/Samples/NVIDIA_GPUDirect"
|
|
```
|
|
|
|
## Build
|
|
|
|
Configure and build the native app:
|
|
|
|
```powershell
|
|
cmake --preset vs2022-x64-debug
|
|
cmake --build --preset build-debug
|
|
```
|
|
|
|
Build the React control UI:
|
|
|
|
```powershell
|
|
cd ui
|
|
npm ci
|
|
npm run build
|
|
```
|
|
|
|
The native app serves `ui/dist` when it exists, otherwise it falls back to the source UI directory during development.
|
|
|
|
## Package
|
|
|
|
Build the UI, build the native Release target, then install into a portable runtime folder:
|
|
|
|
```powershell
|
|
cd ui
|
|
npm ci
|
|
npm run build
|
|
cd ..
|
|
cmake --preset vs2022-x64-release
|
|
cmake --build --preset build-release
|
|
cmake --install build/vs2022-x64-release --config Release --prefix dist/VideoShader
|
|
```
|
|
|
|
The package folder will contain:
|
|
|
|
```text
|
|
dist/VideoShader/
|
|
LoopThroughWithOpenGLCompositing.exe
|
|
dvp.dll
|
|
config/
|
|
shaders/
|
|
ui/dist/
|
|
runtime/templates/
|
|
```
|
|
|
|
You can run `LoopThroughWithOpenGLCompositing.exe` directly from that folder. In packaged mode, the app resolves `config/`, `shaders/`, `ui/dist/`, and `runtime/templates/` relative to the exe folder. In development mode, it still falls back to repo-root discovery.
|
|
|
|
Create a zip for distribution:
|
|
|
|
```powershell
|
|
Compress-Archive -Path dist/VideoShader/* -DestinationPath dist/VideoShader.zip -Force
|
|
```
|
|
|
|
## Tests
|
|
|
|
Run native tests:
|
|
|
|
```powershell
|
|
cmake --build --preset build-debug --target RUN_TESTS
|
|
```
|
|
|
|
Run the UI production build check:
|
|
|
|
```powershell
|
|
cd ui
|
|
npm run build
|
|
```
|
|
|
|
Current native test coverage includes:
|
|
|
|
- JSON parsing and serialization.
|
|
- Parameter normalization and preset filename safety.
|
|
- Shader manifest parsing and package registry scanning.
|
|
|
|
## Runtime Configuration
|
|
|
|
`config/runtime-host.json` controls host behavior:
|
|
|
|
```json
|
|
{
|
|
"shaderLibrary": "shaders",
|
|
"serverPort": 8080,
|
|
"oscPort": 9000,
|
|
"inputVideoFormat": "1080p",
|
|
"inputFrameRate": "59.94",
|
|
"outputVideoFormat": "1080p",
|
|
"outputFrameRate": "59.94",
|
|
"autoReload": true,
|
|
"maxTemporalHistoryFrames": 12,
|
|
"audioEnabled": true,
|
|
"audioChannelCount": 2,
|
|
"audioSampleRate": 48000,
|
|
"audioDelayMode": "matchVideoPreroll",
|
|
"enableExternalKeying": true
|
|
}
|
|
```
|
|
|
|
`inputVideoFormat`/`inputFrameRate` select the DeckLink capture mode. `outputVideoFormat`/`outputFrameRate` select the playout mode. The shader stack runs at input resolution and the final rendered frame is scaled once into the configured output mode. Common examples include `720p`/`50`, `720p`/`59.94`, `1080i`/`50`, `1080i`/`59.94`, `1080p`/`25`, `1080p`/`50`, `1080p`/`59.94`, and `2160p`/`59.94`, depending on card support.
|
|
|
|
`audioEnabled` enables embedded stereo 48 kHz PCM pass-through. Audio is delayed to match the scheduled video preroll and the synchronized level/spectrum data is exposed to shaders.
|
|
|
|
Legacy `videoFormat` and `frameRate` keys are still accepted and apply to both input and output unless the explicit input/output keys are present.
|
|
|
|
The control UI is available at:
|
|
|
|
```text
|
|
http://127.0.0.1:<serverPort>
|
|
```
|
|
|
|
## Control API
|
|
|
|
The local REST control API is documented as an OpenAPI/Swagger spec:
|
|
|
|
```text
|
|
docs/openapi.yaml
|
|
```
|
|
|
|
When the control server is running, the same spec is also served at:
|
|
|
|
```text
|
|
http://127.0.0.1:<serverPort>/docs/openapi.yaml
|
|
http://127.0.0.1:<serverPort>/openapi.yaml
|
|
```
|
|
|
|
A Swagger UI page is available at:
|
|
|
|
```text
|
|
http://127.0.0.1:<serverPort>/docs
|
|
```
|
|
|
|
Use those docs to inspect the `/api/state`, layer control, stack preset, and reload endpoints. Live state updates are also sent over the `/ws` WebSocket.
|
|
|
|
## OSC Control
|
|
|
|
The native host also listens for local OSC parameter control on the configured `oscPort`:
|
|
|
|
```text
|
|
/VideoShaderToys/{LayerNameOrID}/{ParameterNameOrID}
|
|
```
|
|
|
|
For example, `/VideoShaderToys/VHS/intensity` updates the `intensity` parameter on the first matching `VHS` layer. The listener accepts float, integer, string, and boolean OSC values, and validates them through the same shader parameter path as the REST API. See `docs/OSC_CONTROL.md` for details.
|
|
|
|
## Shader Packages
|
|
|
|
Each shader package lives under:
|
|
|
|
```text
|
|
shaders/<id>/
|
|
shader.json
|
|
shader.slang
|
|
```
|
|
|
|
See `SHADER_CONTRACT.md` for the manifest schema, parameter types, texture assets, temporal history support, and the Slang entry point contract.
|
|
|
|
## Generated Files
|
|
|
|
Runtime-generated files are intentionally ignored:
|
|
|
|
- `runtime/shader_cache/active_shader_wrapper.slang`
|
|
- `runtime/shader_cache/active_shader.raw.frag`
|
|
- `runtime/shader_cache/active_shader.frag`
|
|
- `runtime/runtime_state.json`
|
|
- `runtime/stack_presets/*.json`
|
|
|
|
Only `runtime/templates/` and `runtime/README.md` are tracked.
|
|
|
|
## CI
|
|
|
|
The Gitea workflow expects two act runners:
|
|
|
|
- `windows-latest`: builds the native app and runs native tests.
|
|
- `ubuntu-latest`: installs UI dependencies and runs the Vite build.
|
|
|
|
If your Windows runner stores the Blackmagic SDK outside the repo, configure `GPUDIRECT_DIR` in the runner environment or adjust the workflow configure command to pass `-DGPUDIRECT_DIR=...`.
|