aiden/video-shader-toys
1
0
Fork 0
Code Issues Pull Requests Actions Releases 5 Wiki Activity

Update contracts

Browse Source
This commit is contained in:
Aiden
2026-05-03 11:48:49 +10:00
parent fa6789bf86
commit ea9f608f55
6 changed files with 483 additions and 64 deletions
Download Patch File Download Diff File Expand all files Collapse all files

42
.gitea/workflows/ci.yml
View File

@@ -32,7 +32,7 @@ jobs:
ui-ubuntu:
name: React UI Build
runs-on: ubuntu-latest
runs-on: nubuntu-latest
steps:
- name: Checkout
@@ -45,3 +45,43 @@ jobs:
- name: Build UI
working-directory: ui
run: npm run build
package-windows:
name: Windows Release Package
runs-on: windows-latest
needs:
- native-windows
- ui-ubuntu
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Build UI
shell: powershell
working-directory: ui
run: |
npm ci --no-audit --no-fund
npm run build
- name: Configure Release
shell: powershell
run: cmake --preset vs2022-x64-release
- name: Build Release
shell: powershell
run: cmake --build --preset build-release
- name: Install Runtime Package
shell: powershell
run: cmake --install build/vs2022-x64-release --config Release --prefix dist/VideoShader
- name: Zip Runtime Package
shell: powershell
run: Compress-Archive -Path dist/VideoShader/* -DestinationPath dist/VideoShader.zip -Force
- name: Upload Runtime Package
uses: actions/upload-artifact@v4
with:
name: VideoShader-windows-release
path: dist/VideoShader.zip

1
.gitignore vendored
View File

@@ -2,6 +2,7 @@
# Build output
/build/
/dist/
/out/
/.vs/
/apps/*/x64/

29
CMakeLists.txt
View File

@@ -132,4 +132,33 @@ add_custom_command(TARGET LoopThroughWithOpenGLCompositing POST_BUILD
"$<TARGET_FILE_DIR:LoopThroughWithOpenGLCompositing>/dvp.dll"
)
install(TARGETS LoopThroughWithOpenGLCompositing
RUNTIME DESTINATION "."
)
install(FILES "${GPUDIRECT_DIR}/bin/x64/dvp.dll"
DESTINATION "."
)
install(DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/config/"
DESTINATION "config"
)
install(DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/shaders/"
DESTINATION "shaders"
)
install(DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/runtime/templates/"
DESTINATION "runtime/templates"
)
install(FILES "${CMAKE_CURRENT_SOURCE_DIR}/runtime/README.md"
DESTINATION "runtime"
)
install(DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/ui/dist/"
DESTINATION "ui/dist"
OPTIONAL
)
source_group(TREE "${APP_DIR}" FILES ${APP_SOURCES})

34
README.md
View File

@@ -56,6 +56,40 @@ npm run build
The native app serves `ui/dist` when it exists, otherwise it falls back to the source UI directory during development.
## Package
Build the UI, build the native Release target, then install into a portable runtime folder:
```powershell
cd ui
npm ci
npm run build
cd ..
cmake --preset vs2022-x64-release
cmake --build --preset build-release
cmake --install build/vs2022-x64-release --config Release --prefix dist/VideoShader
```
The package folder will contain:
```text
dist/VideoShader/
LoopThroughWithOpenGLCompositing.exe
dvp.dll
config/
shaders/
ui/dist/
runtime/templates/
```
You can run `LoopThroughWithOpenGLCompositing.exe` directly from that folder. In packaged mode, the app resolves `config/`, `shaders/`, `ui/dist/`, and `runtime/templates/` relative to the exe folder. In development mode, it still falls back to repo-root discovery.
Create a zip for distribution:
```powershell
Compress-Archive -Path dist/VideoShader/* -DestinationPath dist/VideoShader.zip -Force
```
## Tests
Run native tests:

423
SHADER_CONTRACT.md
View File

@@ -1,34 +1,294 @@
# Shader Package Contract
Each shader package lives under `shaders/<id>/` and includes:
This document explains how to create shaders for the Video Shader runtime.
- `shader.json`
- `shader.slang`
Each shader is a small package under `shaders/<id>/`:
## Manifest fields
```text
shaders/my-effect/
shader.json
shader.slang
optional-texture.png
```
`shader.json` defines:
The runtime reads `shader.json`, generates a Slang wrapper from `runtime/templates/shader_wrapper.slang.in`, includes your `shader.slang`, compiles the result to GLSL, and exposes the shader in the local control UI.
## Quick Start
Create a folder:
```text
shaders/my-effect/
```
Add `shader.json`:
```json
{
"id": "my-effect",
"name": "My Effect",
"description": "A simple starter shader.",
"category": "Custom",
"entryPoint": "shadeVideo",
"parameters": [
{
"id": "strength",
"label": "Strength",
"type": "float",
"default": 0.5,
"min": 0.0,
"max": 1.0,
"step": 0.01
}
]
}
```
Add `shader.slang`:
```slang
float4 shadeVideo(ShaderContext context)
{
float4 color = context.sourceColor;
color.rgb = lerp(color.rgb, 1.0 - color.rgb, strength);
return saturate(color);
}
```
With `autoReload` enabled in `config/runtime-host.json`, edits to shader source, manifests, and declared texture assets are picked up automatically.
## Manifest Fields
`shader.json` is the runtime-facing description of the shader.
Required fields:
- `id`: package ID used by state/presets. Hyphenated names are OK here, for example `my-effect`.
- `name`: display name in the UI.
- `parameters`: array of exposed controls. Use `[]` if there are no user parameters.
Optional fields:
- `description`: display/help text for the shader library.
- `category`: UI grouping label.
- `entryPoint`: Slang function to call. Defaults to `shadeVideo`.
- `textures`: texture assets to load and expose as samplers.
- `temporal`: history-buffer requirements.
Shader-visible identifiers must be valid Slang-style identifiers:
- `id`
- `name`
- `description`
- `category`
- `entryPoint`
- `parameters`
- optional `textures`
- optional `temporal`
- parameter `id`
- texture `id`
Supported parameter types:
Use letters, numbers, and underscores only, and start with a letter or underscore. For example, `logoTexture` is valid; `logo-texture` is not valid as a shader-visible texture ID.
- `float`
- `vec2`
- `color`
- `bool`
- `enum`
## Slang Entry Point
## Texture assets
Your shader file must implement the manifest `entryPoint`.
Shaders can optionally declare texture assets:
Default:
```slang
float4 shadeVideo(ShaderContext context)
{
return context.sourceColor;
}
```
The runtime owns the real fragment shader entry point. Your function is called from the wrapper, and the runtime handles final bypass/mix behavior:
```slang
return lerp(context.sourceColor, effectedColor, mixValue);
```
That means:
- Return the fully effected color from your function.
- Respect alpha if your shader produces an overlay or sprite.
- The runtime will blend your result with the source according to `mixAmount` and bypass state.
## ShaderContext
Your entry point receives:
```slang
struct ShaderContext
{
float2 uv;
float4 sourceColor;
float2 inputResolution;
float2 outputResolution;
float time;
float frameCount;
float mixAmount;
float bypass;
int sourceHistoryLength;
int temporalHistoryLength;
};
```
Fields:
- `uv`: normalized texture coordinates, usually `0..1`.
- `sourceColor`: decoded RGBA source video at `uv`.
- `inputResolution`: decoded input video resolution in pixels.
- `outputResolution`: output/render resolution in pixels.
- `time`: elapsed runtime time in seconds.
- `frameCount`: incrementing frame counter.
- `mixAmount`: runtime mix amount.
- `bypass`: `1.0` when the layer is bypassed, otherwise `0.0`.
- `sourceHistoryLength`: number of usable source-history frames currently available.
- `temporalHistoryLength`: number of usable temporal frames currently available for this layer.
## Helper Functions
The wrapper provides:
```slang
float4 sampleVideo(float2 uv);
float4 sampleSourceHistory(int framesAgo, float2 uv);
float4 sampleTemporalHistory(int framesAgo, float2 uv);
```
`sampleVideo` samples the live decoded source video.
`sampleSourceHistory` samples previous decoded source frames. `framesAgo` is clamped into the available range. If no history is available, it falls back to `sampleVideo`.
`sampleTemporalHistory` samples previous pre-layer input frames for temporal shaders that request `preLayerInput` history. `framesAgo` is clamped into the available range. If no temporal history is available, it falls back to `sampleVideo`.
Example:
```slang
float4 shadeVideo(ShaderContext context)
{
float4 previous = sampleSourceHistory(1, context.uv);
return lerp(context.sourceColor, previous, 0.35);
}
```
## Parameters
Manifest parameters are exposed to Slang as global values with the same `id`.
Supported types:
| Manifest type | Slang type | JSON value |
| --- | --- | --- |
| `float` | `float` | number |
| `vec2` | `float2` | `[x, y]` |
| `color` | `float4` | `[r, g, b, a]` |
| `bool` | `bool` | `true` or `false` |
| `enum` | `int` | selected option index |
Float example:
```json
{
"id": "brightness",
"label": "Brightness",
"type": "float",
"default": 1.0,
"min": 0.0,
"max": 2.0,
"step": 0.01
}
```
```slang
color.rgb *= brightness;
```
Vector example:
```json
{
"id": "offset",
"label": "Offset",
"type": "vec2",
"default": [0.0, 0.0],
"min": [-0.2, -0.2],
"max": [0.2, 0.2],
"step": [0.001, 0.001]
}
```
```slang
float2 uv = clamp(context.uv + offset, float2(0.0), float2(1.0));
```
Color example:
```json
{
"id": "tint",
"label": "Tint",
"type": "color",
"default": [1.0, 1.0, 1.0, 1.0]
}
```
```slang
color *= tint;
```
Boolean example:
```json
{
"id": "invert",
"label": "Invert",
"type": "bool",
"default": false
}
```
```slang
if (invert)
color.rgb = 1.0 - color.rgb;
```
Enum example:
```json
{
"id": "mode",
"label": "Mode",
"type": "enum",
"default": "normal",
"options": [
{ "value": "normal", "label": "Normal" },
{ "value": "luma", "label": "Luma" },
{ "value": "posterize", "label": "Posterize" }
]
}
```
Enums are stored in presets/state by their string `value`, but exposed to Slang as a zero-based integer index in option order:
```slang
if (mode == 1)
{
float luma = dot(color.rgb, float3(0.2126, 0.7152, 0.0722));
color.rgb = float3(luma);
}
else if (mode == 2)
{
color.rgb = floor(color.rgb * 4.0) / 4.0;
}
```
Parameter validation:
- Float values are clamped to `min`/`max` if provided.
- `vec2` must have exactly 2 numbers.
- `color` must have exactly 4 numbers.
- Enum defaults must match one of the declared option values.
- Non-finite numeric values are rejected.
## Texture Assets
Declare texture assets in the manifest:
```json
{
@@ -41,74 +301,119 @@ Shaders can optionally declare texture assets:
}
```
- `id` becomes a shader-visible sampler name
- `path` is resolved relative to the shader package directory
- texture asset changes trigger shader reload just like shader and manifest edits
Rules:
## Temporal manifests
- `id` must be a valid shader identifier.
- `path` is relative to the shader package directory.
- The file must exist when the manifest is loaded.
- Texture asset changes trigger shader reload.
Shaders can optionally declare temporal history needs:
Texture IDs become `Sampler2D<float4>` globals:
```slang
float4 logo = logoTexture.Sample(logoUv);
```
For sprite or overlay shaders, return premultiplied-looking output if you want clean composition:
```slang
float alpha = logo.a;
return float4(logo.rgb * alpha, alpha);
```
See `shaders/dvd-bounce/` for a complete texture-driven example.
## Temporal Shaders
Temporal shaders can request access to previous frames.
Manifest example:
```json
{
"temporal": {
"enabled": true,
"historySource": "source",
"historyLength": 4
"historySource": "preLayerInput",
"historyLength": 12
}
}
```
Supported temporal history sources:
Supported `historySource` values:
- `source` - decoded source-video history from previous frames
- `preLayerInput` - history of the input arriving at that layer before the shader runs
- `source`: decoded source-video history from previous frames.
- `preLayerInput`: history of the input arriving at this layer before the shader runs.
`historyLength` is requested by the shader and clamped by `config/runtime-host.json` via `maxTemporalHistoryFrames`.
`historyLength` is the requested frame count. The runtime clamps it by `maxTemporalHistoryFrames` in `config/runtime-host.json`.
Temporal history resets automatically when:
Temporal history resets when:
- layers are added, removed, or reordered
- a layer bypass state changes
- a layer changes to a different shader
- a layer changes shader
- a shader is reloaded or recompiled
- render dimensions change
## Slang contract
The runtime owns the fragment entry point, the UYVY-to-RGBA decode pass, and final mix/bypass behavior.
Your `shader.slang` file implements:
Use the available history lengths to avoid assuming history is ready on the first frame:
```slang
float4 shadeVideo(ShaderContext context)
{
return context.sourceColor;
if (context.temporalHistoryLength <= 0)
return context.sourceColor;
float4 oldFrame = sampleTemporalHistory(3, context.uv);
return lerp(context.sourceColor, oldFrame, 0.4);
}
```
Available built-ins through `ShaderContext`:
See `shaders/temporal-ghost-trail/` and `shaders/temporal-low-fps/` for examples.
- `uv`
- `sourceColor` - the already-decoded full-resolution RGBA video color at `uv`
- `inputResolution`
- `outputResolution`
- `time`
- `frameCount`
- `mixAmount`
- `bypass`
- `sourceHistoryLength`
- `temporalHistoryLength`
## Coordinate And Color Notes
Manifest parameters are exposed to the shader as globals named by their `id`.
- `uv` is normalized.
- Use `context.outputResolution` for pixel-sized effects.
- Use `context.inputResolution` when sampling source video by input pixel size.
- `sourceColor` and `sampleVideo` return RGBA values in normalized `0..1` range.
- Prefer `saturate(color)` or explicit `clamp` before returning if your math can overshoot.
Helper function:
- `sampleVideo(float2 uv)` returns decoded full-resolution RGBA video from the live DeckLink input.
- `sampleSourceHistory(int framesAgo, float2 uv)` samples the most recent available source history frame, clamping to the oldest available frame if needed.
- `sampleTemporalHistory(int framesAgo, float2 uv)` samples the most recent available pre-layer history frame for temporal shaders, clamping to the oldest available frame if needed.
Declared texture assets are exposed as `Sampler2D<float4>` globals using the texture `id`, for example:
Pixel-size example:
```slang
float4 logo = logoTexture.Sample(uv);
float2 pixel = 1.0 / max(context.outputResolution, float2(1.0));
float4 right = sampleVideo(context.uv + float2(pixel.x, 0.0));
```
## Reload And Generated Files
When a shader compiles, the runtime writes generated files under `runtime/shader_cache/`:
- `active_shader_wrapper.slang`
- `active_shader.raw.frag`
- `active_shader.frag`
These files are ignored by git and are useful for debugging compiler output. If a shader fails to compile, inspect the wrapper first; it shows the exact generated Slang code including your included shader.
## Common Pitfalls
- Do not use hyphens in parameter IDs, texture IDs, or entry point names.
- Do not declare your own `ShaderContext`, `GlobalParams`, `sampleVideo`, `sampleSourceHistory`, or `sampleTemporalHistory`.
- Do not write a `[shader("fragment")]` entry point in `shader.slang`; the runtime provides it.
- Remember enum globals are integer indexes, not strings.
- Declare every texture in `shader.json`; undeclared texture samplers will not be bound.
- Keep temporal history requests modest. They consume texture units and memory and are capped by runtime config.
- If a parameter appears in the UI but not in Slang, the shader may still compile, but the control has no effect.
- If a Slang name collides with a generated global, rename your parameter or local symbol.
## Minimal Package Checklist
Before committing a new shader package:
- `shader.json` is valid JSON.
- `id` is unique across `shaders/`.
- `entryPoint`, parameter IDs, and texture IDs are valid identifiers.
- `shader.slang` implements the configured entry point.
- Texture files referenced by `textures` exist.
- Enum defaults are present in their `options`.
- Temporal shaders handle short or empty history gracefully.
- The app can reload and compile the shader without errors.

18
apps/LoopThroughWithOpenGLCompositing/RuntimeHost.cpp
View File

@@ -48,6 +48,19 @@ std::vector<double> JsonArrayToNumbers(const JsonValue& value)
return numbers;
}
bool LooksLikePackagedRuntimeRoot(const std::filesystem::path& candidate)
{
return std::filesystem::exists(candidate / "config" / "runtime-host.json") &&
std::filesystem::exists(candidate / "runtime" / "templates" / "shader_wrapper.slang.in") &&
std::filesystem::exists(candidate / "shaders");
}
bool LooksLikeRepoRoot(const std::filesystem::path& candidate)
{
return std::filesystem::exists(candidate / "CMakeLists.txt") &&
std::filesystem::exists(candidate / "apps" / "LoopThroughWithOpenGLCompositing");
}
std::filesystem::path FindRepoRootCandidate()
{
std::vector<std::filesystem::path> rootsToTry;
@@ -66,11 +79,8 @@ std::filesystem::path FindRepoRootCandidate()
std::filesystem::path candidate = startPath;
for (int depth = 0; depth < 10 && !candidate.empty(); ++depth)
{
if (std::filesystem::exists(candidate / "CMakeLists.txt") &&
std::filesystem::exists(candidate / "apps" / "LoopThroughWithOpenGLCompositing"))
{
if (LooksLikePackagedRuntimeRoot(candidate) || LooksLikeRepoRoot(candidate))
return candidate;
}
candidate = candidate.parent_path();
}