diff --git a/Home.md b/Home.md index 5d08b7b..77f6672 100644 --- a/Home.md +++ b/Home.md @@ -1 +1,48 @@ -Welcome to the Wiki. \ No newline at end of file +# Video Shader Toys Wiki + +Video Shader Toys is a live video shader host for Windows. It takes DeckLink video input, runs it through a stack of Slang shader layers, and exposes a local browser control surface, REST API, and OSC control path for show control. + +This wiki is written for people using the system during setup, look building, rehearsal, and live operation. + +## Start Here + +- [Operator Interface](Operator-Interface): using the browser control UI, layer stack, shader picker, presets, and status panels. +- [Shader Library](Shaders): index of the included shaders. +- [Making Shaders](Making-Shaders): how to build new shader packages using the shader contract. +- [Runtime Host Configuration](Runtime-Host-Configuration): what `config/runtime-host.json` controls. +- [REST API](REST-API): local HTTP endpoints and WebSocket state updates. +- [OSC Control](OSC-Control): controlling shader parameters from OSC tools. + +## Included Shaders + +| Shader | Main Use | +| --- | --- | +| [Black and White](Shader-Black-and-White) | Fast monochrome conversion. | +| [Composition Guides](Shader-Composition-Guides) | Rule-of-thirds and center framing overlays. | +| [Data Mosh](Shader-Data-Mosh) | Temporal block smear and chroma glitch. | +| [DVD Bounce](Shader-DVD-Bounce) | Transparent bouncing DVD logo overlay. | +| [False Color](Shader-False-Color) | Exposure and luma debugging. | +| [Fisheye Reproject](Shader-Fisheye-Reproject) | Fisheye-to-virtual-camera reprojection. | +| [Gaussian Blur](Shader-Gaussian-Blur) | Soft blur with radius, strength, and sample control. | +| [Greenscreen Key](Shader-Greenscreen-Key) | Green screen alpha key with despill. | +| [Pixelate](Shader-Pixelate) | Low-resolution block effect and optional grid. | +| [Safe Area Guides](Shader-Safe-Area-Guides) | Broadcast safe-area guides and aspect mattes. | +| [Studio Color](Shader-Studio-Color) | Basic color controls and contract example. | +| [Temporal Echo](Shader-Temporal-Echo) | Multi-frame echo trails with decay and tint. | +| [Temporal Ghost Trail](Shader-Temporal-Ghost-Trail) | Soft trailing blend from recent frames. | +| [Temporal Low FPS](Shader-Temporal-Low-FPS) | Deliberate choppy held-frame playback. | +| [VHS](Shader-VHS) | Analog tape distortion, smear, bloom, noise, and fade. | +| [Video Cube](Shader-Video-Cube) | Live video mapped onto a rotating cube. | +| [Video Transform](Shader-Video-Transform) | Zoom, pan, rotate, and edge handling. | +| [Waveform Overlay](Shader-Waveform-Overlay) | Lightweight luma waveform overlay. | + +## Typical Workflow + +1. Start the native host. +2. Open `http://127.0.0.1:8080`, or use the port shown in the runtime config. +3. Add one or more shader layers. +4. Drag layers to change processing order. +5. Expand a layer to adjust parameters. +6. Save useful looks as stack presets. +7. Use OSC or REST only when you need external control or automation. + diff --git a/Making-Shaders.md b/Making-Shaders.md new file mode 100644 index 0000000..cecf06d --- /dev/null +++ b/Making-Shaders.md @@ -0,0 +1,199 @@ +# Making Shaders + +Shader packages are small folders loaded from the shader library. A basic package looks like: + +```text +shaders/my-effect/ + shader.json + shader.slang +``` + +The runtime reads the manifest, generates a wrapper, includes your Slang code, compiles it to GLSL, and exposes the shader in the operator UI. + +## Minimal Shader + +`shader.json`: + +```json +{ + "id": "my-effect", + "name": "My Effect", + "description": "A simple starter shader.", + "category": "Custom", + "entryPoint": "shadeVideo", + "parameters": [ + { + "id": "strength", + "label": "Strength", + "type": "float", + "default": 0.5, + "min": 0.0, + "max": 1.0, + "step": 0.01 + } + ] +} +``` + +`shader.slang`: + +```slang +float4 shadeVideo(ShaderContext context) +{ + float4 color = context.sourceColor; + color.rgb = lerp(color.rgb, 1.0 - color.rgb, strength); + return saturate(color); +} +``` + +With `autoReload` enabled, edits should reload automatically. + +## Manifest Fields + +| Field | Required | Purpose | +| --- | --- | --- | +| `id` | Yes | Stable package ID used by state, presets, API, and OSC. Hyphens are OK here. | +| `name` | Yes | Human-readable name in the UI. | +| `parameters` | Yes | Exposed controls. Use `[]` when there are none. | +| `description` | No | Short help text in shader listings. | +| `category` | No | UI grouping label. | +| `entryPoint` | No | Function to call. Defaults to `shadeVideo`. | +| `textures` | No | Texture assets to bind as samplers. | +| `temporal` | No | Previous-frame history request. | + +Shader-visible identifiers must be valid Slang identifiers: letters, numbers, and underscores only, starting with a letter or underscore. This applies to `entryPoint`, parameter IDs, and texture IDs. + +## Entry Point + +Your shader implements the function named by `entryPoint`. + +```slang +float4 shadeVideo(ShaderContext context) +{ + return context.sourceColor; +} +``` + +Do not write your own fragment entry point. The runtime owns that. Your function returns the fully effected color, and the wrapper handles runtime mix and bypass behavior. + +## Shader Context + +Your function receives: + +```slang +struct ShaderContext +{ + float2 uv; + float4 sourceColor; + float2 inputResolution; + float2 outputResolution; + float time; + float frameCount; + float mixAmount; + float bypass; + int sourceHistoryLength; + int temporalHistoryLength; +}; +``` + +Use `uv` for normalized coordinates, `sourceColor` for the current layer input, `time` for animation, and `outputResolution` or `inputResolution` for pixel-sized effects. + +## Sampling Helpers + +```slang +float4 sampleVideo(float2 uv); +float4 sampleSourceHistory(int framesAgo, float2 uv); +float4 sampleTemporalHistory(int framesAgo, float2 uv); +``` + +`sampleVideo` samples the current decoded source. The history functions safely clamp requested frame numbers and fall back when no history is ready. + +## Parameters + +Manifest parameters become global shader variables with matching IDs. + +| Manifest Type | Slang Type | JSON/API Value | +| --- | --- | --- | +| `float` | `float` | number | +| `vec2` | `float2` | `[x, y]` | +| `color` | `float4` | `[r, g, b, a]` | +| `bool` | `bool` | `true` or `false` | +| `enum` | `int` | string in JSON, zero-based index in Slang | + +Float values are clamped to `min` and `max` when present. `vec2` needs exactly two numbers. `color` needs exactly four numbers. Enum defaults must match an option value. + +## Textures + +Declare every texture in `shader.json`: + +```json +{ + "textures": [ + { + "id": "logoTexture", + "path": "logo.png" + } + ] +} +``` + +The path is relative to the shader package folder. The texture ID becomes a sampler: + +```slang +float4 logo = logoTexture.Sample(logoUv); +``` + +For overlays or sprites, return useful alpha. `DVD Bounce` is a good texture-driven example. + +## Temporal Shaders + +Temporal shaders request previous frames: + +```json +{ + "temporal": { + "enabled": true, + "historySource": "preLayerInput", + "historyLength": 12 + } +} +``` + +Supported history sources: + +| Source | Meaning | +| --- | --- | +| `source` | Previous decoded source frames. | +| `preLayerInput` | Previous frames arriving at this layer before the shader runs. | + +The runtime clamps requested history by `maxTemporalHistoryFrames`. Temporal history resets when layers are added, removed, reordered, bypassed, changed, reloaded, or when render dimensions change. + +## Debugging + +Generated shader files are written under: + +```text +runtime/shader_cache/ +``` + +Useful files: + +| File | Use | +| --- | --- | +| `active_shader_wrapper.slang` | See the exact generated Slang wrapper. | +| `active_shader.raw.frag` | Raw GLSL from `slangc`. | +| `active_shader.frag` | Patched GLSL used by OpenGL. | + +If compilation fails, start with `active_shader_wrapper.slang` and the UI compiler panel. + +## Checklist + +- `shader.json` is valid JSON. +- `id` is unique. +- `entryPoint`, parameter IDs, and texture IDs are valid shader identifiers. +- `shader.slang` implements the configured entry point. +- Texture files exist. +- Enum defaults are valid option values. +- Temporal shaders handle empty or short history. +- The shader compiles after reload. + diff --git a/OSC-Control.md b/OSC-Control.md new file mode 100644 index 0000000..488c473 --- /dev/null +++ b/OSC-Control.md @@ -0,0 +1,118 @@ +# OSC Control + +OSC lets external tools control shader parameters without using the browser UI. The listener is local-only and binds to: + +```text +127.0.0.1: +``` + +By default: + +```text +127.0.0.1:9000 +``` + +Set `oscPort` to `0` in `config/runtime-host.json` to disable OSC. + +## Address Format + +Send messages to: + +```text +/VideoShaderToys/{LayerNameOrID}/{ParameterNameOrID} +``` + +Examples: + +```text +/VideoShaderToys/layer-1/brightness +/VideoShaderToys/VHS/intensity +/VideoShaderToys/vhs/wiggle +/VideoShaderToys/fisheye-reproject/panDegrees +/VideoShaderToys/video-transform/pan +``` + +## Layer Matching + +The layer part can be: + +| Key Type | Example | +| --- | --- | +| Internal layer ID | `layer-1` | +| Shader package ID | `vhs` | +| Shader display name | `VHS` | + +Exact matches are tried first. If that fails, matching ignores spaces, underscores, hyphens, and casing. + +If multiple layers use the same shader ID or name, OSC controls the first matching layer in the stack. Use the internal layer ID when you need to control a specific duplicate. + +## Parameter Matching + +The parameter part can be: + +| Key Type | Example | +| --- | --- | +| Parameter ID | `panDegrees` | +| Parameter label | `Pan` | + +The UI shows a copyable OSC address beside each parameter, which is the easiest way to avoid typos. + +## Values + +Supported OSC argument types: + +| OSC Type | Meaning | +| --- | --- | +| `f` | 32-bit float | +| `d` | 64-bit float | +| `i` | integer | +| `s` | string | +| `T` / `F` | boolean true / false | + +Single-argument messages become scalar values. Multi-argument messages become JSON arrays, which is how OSC drives `vec2` and `color` parameters. + +Examples: + +```text +/VideoShaderToys/fisheye-reproject/panDegrees 45.0 +/VideoShaderToys/fisheye-reproject/fisheyeModel "equisolid" +/VideoShaderToys/video-transform/pan 0.25 -0.5 +/VideoShaderToys/composition-guides/lineColor 1.0 0.8 0.1 1.0 +``` + +Values are validated through the same parameter path as the REST API. Invalid addresses or values are ignored and reported to native debug output. + +## Open Stage Control Examples + +Simple scalar widget: + +```json +{ + "address": "/VideoShaderToys/fisheye-reproject/panDegrees", + "target": "127.0.0.1:9000", + "decimals": "2f" +} +``` + +XY pad controlling one `vec2` parameter: + +```js +var x = Array.isArray(value) ? Number(value[0]) : 0; +var y = Array.isArray(value) ? Number(value[1]) : 0; +send( + "127.0.0.1:9000", + "/VideoShaderToys/video-transform/pan", + {type: "f", value: x}, + {type: "f", value: y} +); +``` + +XY pad controlling two scalar parameters: + +```js +var pan = Array.isArray(value) ? Number(value[0]) : 0; +var tilt = Array.isArray(value) ? Number(value[1]) : 0; +send("127.0.0.1:9000", "/VideoShaderToys/fisheye-reproject/panDegrees", {type: "f", value: pan}); +send("127.0.0.1:9000", "/VideoShaderToys/fisheye-reproject/tiltDegrees", {type: "f", value: tilt}); +``` + diff --git a/Operator-Interface.md b/Operator-Interface.md new file mode 100644 index 0000000..0dbcca7 --- /dev/null +++ b/Operator-Interface.md @@ -0,0 +1,95 @@ +# Operator Interface + +The operator interface is the local browser control surface served by the native host. By default it is available at: + +```text +http://127.0.0.1:8080 +``` + +If `serverPort` is changed in `config/runtime-host.json`, use that port instead. If the preferred port is busy, the host may bind to a nearby port. + +## What You See + +The top summary shows: + +| Field | Meaning | +| --- | --- | +| Shaders | Number of shader packages loaded from the shader library. | +| Layers | Number of active processing layers. | +| Signal | Whether the host currently sees video input. | +| Render | Current render time in milliseconds. | + +The status area shows runtime state, video mode, output mode, render budget use, and compiler output. The compiler panel is the first place to look if a shader does not load. + +## Layer Stack + +Layers process from top to bottom. Each layer receives the output of the layer above it, applies its shader, then passes the result to the next layer. + +Common actions: + +| Action | How | +| --- | --- | +| Add a layer | Choose a shader in the add layer picker and click `Add`. | +| Reorder layers | Drag a layer by its drag handle. | +| Open controls | Click `Controls` or the layer title. | +| Bypass a layer | Toggle `Bypass`. | +| Remove a layer | Click the remove button. | +| Change a layer's shader | Expand the layer and choose a different shader. | +| Reset parameters | Expand the layer and click `Reset`. | + +Changing order, bypass state, or shader assignment may reset temporal history. That is expected for temporal effects. + +## Parameters + +Shader parameters are generated from each shader's `shader.json` manifest. + +| Parameter Type | UI Control | +| --- | --- | +| `float` | Slider plus numeric input. | +| `vec2` | Two numeric inputs. | +| `color` | Color picker plus alpha value. | +| `bool` | Toggle. | +| `enum` | Dropdown. | + +Each parameter shows a copyable OSC route such as: + +```text +/VideoShaderToys/vhs/wiggle +``` + +Use that route in an OSC controller when you want live external control. + +## Stack Presets + +Stack presets save the current layer chain, including layer order, shader assignments, bypass state, and parameter values. + +Use `Save stack` to create a named preset. Use `Recall stack` to load one. Presets are stored under: + +```text +runtime/stack_presets/ +``` + +The runtime also autosaves the latest working state to: + +```text +runtime/runtime_state.json +``` + +Autosave is for restoring the local machine after restart. Named presets are for looks you intentionally want to recall. + +## Reload Shader + +The `Reload shader` button asks the runtime to reload and recompile shaders. Use it after changing a shader package when auto reload is off, or when you want to force a clean refresh. + +## Reading Errors + +If the UI says `Compile Error`, open the compiler panel. Common causes are: + +- Invalid `shader.json`. +- A parameter, texture, or entry point name that is not a valid shader identifier. +- A missing texture file. +- Slang code that does not compile. +- A shader that references a parameter not declared in the manifest. + +With auto reload enabled, fixing the file should cause the host to try again automatically. + diff --git a/REST-API.md b/REST-API.md new file mode 100644 index 0000000..914ed98 --- /dev/null +++ b/REST-API.md @@ -0,0 +1,221 @@ +# REST API + +The native host exposes a local REST API on the same server as the browser UI. + +Default base URL: + +```text +http://127.0.0.1:8080 +``` + +The API is intended for local automation, custom control panels, and the bundled React UI. Mutating requests return: + +```json +{ "ok": true } +``` + +Failures return: + +```json +{ "ok": false, "error": "Reason here." } +``` + +## Interactive Docs + +When the host is running: + +```text +http://127.0.0.1:/docs +``` + +The OpenAPI spec is served at: + +```text +http://127.0.0.1:/docs/openapi.yaml +http://127.0.0.1:/openapi.yaml +``` + +The repository copy is: + +```text +docs/openapi.yaml +``` + +## State + +### Get Runtime State + +```http +GET /api/state +``` + +Returns the full runtime state: app config, runtime status, video status, DeckLink status, performance, available shaders, stack presets, and active layers. + +Example: + +```powershell +curl http://127.0.0.1:8080/api/state +``` + +## WebSocket State Stream + +Connect to: + +```text +ws://127.0.0.1:/ws +``` + +The server sends full runtime state JSON when a client connects and whenever state changes. Use this for live control surfaces that need to stay in sync without polling. + +## Layer Endpoints + +### Add Layer + +```http +POST /api/layers/add +``` + +Body: + +```json +{ "shaderId": "studio-color" } +``` + +### Remove Layer + +```http +POST /api/layers/remove +``` + +Body: + +```json +{ "layerId": "layer-1" } +``` + +### Move Layer By Direction + +```http +POST /api/layers/move +``` + +Body: + +```json +{ "layerId": "layer-1", "direction": -1 } +``` + +Negative moves up the stack. Positive moves down. + +### Reorder Layer To Index + +```http +POST /api/layers/reorder +``` + +Body: + +```json +{ "layerId": "layer-1", "targetIndex": 0 } +``` + +### Set Bypass + +```http +POST /api/layers/set-bypass +``` + +Body: + +```json +{ "layerId": "layer-1", "bypass": true } +``` + +### Change Shader + +```http +POST /api/layers/set-shader +``` + +Body: + +```json +{ "layerId": "layer-1", "shaderId": "vhs" } +``` + +### Update Parameter + +```http +POST /api/layers/update-parameter +``` + +Body: + +```json +{ "layerId": "layer-1", "parameterId": "brightness", "value": 1.25 } +``` + +The `value` type depends on the shader parameter: + +| Parameter Type | JSON Value | +| --- | --- | +| `float` | number | +| `vec2` | `[x, y]` | +| `color` | `[r, g, b, a]` | +| `bool` | boolean | +| `enum` | string option value | + +### Reset Parameters + +```http +POST /api/layers/reset-parameters +``` + +Body: + +```json +{ "layerId": "layer-1" } +``` + +## Stack Presets + +### Save Preset + +```http +POST /api/stack-presets/save +``` + +Body: + +```json +{ "presetName": "live-show-look" } +``` + +### Load Preset + +```http +POST /api/stack-presets/load +``` + +Body: + +```json +{ "presetName": "live-show-look" } +``` + +## Runtime Actions + +### Reload Shaders + +```http +POST /api/reload +``` + +Body: + +```json +{} +``` + +Successful mutating requests broadcast the updated state over `/ws`. + diff --git a/Runtime-Host-Configuration.md b/Runtime-Host-Configuration.md new file mode 100644 index 0000000..bb50893 --- /dev/null +++ b/Runtime-Host-Configuration.md @@ -0,0 +1,80 @@ +# Runtime Host Configuration + +Runtime settings live in: + +```text +config/runtime-host.json +``` + +The default configuration is: + +```json +{ + "shaderLibrary": "shaders", + "serverPort": 8080, + "oscPort": 9000, + "inputVideoFormat": "1080p", + "inputFrameRate": "59.94", + "outputVideoFormat": "1080p", + "outputFrameRate": "59.94", + "autoReload": true, + "maxTemporalHistoryFrames": 12, + "enableExternalKeying": true +} +``` + +## Settings + +| Setting | What It Does | +| --- | --- | +| `shaderLibrary` | Folder containing shader packages. Usually `shaders`. | +| `serverPort` | Preferred local HTTP control server port. The browser UI, REST API, Swagger docs, and WebSocket use this server. | +| `oscPort` | Local UDP port for OSC parameter control. Use `0` to disable OSC. | +| `inputVideoFormat` | DeckLink capture format, such as `1080p` or `720p`. | +| `inputFrameRate` | DeckLink capture frame rate, such as `59.94`, `50`, or `25`. | +| `outputVideoFormat` | DeckLink output/playout format. | +| `outputFrameRate` | DeckLink output/playout frame rate. | +| `autoReload` | When true, shader manifests, shader source, and declared texture changes are reloaded automatically. | +| `maxTemporalHistoryFrames` | Upper limit for temporal history buffers. Individual shaders can request fewer frames. | +| `enableExternalKeying` | Requests external keying when the DeckLink device and keyer path support it. | + +## Video Modes + +The shader stack renders at the input resolution. The final rendered frame is then scaled once into the configured output mode. + +Common examples include: + +```text +720p / 50 +720p / 59.94 +1080i / 50 +1080i / 59.94 +1080p / 25 +1080p / 50 +1080p / 59.94 +2160p / 59.94 +``` + +Actual availability depends on the DeckLink card and signal path. + +## Legacy Keys + +Older configs may use: + +```json +{ + "videoFormat": "1080p", + "frameRate": "59.94" +} +``` + +Those keys are still accepted. They apply to both input and output unless the newer explicit input/output keys are present. + +## Practical Advice + +- Keep `autoReload` on while developing shaders. +- Keep `maxTemporalHistoryFrames` modest during live work; temporal history consumes texture memory. +- Set `oscPort` to `0` if you do not need OSC and want fewer moving parts. +- Use matching input and output modes when possible for the simplest signal path. +- If the UI cannot be reached on the expected port, check the runtime status for the actual bound port. + diff --git a/Shader-Black-and-White.md b/Shader-Black-and-White.md new file mode 100644 index 0000000..2b553c5 --- /dev/null +++ b/Shader-Black-and-White.md @@ -0,0 +1,25 @@ +# Black and White + +| Field | Value | +| --- | --- | +| Shader ID | `black-and-white` | +| Category | Built-in | +| Package | `shaders/black-and-white/` | + +Black and White converts the incoming image to grayscale using Rec. 709-style luma weights. It preserves the source alpha. + +## Good For + +- Fast monochrome looks. +- Checking contrast without color distraction. +- Building a simple base layer before color or utility overlays. + +## Controls + +This shader has no user parameters. + +## Tips + +- Use layer bypass to compare color and monochrome quickly. +- Put utility overlays such as safe guides after this layer if you want guide colors to stay visible. + diff --git a/Shader-Composition-Guides.md b/Shader-Composition-Guides.md new file mode 100644 index 0000000..4617559 --- /dev/null +++ b/Shader-Composition-Guides.md @@ -0,0 +1,28 @@ +# Composition Guides + +| Field | Value | +| --- | --- | +| Shader ID | `composition-guides` | +| Category | Utility | +| Package | `shaders/composition-guides/` | + +Composition Guides overlays rule-of-thirds lines and an optional center crosshair over the current image. It is intended for camera lineup, framing, and quick operator checks. + +## Controls + +| Control | Type | Default | What It Does | +| --- | --- | --- | --- | +| Rule of Thirds | `bool` | On | Shows vertical and horizontal thirds. | +| Center Crosshair | `bool` | On | Shows a center mark. | +| Line Color | `color` | White | Guide line color and alpha source. | +| Line Opacity | `float` | 0.65 | Overall visibility. | +| Line Thickness | `float` | 2 px | Thickness of guide lines. | +| Crosshair Size | `float` | 54 px | Length of the center crosshair. | +| Crosshair Gap | `float` | 10 px | Empty gap around the exact center. | + +## Tips + +- Put this layer near the bottom of the stack if you want it visible over the final look. +- Use a bright color for camera lineup, then bypass the layer for program output. +- If the guide is too strong on bright content, lower Line Opacity before changing color. + diff --git a/Shader-DVD-Bounce.md b/Shader-DVD-Bounce.md new file mode 100644 index 0000000..c5a7e09 --- /dev/null +++ b/Shader-DVD-Bounce.md @@ -0,0 +1,27 @@ +# DVD Bounce + +| Field | Value | +| --- | --- | +| Shader ID | `dvd-bounce` | +| Category | Built-in | +| Package | `shaders/dvd-bounce/` | +| Texture | `DVD_Logo.png` as `dvdLogoTexture` | + +DVD Bounce draws a transparent bouncing DVD-style logo over the current stack. The logo changes hue as it hits screen edges and can add a soft glow. + +## Controls + +| Control | Type | Default | What It Does | +| --- | --- | --- | --- | +| Logo Scale | `float` | 0.28 | Logo size relative to the smaller output dimension. | +| Bounce Speed | `float` | 0.22 | Motion speed. | +| Edge Padding | `float` | 0.018 | Margin from frame edges. | +| Glow | `float` | 0.18 | Colored glow around the logo. | +| Alpha | `float` | 1.0 | Overall overlay opacity. | + +## Tips + +- Put this layer near the bottom if it should sit on top of other effects. +- Reduce Alpha for a watermark-style overlay. +- Increase Edge Padding if the logo feels too close to frame edges. + diff --git a/Shader-Data-Mosh.md b/Shader-Data-Mosh.md new file mode 100644 index 0000000..c1f403a --- /dev/null +++ b/Shader-Data-Mosh.md @@ -0,0 +1,27 @@ +# Data Mosh + +| Field | Value | +| --- | --- | +| Shader ID | `data-mosh` | +| Category | Glitch | +| Package | `shaders/data-mosh/` | +| Temporal | `preLayerInput`, 8 requested frames | + +Data Mosh creates a blocky temporal glitch by sampling older pre-layer frames, tearing rows horizontally, offsetting blocks, and adding chroma separation. + +## Controls + +| Control | Type | Default | What It Does | +| --- | --- | --- | --- | +| Amount | `float` | 0.45 | Overall strength of the temporal block replacement. | +| Block Count | `vec2` | `[32, 18]` | Number of blocks across X and Y. Higher values make smaller blocks. | +| Tear | `float` | 0.18 | Horizontal row tearing strength. | +| Chroma Shift | `float` | 1.8 | Red/blue channel separation in pixels. | + +## Tips + +- Lower Block Count makes larger, chunkier smears. +- Increase Amount first, then Tear, for a controlled glitch build. +- This is temporal, so it may settle after a few frames and reset after layer changes. +- Place it before color correction if you want later layers to tame the look. + diff --git a/Shader-False-Color.md b/Shader-False-Color.md new file mode 100644 index 0000000..7b11b99 --- /dev/null +++ b/Shader-False-Color.md @@ -0,0 +1,25 @@ +# False Color + +| Field | Value | +| --- | --- | +| Shader ID | `false-color` | +| Category | Utility | +| Package | `shaders/false-color/` | + +False Color maps image luminance to exposure-assist colors. It is useful for camera setup, debugging luma ranges, and seeing how other shaders affect brightness. + +## Controls + +| Control | Type | Default | What It Does | +| --- | --- | --- | --- | +| Blend | `float` | 1.0 | Mix between source image and false-color output. | +| Show Luma | `bool` | Off | Blends grayscale luma into the false-color map. | +| Lift | `float` | 0.0 | Offsets measured luma before mapping. | +| Gain | `float` | 1.0 | Scales measured luma before mapping. | + +## Tips + +- Use full Blend for exposure checks. +- Use partial Blend when checking a look without losing the underlying picture. +- Adjust Lift and Gain only if you are intentionally inspecting a shifted range. + diff --git a/Shader-Fisheye-Reproject.md b/Shader-Fisheye-Reproject.md new file mode 100644 index 0000000..8eea267 --- /dev/null +++ b/Shader-Fisheye-Reproject.md @@ -0,0 +1,44 @@ +# Fisheye Reproject + +| Field | Value | +| --- | --- | +| Shader ID | `fisheye-reproject` | +| Category | Projection | +| Package | `shaders/fisheye-reproject/` | + +Fisheye Reproject treats the source as a fisheye camera image and inverse-projects it into a virtual rectilinear or cylindrical camera view. It gives you pan, tilt, roll, lens model, optical center, radius, and outside color controls. + +## Controls + +| Control | Type | Default | What It Does | +| --- | --- | --- | --- | +| Lens FOV | `float` | 190 | Fisheye lens field of view in degrees. | +| Optical Center | `vec2` | `[0.5, 0.5]` | Fisheye center in normalized source coordinates. | +| Fisheye Radius | `vec2` | `[0.5, 0.885]` | Horizontal and vertical fisheye radius. | +| Virtual FOV | `float` | 75 | Field of view of the virtual output camera. | +| Base Pan | `float` | 0 | Fixed pan offset. | +| Base Tilt | `float` | 0 | Fixed tilt offset. | +| Base Roll | `float` | 0 | Fixed roll offset. | +| Pan | `float` | 0 | Live pan control. | +| Tilt | `float` | 0 | Live tilt control. | +| Roll | `float` | 0 | Live roll control. | +| Fisheye Model | `enum` | Equidistant | Lens projection model. | +| Output Projection | `enum` | Rectilinear | Rectilinear or cylindrical output. | +| Outside Color | `color` | Black | Fill color outside valid source projection. | + +## Lens Models + +| Option | Use | +| --- | --- | +| Equidistant | Common action-camera and dome-style starting point. | +| Equisolid | Try if edges feel compressed or expanded incorrectly. | +| Stereographic | Try for some wide-angle lenses. | +| Orthographic | Try for different fisheye calibration behavior. | + +## Tips + +- Start by setting Optical Center and Fisheye Radius until the source circle lines up. +- Set Lens FOV next, then Virtual FOV. +- Use Base Pan/Tilt/Roll for calibration and Pan/Tilt/Roll for live movement. +- Cylindrical output can feel more natural for very wide horizontal pans. + diff --git a/Shader-Gaussian-Blur.md b/Shader-Gaussian-Blur.md new file mode 100644 index 0000000..890e177 --- /dev/null +++ b/Shader-Gaussian-Blur.md @@ -0,0 +1,25 @@ +# Gaussian Blur + +| Field | Value | +| --- | --- | +| Shader ID | `gaussian-blur` | +| Category | Built-in | +| Package | `shaders/gaussian-blur/` | + +Gaussian Blur softens the current image by sampling neighboring pixels with a Gaussian-style weight. + +## Controls + +| Control | Type | Default | What It Does | +| --- | --- | --- | --- | +| Radius | `float` | 2.0 | Pixel distance between blur samples. | +| Strength | `float` | 1.0 | Mix between original and blurred image. | +| Samples | `float` | 2 | Sample radius. Higher values are smoother but heavier. | + +## Tips + +- Increase Radius for a wider blur. +- Increase Samples for a smoother blur, but watch render time. +- Lower Strength for a gentle diffusion layer. +- Put blur before VHS, color, or overlay shaders when you want those later effects to stay crisp. + diff --git a/Shader-Greenscreen-Key.md b/Shader-Greenscreen-Key.md new file mode 100644 index 0000000..c5754da --- /dev/null +++ b/Shader-Greenscreen-Key.md @@ -0,0 +1,31 @@ +# Greenscreen Key + +| Field | Value | +| --- | --- | +| Shader ID | `greenscreen-key` | +| Category | Built-in | +| Package | `shaders/greenscreen-key/` | + +Greenscreen Key removes a green screen background and outputs transparent alpha for compositing. The shader returns premultiplied-looking color by multiplying the keyed RGB by alpha. + +## Controls + +| Control | Type | Default | What It Does | +| --- | --- | --- | --- | +| Screen Color | `color` | Green | Target key color. | +| Threshold | `float` | 0.24 | Distance from key color where pixels become foreground. | +| Softness | `float` | 0.12 | Feather around the key edge. | +| Edge Softness | `float` | 0.08 | Extra edge smoothing. | +| Erode/Dilate | `float` | 0.0 | Shrinks or expands the matte. | +| Despill | `float` | 0.45 | Reduces green contamination in foreground. | +| Edge Boost | `float` | 0.08 | Raises edge alpha. | +| Clip Black | `float` | 0.0 | Cuts low alpha values to black/transparent. | +| Clip White | `float` | 1.0 | Pushes high alpha values to full opacity. | + +## Tips + +- Start with Screen Color, Threshold, and Softness. +- Use Erode/Dilate sparingly to clean edges. +- Increase Despill when skin, hair, or props pick up green. +- Use Clip Black and Clip White after the matte is roughly correct. + diff --git a/Shader-Pixelate.md b/Shader-Pixelate.md new file mode 100644 index 0000000..fe11b18 --- /dev/null +++ b/Shader-Pixelate.md @@ -0,0 +1,24 @@ +# Pixelate + +| Field | Value | +| --- | --- | +| Shader ID | `pixelate` | +| Category | Utility | +| Package | `shaders/pixelate/` | + +Pixelate samples the image at the center of virtual cells, creating a low-resolution block effect. It can also draw a grid between cells. + +## Controls + +| Control | Type | Default | What It Does | +| --- | --- | --- | --- | +| Pixel Count | `vec2` | `[96, 54]` | Number of virtual pixels across X and Y. | +| Grid | `float` | 0.0 | Grid line visibility. | +| Grid Color | `color` | Black | Grid color and alpha. | + +## Tips + +- Lower Pixel Count for larger blocks. +- Match X/Y counts to the output aspect ratio to avoid stretched cells. +- Use a partial Grid amount for a game-screen or LED-panel feel. + diff --git a/Shader-Safe-Area-Guides.md b/Shader-Safe-Area-Guides.md new file mode 100644 index 0000000..997d0e8 --- /dev/null +++ b/Shader-Safe-Area-Guides.md @@ -0,0 +1,29 @@ +# Safe Area Guides + +| Field | Value | +| --- | --- | +| Shader ID | `safe-area-guides` | +| Category | Utility | +| Package | `shaders/safe-area-guides/` | + +Safe Area Guides overlays broadcast action safe, title safe, optional center marks, and optional aspect-ratio mattes. + +## Controls + +| Control | Type | Default | What It Does | +| --- | --- | --- | --- | +| Action Safe | `bool` | On | Shows 5 percent inset action-safe rectangle. | +| Title Safe | `bool` | On | Shows 10 percent inset title-safe rectangle. | +| Center Marks | `bool` | On | Shows short center guides. | +| Line Color | `color` | White | Guide color and alpha source. | +| Line Opacity | `float` | 0.65 | Guide visibility. | +| Line Thickness | `float` | 2 px | Thickness of guide lines. | +| Aspect Matte | `enum` | None | Adds a matte for 2.39:1, 1.85:1, or 1:1. | +| Matte Opacity | `float` | 0.35 | Darkness of the aspect matte. | + +## Tips + +- Use this during layout and rehearsal, then bypass for final output if guides should not be visible. +- Put it near the bottom of the layer stack so it overlays the final image. +- Use Matte Opacity instead of Line Opacity when adjusting aspect preview darkness. + diff --git a/Shader-Studio-Color.md b/Shader-Studio-Color.md new file mode 100644 index 0000000..369fc30 --- /dev/null +++ b/Shader-Studio-Color.md @@ -0,0 +1,34 @@ +# Studio Color + +| Field | Value | +| --- | --- | +| Shader ID | `studio-color` | +| Category | Built-in | +| Package | `shaders/studio-color/` | + +Studio Color is a practical color utility and a clear example of the shader parameter contract. It demonstrates float, vector, color, boolean, and enum controls. + +## Controls + +| Control | Type | Default | What It Does | +| --- | --- | --- | --- | +| Brightness | `float` | 1.0 | Multiplies RGB brightness. | +| Offset | `vec2` | `[0, 0]` | Samples the image from an offset UV position. | +| Tint | `color` | White | Multiplies the full color by a tint. | +| Invert | `bool` | Off | Inverts RGB. | +| Mode | `enum` | Normal | Normal, Luma, or Posterize output. | + +## Modes + +| Mode | Result | +| --- | --- | +| Normal | Applies brightness, offset, tint, and optional invert. | +| Luma | Converts the result to grayscale. | +| Posterize | Quantizes the result into four brightness steps. | + +## Tips + +- Use it as a quick utility layer for brightness and tint. +- Use it as a reference when creating custom shaders because it exercises every supported parameter type. +- Offset is clamped to the frame, so it will not wrap. + diff --git a/Shader-Temporal-Echo.md b/Shader-Temporal-Echo.md new file mode 100644 index 0000000..35029af --- /dev/null +++ b/Shader-Temporal-Echo.md @@ -0,0 +1,27 @@ +# Temporal Echo + +| Field | Value | +| --- | --- | +| Shader ID | `temporal-echo` | +| Category | Temporal | +| Package | `shaders/temporal-echo/` | +| Temporal | `preLayerInput`, 12 requested frames | + +Temporal Echo blends multiple older frames into the current image with decay and tint. It creates visible motion echoes rather than a simple blur. + +## Controls + +| Control | Type | Default | What It Does | +| --- | --- | --- | --- | +| Echo Amount | `float` | 0.55 | Mix between current frame and echo composite. | +| Decay | `float` | 0.72 | How quickly older echoes fade. | +| Frame Stride | `float` | 2 | Spacing between sampled history frames. | +| Echo Tint | `color` | Pale blue | Tint and alpha applied to echo frames. | + +## Tips + +- Increase Frame Stride for more separated echoes. +- Lower Decay for shorter trails. +- Use Echo Tint alpha to reduce echo strength without changing color. +- This shader resets history after stack and bypass changes. + diff --git a/Shader-Temporal-Ghost-Trail.md b/Shader-Temporal-Ghost-Trail.md new file mode 100644 index 0000000..d2b38df --- /dev/null +++ b/Shader-Temporal-Ghost-Trail.md @@ -0,0 +1,24 @@ +# Temporal Ghost Trail + +| Field | Value | +| --- | --- | +| Shader ID | `temporal-ghost-trail` | +| Category | Built-in | +| Package | `shaders/temporal-ghost-trail/` | +| Temporal | `preLayerInput`, 12 requested frames | + +Temporal Ghost Trail blends recent pre-layer frames with the current frame for a soft motion trail. + +## Controls + +| Control | Type | Default | What It Does | +| --- | --- | --- | --- | +| Current Mix | `float` | 0.72 | Weight of the current frame. | +| Trail Mix | `float` | 0.28 | Weight of previous frames. | + +## Tips + +- Increase Trail Mix for stronger ghosts. +- Lower Current Mix when you want motion to dominate over the live frame. +- Place before color or stylization layers if you want the trail treated as part of the image. + diff --git a/Shader-Temporal-Low-FPS.md b/Shader-Temporal-Low-FPS.md new file mode 100644 index 0000000..cba3c18 --- /dev/null +++ b/Shader-Temporal-Low-FPS.md @@ -0,0 +1,24 @@ +# Temporal Low FPS + +| Field | Value | +| --- | --- | +| Shader ID | `temporal-low-fps` | +| Category | Built-in | +| Package | `shaders/temporal-low-fps/` | +| Temporal | `preLayerInput`, 8 requested frames | + +Temporal Low FPS holds recent frames to create choppy, deliberately low-frame-rate motion while the host itself keeps running at the configured video rate. + +## Controls + +| Control | Type | Default | What It Does | +| --- | --- | --- | --- | +| Hold Frames | `float` | 3.0 | How many frames to hold before updating. | +| Blend | `float` | 1.0 | Mix between live source and held-frame result. | + +## Tips + +- Use whole-number Hold Frames for the cleanest stepped motion. +- Use partial Blend to add stutter without fully freezing motion. +- This shader needs a few frames of history after reload before the held-frame look stabilizes. + diff --git a/Shader-VHS.md b/Shader-VHS.md new file mode 100644 index 0000000..d7bb4ee --- /dev/null +++ b/Shader-VHS.md @@ -0,0 +1,33 @@ +# VHS + +| Field | Value | +| --- | --- | +| Shader ID | `vhs` | +| Category | Built-in | +| Package | `shaders/vhs/` | + +VHS creates an analog tape look with horizontal wiggle, YIQ-style smear, chromatic aberration, halation, bloom, noise, fade, and vignette. + +## Controls + +| Control | Type | Default | What It Does | +| --- | --- | --- | --- | +| Wiggle | `float` | 0.03 | Strength of horizontal/vertical instability. | +| Wiggle Speed | `float` | 25 | Animation speed for tape wobble. | +| Smear | `float` | 1.0 | Horizontal color/luma smear. | +| Blur Samples | `float` | 15 | Samples used for the tape blur. | +| Vignette | `float` | 0.18 | Darkening near edges. | +| Aberration | `float` | 0.75 | Red/blue separation away from center. | +| Halation | `float` | 0.12 | Warm glow around bright areas. | +| Bloom | `float` | 0.18 | Soft bright-area bloom. | +| Fade | `float` | 0.22 | Washed, aged image response. | +| Noise | `float` | 0.055 | Animated chroma/luma noise amount. | +| Noise Size | `float` | 1.0 | Grain chunk size. | + +## Tips + +- For a clean analog grade, start with Smear, Halation, Bloom, and Fade. +- For harsher tape damage, increase Wiggle and Noise. +- Blur Samples affects cost. Reduce it if render time is tight. +- Put VHS late in the stack if you want it to degrade the whole finished look. + diff --git a/Shader-Video-Cube.md b/Shader-Video-Cube.md new file mode 100644 index 0000000..741735a --- /dev/null +++ b/Shader-Video-Cube.md @@ -0,0 +1,25 @@ +# Video Cube + +| Field | Value | +| --- | --- | +| Shader ID | `video-cube` | +| Category | Built-in | +| Package | `shaders/video-cube/` | + +Video Cube raycasts a rotating cube in screen space and maps the live video onto its faces. Areas outside the cube show a dark background mixed with the source. + +## Controls + +| Control | Type | Default | What It Does | +| --- | --- | --- | --- | +| Spin Speed | `float` | 1.0 | Cube rotation speed. | +| Cube Scale | `float` | 0.85 | Cube size. | +| Face Zoom | `float` | 1.0 | Scale of video mapped onto each face. | +| Background Mix | `float` | 0.15 | Amount of source video visible in the background. | + +## Tips + +- Lower Background Mix for a more isolated object. +- Increase Face Zoom if the mapped video feels too wide on each face. +- Keep Cube Scale below the maximum if you want the full object visible. + diff --git a/Shader-Video-Transform.md b/Shader-Video-Transform.md new file mode 100644 index 0000000..a64d63d --- /dev/null +++ b/Shader-Video-Transform.md @@ -0,0 +1,35 @@ +# Video Transform + +| Field | Value | +| --- | --- | +| Shader ID | `video-transform` | +| Category | Utility | +| Package | `shaders/video-transform/` | + +Video Transform zooms, pans, and rotates the image by remapping output pixels back into source UV space. + +## Controls + +| Control | Type | Default | What It Does | +| --- | --- | --- | --- | +| Zoom | `float` | 1.0 | Scale of the source image. Values above 1 zoom in. | +| Pan | `vec2` | `[0, 0]` | Horizontal and vertical offset. | +| Rotation | `float` | 0 | Rotation in degrees. | +| Edge Mode | `enum` | Black | How samples outside the source frame are handled. | +| Outside Color | `color` | Black | Fill color when Edge Mode is Black. | + +## Edge Modes + +| Mode | Result | +| --- | --- | +| Black | Outside areas use Outside Color. | +| Clamp | Holds the nearest edge pixel. | +| Wrap | Wraps around the source image. | +| Mirror | Mirrors the source image at edges. | + +## Tips + +- Use this before keying or overlays if you need to line up the source first. +- Use Clamp or Mirror for rotations where black corners are unwanted. +- Use OSC on `pan` for XY pad control. + diff --git a/Shader-Waveform-Overlay.md b/Shader-Waveform-Overlay.md new file mode 100644 index 0000000..8e588c3 --- /dev/null +++ b/Shader-Waveform-Overlay.md @@ -0,0 +1,34 @@ +# Waveform Overlay + +| Field | Value | +| --- | --- | +| Shader ID | `waveform-overlay` | +| Category | Utility | +| Package | `shaders/waveform-overlay/` | +| Textures | `0.png`, `25.png`, `50.png`, `75.png`, `100.png` | + +Waveform Overlay draws a lightweight luma waveform inside a movable overlay box. It includes grid lines and texture labels for 0, 25, 50, 75, and 100. + +## Controls + +| Control | Type | Default | What It Does | +| --- | --- | --- | --- | +| Overlay Scale | `float` | 0.4 | Width of the overlay. | +| Overlay Position | `vec2` | `[0.24, 0.76]` | Overlay center in normalized screen space. | +| Overlay Padding | `float` | 0.08 | Internal padding around the waveform area. | +| Waveform Opacity | `float` | 0.75 | Luma trace visibility. | +| Background | `float` | 0.75 | Darkness of the overlay background. | +| Line Thickness | `float` | 1.5 | Thickness of waveform hits. | +| Grid Opacity | `float` | 1.0 | Visibility of reference grid lines. | +| Waveform Samples | `float` | 64 | Vertical samples used to build the waveform. | +| Waveform Gain | `float` | 12 | Intensity of the trace. | +| Noise Reduction | `float` | 0.08 | Suppresses faint waveform noise. | +| Waveform Color | `color` | White | Trace color and alpha. | + +## Tips + +- Use Overlay Position to keep the scope clear of important picture content. +- Increase Waveform Samples for more detail, but keep an eye on render time. +- Raise Noise Reduction if the trace looks too fuzzy. +- Put this layer last when you want the waveform visible over the final output. + diff --git a/Shaders.md b/Shaders.md new file mode 100644 index 0000000..ebcd832 --- /dev/null +++ b/Shaders.md @@ -0,0 +1,50 @@ +# Shader Library + +Each included shader is a package under `shaders//` in the main `video-shader` repository. A package normally contains a `shader.json` manifest and a `shader.slang` source file. Some shaders also include texture assets. + +The browser UI lists shaders by display name and category. Presets and automation use the shader package `id`. + +## Categories + +| Category | Shaders | +| --- | --- | +| Built-in | Black and White, DVD Bounce, Gaussian Blur, Greenscreen Key, Studio Color, Temporal Ghost Trail, Temporal Low FPS, VHS, Video Cube | +| Utility | Composition Guides, False Color, Pixelate, Safe Area Guides, Video Transform, Waveform Overlay | +| Glitch | Data Mosh | +| Projection | Fisheye Reproject | +| Temporal | Temporal Echo | + +## Temporal Shaders + +Temporal shaders use previous frames. They need history buffers, so they can cost more memory and can briefly look different immediately after startup, reload, layer reorder, bypass changes, or shader changes. + +| Shader | History Source | Requested Frames | +| --- | --- | --- | +| Data Mosh | `preLayerInput` | 8 | +| Temporal Echo | `preLayerInput` | 12 | +| Temporal Ghost Trail | `preLayerInput` | 12 | +| Temporal Low FPS | `preLayerInput` | 8 | + +The runtime may clamp these requests using `maxTemporalHistoryFrames` in `runtime-host.json`. + +## Pages + +- [Black and White](Shader-Black-and-White) +- [Composition Guides](Shader-Composition-Guides) +- [Data Mosh](Shader-Data-Mosh) +- [DVD Bounce](Shader-DVD-Bounce) +- [False Color](Shader-False-Color) +- [Fisheye Reproject](Shader-Fisheye-Reproject) +- [Gaussian Blur](Shader-Gaussian-Blur) +- [Greenscreen Key](Shader-Greenscreen-Key) +- [Pixelate](Shader-Pixelate) +- [Safe Area Guides](Shader-Safe-Area-Guides) +- [Studio Color](Shader-Studio-Color) +- [Temporal Echo](Shader-Temporal-Echo) +- [Temporal Ghost Trail](Shader-Temporal-Ghost-Trail) +- [Temporal Low FPS](Shader-Temporal-Low-FPS) +- [VHS](Shader-VHS) +- [Video Cube](Shader-Video-Cube) +- [Video Transform](Shader-Video-Transform) +- [Waveform Overlay](Shader-Waveform-Overlay) +