video-shader-toys/SHADER_CONTRACT.md

Shader Package Contract

This document explains how to create shaders for the Video Shader runtime.

Each shader is a small package under shaders/<id>/:

shaders/my-effect/
  shader.json
  shader.slang
  optional-texture.png

The runtime reads shader.json, generates a Slang wrapper from runtime/templates/shader_wrapper.slang.in, includes your shader.slang, compiles the result to GLSL, and exposes the shader in the local control UI.

Quick Start

Create a folder:

shaders/my-effect/

Add shader.json:

{
  "id": "my-effect",
  "name": "My Effect",
  "description": "A simple starter shader.",
  "category": "Custom",
  "entryPoint": "shadeVideo",
  "parameters": [
    {
      "id": "strength",
      "label": "Strength",
      "type": "float",
      "default": 0.5,
      "min": 0.0,
      "max": 1.0,
      "step": 0.01
    }
  ]
}

Add shader.slang:

float4 shadeVideo(ShaderContext context)
{
    float4 color = context.sourceColor;
    color.rgb = lerp(color.rgb, 1.0 - color.rgb, strength);
    return saturate(color);
}

With autoReload enabled in config/runtime-host.json, edits to shader source, manifests, and declared texture assets are picked up automatically.

Manifest Fields

shader.json is the runtime-facing description of the shader.

Required fields:

• id: package ID used by state/presets. Hyphenated names are OK here, for example my-effect.
• name: display name in the UI.
• parameters: array of exposed controls. Use [] if there are no user parameters.

Optional fields:

• description: display/help text for the shader library.
• category: UI grouping label.
• entryPoint: Slang function to call. Defaults to shadeVideo.
• textures: texture assets to load and expose as samplers.
• fonts: packaged font assets for live text parameters.
• temporal: history-buffer requirements.

Shader-visible identifiers must be valid Slang-style identifiers:

• entryPoint
• parameter id
• texture id
• font id

Use letters, numbers, and underscores only, and start with a letter or underscore. For example, logoTexture is valid; logo-texture is not valid as a shader-visible texture ID.

Slang Entry Point

Your shader file must implement the manifest entryPoint.

Default:

float4 shadeVideo(ShaderContext context)
{
    return context.sourceColor;
}

The runtime owns the real fragment shader entry point. Your function is called from the wrapper, and the runtime handles final bypass/mix behavior:

return lerp(context.sourceColor, effectedColor, mixValue);

That means:

• Return the fully effected color from your function.
• Respect alpha if your shader produces an overlay or sprite.
• The runtime will blend your result with the source according to mixAmount and bypass state.

ShaderContext

Your entry point receives:

struct ShaderContext
{
    float2 uv;
    float4 sourceColor;
    float2 inputResolution;
    float2 outputResolution;
    float time;
    float frameCount;
    float mixAmount;
    float bypass;
    int sourceHistoryLength;
    int temporalHistoryLength;
};

Fields:

• uv: normalized texture coordinates, usually 0..1.
• sourceColor: decoded RGBA source video at uv.
• inputResolution: decoded input video resolution in pixels.
• outputResolution: shader render resolution in pixels. The current pipeline renders the shader stack at input resolution, then scales the final frame to the configured DeckLink output mode.
• time: elapsed runtime time in seconds.
• frameCount: incrementing frame counter.
• mixAmount: runtime mix amount.
• bypass: 1.0 when the layer is bypassed, otherwise 0.0.
• sourceHistoryLength: number of usable source-history frames currently available.
• temporalHistoryLength: number of usable temporal frames currently available for this layer.

Helper Functions

The wrapper provides:

float4 sampleVideo(float2 uv);
float4 sampleSourceHistory(int framesAgo, float2 uv);
float4 sampleTemporalHistory(int framesAgo, float2 uv);

sampleVideo samples the live decoded source video.

sampleSourceHistory samples previous decoded source frames. framesAgo is clamped into the available range. If no history is available, it falls back to sampleVideo.

sampleTemporalHistory samples previous pre-layer input frames for temporal shaders that request preLayerInput history. framesAgo is clamped into the available range. If no temporal history is available, it falls back to sampleVideo.

Example:

float4 shadeVideo(ShaderContext context)
{
    float4 previous = sampleSourceHistory(1, context.uv);
    return lerp(context.sourceColor, previous, 0.35);
}

Parameters

Manifest parameters are exposed to Slang as global values with the same id.

Supported types:

Manifest type	Slang type	JSON value
float	float	number
vec2	float2	[x, y]
color	float4	[r, g, b, a]
bool	bool	true or false
enum	int	selected option index
text	generated texture/helper	string

Float example:

{
  "id": "brightness",
  "label": "Brightness",
  "type": "float",
  "default": 1.0,
  "min": 0.0,
  "max": 2.0,
  "step": 0.01
}

color.rgb *= brightness;

Vector example:

{
  "id": "offset",
  "label": "Offset",
  "type": "vec2",
  "default": [0.0, 0.0],
  "min": [-0.2, -0.2],
  "max": [0.2, 0.2],
  "step": [0.001, 0.001]
}

float2 uv = clamp(context.uv + offset, float2(0.0), float2(1.0));

Color example:

{
  "id": "tint",
  "label": "Tint",
  "type": "color",
  "default": [1.0, 1.0, 1.0, 1.0]
}

color *= tint;

Boolean example:

{
  "id": "invert",
  "label": "Invert",
  "type": "bool",
  "default": false
}

if (invert)
    color.rgb = 1.0 - color.rgb;

Enum example:

{
  "id": "mode",
  "label": "Mode",
  "type": "enum",
  "default": "normal",
  "options": [
    { "value": "normal", "label": "Normal" },
    { "value": "luma", "label": "Luma" },
    { "value": "posterize", "label": "Posterize" }
  ]
}

Enums are stored in presets/state by their string value, but exposed to Slang as a zero-based integer index in option order:

if (mode == 1)
{
    float luma = dot(color.rgb, float3(0.2126, 0.7152, 0.0722));
    color.rgb = float3(luma);
}
else if (mode == 2)
{
    color.rgb = floor(color.rgb * 4.0) / 4.0;
}

Text example:

{
  "fonts": [
    { "id": "inter", "path": "fonts/Inter-Regular.ttf" }
  ],
  "parameters": [
    {
      "id": "titleText",
      "label": "Title",
      "type": "text",
      "default": "LIVE",
      "font": "inter",
      "maxLength": 64
    }
  ]
}

Text parameters are runtime-owned strings. They are not emitted as uniform values. Instead, the runtime renders the current string into a single-line SDF mask texture and the shader wrapper exposes helpers based on the parameter id:

float mask = sampleTitleText(textUv);
float4 premultipliedText = drawTitleText(textUv, float4(1.0, 1.0, 1.0, 1.0));

Text is currently limited to printable ASCII. maxLength defaults to 64 and is clamped to 1..256. The optional font field references a packaged font declared in fonts; if no font is specified, the runtime uses its fallback sans-serif renderer.

Parameter validation:

• Float values are clamped to min/max if provided.
• vec2 must have exactly 2 numbers.
• color must have exactly 4 numbers.
• Enum defaults must match one of the declared option values.
• Text defaults must be strings. Non-printable characters are dropped and values are clamped to maxLength.
• Non-finite numeric values are rejected.

Texture Assets

Declare texture assets in the manifest:

{
  "textures": [
    {
      "id": "logoTexture",
      "path": "logo.png"
    }
  ]
}

Rules:

• id must be a valid shader identifier.
• path is relative to the shader package directory.
• The file must exist when the manifest is loaded.
• Texture asset changes trigger shader reload.

Texture IDs become Sampler2D<float4> globals:

float4 logo = logoTexture.Sample(logoUv);

For sprite or overlay shaders, return premultiplied-looking output if you want clean composition:

float alpha = logo.a;
return float4(logo.rgb * alpha, alpha);

See shaders/dvd-bounce/ for a complete texture-driven example.

Font Assets

Declare packaged font assets in the manifest:

{
  "fonts": [
    {
      "id": "inter",
      "path": "fonts/Inter-Regular.ttf"
    }
  ]
}

Rules:

• id must be a valid shader identifier.
• path is relative to the shader package directory.
• The file must exist when the manifest is loaded.
• Font asset changes trigger shader reload.
• V1 text layout is single-line; shaders position and scale the generated text texture themselves.

See shaders/text-overlay/ for a complete live text example. The sample bundles Roboto Regular and includes its OFL license beside the font file.

Temporal Shaders

Temporal shaders can request access to previous frames.

Manifest example:

{
  "temporal": {
    "enabled": true,
    "historySource": "preLayerInput",
    "historyLength": 12
  }
}

Supported historySource values:

• source: decoded source-video history from previous frames.
• preLayerInput: history of the input arriving at this layer before the shader runs.

historyLength is the requested frame count. The runtime clamps it by maxTemporalHistoryFrames in config/runtime-host.json.

Temporal history resets when:

• layers are added, removed, or reordered
• a layer bypass state changes
• a layer changes shader
• a shader is reloaded or recompiled
• render dimensions change

Use the available history lengths to avoid assuming history is ready on the first frame:

float4 shadeVideo(ShaderContext context)
{
    if (context.temporalHistoryLength <= 0)
        return context.sourceColor;

    float4 oldFrame = sampleTemporalHistory(3, context.uv);
    return lerp(context.sourceColor, oldFrame, 0.4);
}

See shaders/temporal-ghost-trail/ and shaders/temporal-low-fps/ for examples.

Coordinate And Color Notes

• uv is normalized.
• Use context.outputResolution for pixel-sized effects.
• Use context.inputResolution when sampling source video by input pixel size.
• sourceColor and sampleVideo return RGBA values in normalized 0..1 range.
• Prefer saturate(color) or explicit clamp before returning if your math can overshoot.

Pixel-size example:

float2 pixel = 1.0 / max(context.outputResolution, float2(1.0));
float4 right = sampleVideo(context.uv + float2(pixel.x, 0.0));

Reload And Generated Files

When a shader compiles, the runtime writes generated files under runtime/shader_cache/:

• active_shader_wrapper.slang
• active_shader.raw.frag
• active_shader.frag

These files are ignored by git and are useful for debugging compiler output. If a shader fails to compile, inspect the wrapper first; it shows the exact generated Slang code including your included shader.

Common Pitfalls

• Do not use hyphens in parameter IDs, texture IDs, or entry point names.
• Do not declare your own ShaderContext, GlobalParams, sampleVideo, sampleSourceHistory, or sampleTemporalHistory.
• Do not write a [shader("fragment")] entry point in shader.slang; the runtime provides it.
• Remember enum globals are integer indexes, not strings.
• Declare every texture in shader.json; undeclared texture samplers will not be bound.
• Declare packaged fonts in shader.json when text parameters should use a specific font.
• Keep temporal history requests modest. They consume texture units and memory and are capped by runtime config.
• If a parameter appears in the UI but not in Slang, the shader may still compile, but the control has no effect.
• If a Slang name collides with a generated global, rename your parameter or local symbol.

Minimal Package Checklist

Before committing a new shader package:

• shader.json is valid JSON.
• id is unique across shaders/.
• entryPoint, parameter IDs, and texture IDs are valid identifiers.
• shader.slang implements the configured entry point.
• Texture files referenced by textures exist.
• Font files referenced by fonts exist.
• Enum defaults are present in their options.
• Temporal shaders handle short or empty history gracefully.
• The app can reload and compile the shader without errors.