video-shader-toys/SHADER_CONTRACT.md

# Shader Package Contract

This document explains how to create shaders for the Video Shader runtime.

Each shader is a small package under `shaders/<id>/`:

```text
shaders/my-effect/
  shader.json
  shader.slang
  optional-texture.png
```

The runtime reads `shader.json`, generates a Slang wrapper from `runtime/templates/shader_wrapper.slang.in`, includes your `shader.slang`, compiles the result to GLSL, and exposes the shader in the local control UI.

## Quick Start

Create a folder:

```text
shaders/my-effect/
```

Add `shader.json`:

```json
{
  "id": "my-effect",
  "name": "My Effect",
  "description": "A simple starter shader.",
  "category": "Custom",
  "entryPoint": "shadeVideo",
  "parameters": [
    {
      "id": "strength",
      "label": "Strength",
      "type": "float",
      "default": 0.5,
      "min": 0.0,
      "max": 1.0,
      "step": 0.01
    }
  ]
}
```

Add `shader.slang`:

```slang
float4 shadeVideo(ShaderContext context)
{
    float4 color = context.sourceColor;
    color.rgb = lerp(color.rgb, 1.0 - color.rgb, strength);
    return saturate(color);
}
```

With `autoReload` enabled in `config/runtime-host.json`, edits to shader source, manifests, and declared texture assets are picked up automatically.

## Manifest Fields

`shader.json` is the runtime-facing description of the shader.

Required fields:

- `id`: package ID used by state/presets. Hyphenated names are OK here, for example `my-effect`.
- `name`: display name in the UI.
- `parameters`: array of exposed controls. Use `[]` if there are no user parameters.

Optional fields:

- `description`: display/help text for the shader library.
- `category`: UI grouping label.
- `entryPoint`: Slang function to call. Defaults to `shadeVideo`.
- `textures`: texture assets to load and expose as samplers.
- `temporal`: history-buffer requirements.

Shader-visible identifiers must be valid Slang-style identifiers:

- `entryPoint`
- parameter `id`
- texture `id`

Use letters, numbers, and underscores only, and start with a letter or underscore. For example, `logoTexture` is valid; `logo-texture` is not valid as a shader-visible texture ID.

## Slang Entry Point

Your shader file must implement the manifest `entryPoint`.

Default:

```slang
float4 shadeVideo(ShaderContext context)
{
    return context.sourceColor;
}
```

The runtime owns the real fragment shader entry point. Your function is called from the wrapper, and the runtime handles final bypass/mix behavior:

```slang
return lerp(context.sourceColor, effectedColor, mixValue);
```

That means:

- Return the fully effected color from your function.
- Respect alpha if your shader produces an overlay or sprite.
- The runtime will blend your result with the source according to `mixAmount` and bypass state.

## ShaderContext

Your entry point receives:

```slang
struct ShaderContext
{
    float2 uv;
    float4 sourceColor;
    float2 inputResolution;
    float2 outputResolution;
    float time;
    float frameCount;
    float mixAmount;
    float bypass;
    int sourceHistoryLength;
    int temporalHistoryLength;
};
```

Fields:

- `uv`: normalized texture coordinates, usually `0..1`.
- `sourceColor`: decoded RGBA source video at `uv`.
- `inputResolution`: decoded input video resolution in pixels.
- `outputResolution`: shader render resolution in pixels. The current pipeline renders the shader stack at input resolution, then scales the final frame to the configured DeckLink output mode.
- `time`: elapsed runtime time in seconds.
- `frameCount`: incrementing frame counter.
- `mixAmount`: runtime mix amount.
- `bypass`: `1.0` when the layer is bypassed, otherwise `0.0`.
- `sourceHistoryLength`: number of usable source-history frames currently available.
- `temporalHistoryLength`: number of usable temporal frames currently available for this layer.

## Helper Functions

The wrapper provides:

```slang
float4 sampleVideo(float2 uv);
float4 sampleSourceHistory(int framesAgo, float2 uv);
float4 sampleTemporalHistory(int framesAgo, float2 uv);
```

`sampleVideo` samples the live decoded source video.

`sampleSourceHistory` samples previous decoded source frames. `framesAgo` is clamped into the available range. If no history is available, it falls back to `sampleVideo`.

`sampleTemporalHistory` samples previous pre-layer input frames for temporal shaders that request `preLayerInput` history. `framesAgo` is clamped into the available range. If no temporal history is available, it falls back to `sampleVideo`.

Example:

```slang
float4 shadeVideo(ShaderContext context)
{
    float4 previous = sampleSourceHistory(1, context.uv);
    return lerp(context.sourceColor, previous, 0.35);
}
```

## Parameters

Manifest parameters are exposed to Slang as global values with the same `id`.

Supported types:

| Manifest type | Slang type | JSON value |
| --- | --- | --- |
| `float` | `float` | number |
| `vec2` | `float2` | `[x, y]` |
| `color` | `float4` | `[r, g, b, a]` |
| `bool` | `bool` | `true` or `false` |
| `enum` | `int` | selected option index |

Float example:

```json
{
  "id": "brightness",
  "label": "Brightness",
  "type": "float",
  "default": 1.0,
  "min": 0.0,
  "max": 2.0,
  "step": 0.01
}
```

```slang
color.rgb *= brightness;
```

Vector example:

```json
{
  "id": "offset",
  "label": "Offset",
  "type": "vec2",
  "default": [0.0, 0.0],
  "min": [-0.2, -0.2],
  "max": [0.2, 0.2],
  "step": [0.001, 0.001]
}
```

```slang
float2 uv = clamp(context.uv + offset, float2(0.0), float2(1.0));
```

Color example:

```json
{
  "id": "tint",
  "label": "Tint",
  "type": "color",
  "default": [1.0, 1.0, 1.0, 1.0]
}
```

```slang
color *= tint;
```

Boolean example:

```json
{
  "id": "invert",
  "label": "Invert",
  "type": "bool",
  "default": false
}
```

```slang
if (invert)
    color.rgb = 1.0 - color.rgb;
```

Enum example:

```json
{
  "id": "mode",
  "label": "Mode",
  "type": "enum",
  "default": "normal",
  "options": [
    { "value": "normal", "label": "Normal" },
    { "value": "luma", "label": "Luma" },
    { "value": "posterize", "label": "Posterize" }
  ]
}
```

Enums are stored in presets/state by their string `value`, but exposed to Slang as a zero-based integer index in option order:

```slang
if (mode == 1)
{
    float luma = dot(color.rgb, float3(0.2126, 0.7152, 0.0722));
    color.rgb = float3(luma);
}
else if (mode == 2)
{
    color.rgb = floor(color.rgb * 4.0) / 4.0;
}
```

Parameter validation:

- Float values are clamped to `min`/`max` if provided.
- `vec2` must have exactly 2 numbers.
- `color` must have exactly 4 numbers.
- Enum defaults must match one of the declared option values.
- Non-finite numeric values are rejected.

## Texture Assets

Declare texture assets in the manifest:

```json
{
  "textures": [
    {
      "id": "logoTexture",
      "path": "logo.png"
    }
  ]
}
```

Rules:

- `id` must be a valid shader identifier.
- `path` is relative to the shader package directory.
- The file must exist when the manifest is loaded.
- Texture asset changes trigger shader reload.

Texture IDs become `Sampler2D<float4>` globals:

```slang
float4 logo = logoTexture.Sample(logoUv);
```

For sprite or overlay shaders, return premultiplied-looking output if you want clean composition:

```slang
float alpha = logo.a;
return float4(logo.rgb * alpha, alpha);
```

See `shaders/dvd-bounce/` for a complete texture-driven example.

## Temporal Shaders

Temporal shaders can request access to previous frames.

Manifest example:

```json
{
  "temporal": {
    "enabled": true,
    "historySource": "preLayerInput",
    "historyLength": 12
  }
}
```

Supported `historySource` values:

- `source`: decoded source-video history from previous frames.
- `preLayerInput`: history of the input arriving at this layer before the shader runs.

`historyLength` is the requested frame count. The runtime clamps it by `maxTemporalHistoryFrames` in `config/runtime-host.json`.

Temporal history resets when:

- layers are added, removed, or reordered
- a layer bypass state changes
- a layer changes shader
- a shader is reloaded or recompiled
- render dimensions change

Use the available history lengths to avoid assuming history is ready on the first frame:

```slang
float4 shadeVideo(ShaderContext context)
{
    if (context.temporalHistoryLength <= 0)
        return context.sourceColor;

    float4 oldFrame = sampleTemporalHistory(3, context.uv);
    return lerp(context.sourceColor, oldFrame, 0.4);
}
```

See `shaders/temporal-ghost-trail/` and `shaders/temporal-low-fps/` for examples.

## Coordinate And Color Notes

- `uv` is normalized.
- Use `context.outputResolution` for pixel-sized effects.
- Use `context.inputResolution` when sampling source video by input pixel size.
- `sourceColor` and `sampleVideo` return RGBA values in normalized `0..1` range.
- Prefer `saturate(color)` or explicit `clamp` before returning if your math can overshoot.

Pixel-size example:

```slang
float2 pixel = 1.0 / max(context.outputResolution, float2(1.0));
float4 right = sampleVideo(context.uv + float2(pixel.x, 0.0));
```

## Reload And Generated Files

When a shader compiles, the runtime writes generated files under `runtime/shader_cache/`:

- `active_shader_wrapper.slang`
- `active_shader.raw.frag`
- `active_shader.frag`

These files are ignored by git and are useful for debugging compiler output. If a shader fails to compile, inspect the wrapper first; it shows the exact generated Slang code including your included shader.

## Common Pitfalls

- Do not use hyphens in parameter IDs, texture IDs, or entry point names.
- Do not declare your own `ShaderContext`, `GlobalParams`, `sampleVideo`, `sampleSourceHistory`, or `sampleTemporalHistory`.
- Do not write a `[shader("fragment")]` entry point in `shader.slang`; the runtime provides it.
- Remember enum globals are integer indexes, not strings.
- Declare every texture in `shader.json`; undeclared texture samplers will not be bound.
- Keep temporal history requests modest. They consume texture units and memory and are capped by runtime config.
- If a parameter appears in the UI but not in Slang, the shader may still compile, but the control has no effect.
- If a Slang name collides with a generated global, rename your parameter or local symbol.

## Minimal Package Checklist

Before committing a new shader package:

- `shader.json` is valid JSON.
- `id` is unique across `shaders/`.
- `entryPoint`, parameter IDs, and texture IDs are valid identifiers.
- `shader.slang` implements the configured entry point.
- Texture files referenced by `textures` exist.
- Enum defaults are present in their `options`.
- Temporal shaders handle short or empty history gracefully.
- The app can reload and compile the shader without errors.