# Shader Package Contract

This document explains how to create shaders for the Video Shader runtime.

Each shader is a small package under `shaders/<id>/`:

```text
shaders/my-effect/
  shader.json
  shader.slang
  optional-texture.png
```

The runtime reads `shader.json`, generates a Slang wrapper from `runtime/templates/shader_wrapper.slang.in`, includes your `shader.slang`, compiles the result to GLSL, and exposes the shader in the local control UI.

## Quick Start

Create a folder:

```text
shaders/my-effect/
```

Add `shader.json`:

```json
{
  "id": "my-effect",
  "name": "My Effect",
  "description": "A simple starter shader.",
  "category": "Custom",
  "entryPoint": "shadeVideo",
  "parameters": [
    {
      "id": "strength",
      "label": "Strength",
      "type": "float",
      "default": 0.5,
      "min": 0.0,
      "max": 1.0,
      "step": 0.01
    }
  ]
}
```

Add `shader.slang`:

```slang
float4 shadeVideo(ShaderContext context)
{
    float4 color = context.sourceColor;
    color.rgb = lerp(color.rgb, 1.0 - color.rgb, strength);
    return saturate(color);
}
```

With `autoReload` enabled in `config/runtime-host.json`, edits to shader source, manifests, and declared texture assets are picked up automatically.

## Guidance For Shaders

When generating a new shader package, prefer matching the existing runtime contract over copying code verbatim from Shadertoy, GLSL sandbox sites, or WebGL demos.

Important rules:

- Generate a complete package: `shaders/<id>/shader.json` and `shaders/<id>/shader.slang`.
- Use `float4 shadeVideo(ShaderContext context)` unless the manifest explicitly sets a different `entryPoint`.
- Do not create `mainImage`, `main`, `fragColor`, `iResolution`, `iTime`, `iChannel0`, or a fragment shader attribute layout. The runtime wrapper provides the real fragment entry point.
- Replace Shadertoy `fragCoord` with `context.uv * context.outputResolution`.
- Replace `iResolution.xy` with `context.outputResolution`.
- Replace `iTime` with `context.time`.
- Replace `iFrame` with `context.frameCount`.
- Replace source-video `iChannel0` sampling with `sampleVideo(uv)` or `context.sourceColor`.
- Use Slang/HLSL names and syntax: `float2`, `float3`, `float4`, `float2x2`, `lerp`, `frac`, `saturate`, and `mul(matrix, vector)`.
- Do not use GLSL-only types/functions such as `vec2`, `vec3`, `vec4`, `mat2`, `mix`, `fract`, `mod`, `texture`, or `mainImage`.
- Keep parameter IDs, texture IDs, font IDs, and function entry points as valid shader identifiers: letters, numbers, and underscores only, starting with a letter or underscore.
- Add only controls that are actually used by the shader.
- Prefer a small number of clear controls with conservative defaults.
- Keep shaders deterministic unless randomness is an explicit feature. For stable process-level variation, use `context.startupRandom`; for per-pixel pseudo-randomness, hash from `uv`, pixel coordinates, `frameCount`, or trigger values.
- If adapting third-party code, include attribution and source URL in the manifest description when the license allows adaptation.
- If the source license is unclear or incompatible, do not add the shader package.

Before finishing, compile-check the shader through the runtime wrapper or launch the app and verify the shader appears without an error in the selector.

## Manifest Fields

`shader.json` is the runtime-facing description of the shader.

Required fields:

- `id`: package ID used by state/presets. Hyphenated names are OK here, for example `my-effect`.
- `name`: display name in the UI.
- `parameters`: array of exposed controls. Use `[]` if there are no user parameters.

Optional fields:

- `description`: display/help text for the shader library.
- `category`: UI grouping label.
- `entryPoint`: Slang function to call. Defaults to `shadeVideo`.
- `textures`: texture assets to load and expose as samplers.
- `fonts`: packaged font assets for live text parameters.
- `temporal`: history-buffer requirements.

Shader-visible identifiers must be valid Slang-style identifiers:

- `entryPoint`
- parameter `id`
- texture `id`
- font `id`

Use letters, numbers, and underscores only, and start with a letter or underscore. For example, `logoTexture` is valid; `logo-texture` is not valid as a shader-visible texture ID.

## Slang Entry Point

Your shader file must implement the manifest `entryPoint`.

Default:

```slang
float4 shadeVideo(ShaderContext context)
{
    return context.sourceColor;
}
```

The runtime owns the real fragment shader entry point. Your function is called from the wrapper, and the runtime handles final bypass/mix behavior:

```slang
return lerp(context.sourceColor, effectedColor, mixValue);
```

That means:

- Return the fully effected color from your function.
- Respect alpha if your shader produces an overlay or sprite.
- The runtime will blend your result with the source according to `mixAmount` and bypass state.

## ShaderContext

Your entry point receives:

```slang
struct ShaderContext
{
    float2 uv;
    float4 sourceColor;
    float2 inputResolution;
    float2 outputResolution;
    float time;
    float utcTimeSeconds;
    float utcOffsetSeconds;
    float startupRandom;
    float frameCount;
    float mixAmount;
    float bypass;
    int sourceHistoryLength;
    int temporalHistoryLength;
};
```

Fields:

- `uv`: normalized texture coordinates, usually `0..1`.
- `sourceColor`: decoded RGBA source video at `uv`.
- `inputResolution`: decoded input video resolution in pixels.
- `outputResolution`: shader render resolution in pixels. The current pipeline renders the shader stack at input resolution, then scales the final frame to the configured video I/O output mode.
- `time`: elapsed runtime time in seconds.
- `utcTimeSeconds`: current UTC time of day from the host PC clock, expressed as seconds since UTC midnight.
- `utcOffsetSeconds`: host PC local UTC offset in seconds. Add this to `utcTimeSeconds` and wrap to `0..86400` to get local time of day.
- `startupRandom`: random `0..1` value generated once when the host process starts. It stays constant for the lifetime of the app and changes on the next launch.
- `frameCount`: incrementing frame counter.
- `mixAmount`: runtime mix amount.
- `bypass`: `1.0` when the layer is bypassed, otherwise `0.0`.
- `sourceHistoryLength`: number of usable source-history frames currently available.
- `temporalHistoryLength`: number of usable temporal frames currently available for this layer.

Color/precision notes:

- `context.sourceColor`, `sampleVideo()`, and temporal history samples are display-referred Rec.709-like RGB, not linear-light RGB.
- The current DeckLink backend prefers 10-bit YUV capture and output when the card/mode supports it, with automatic 8-bit fallback.
- Internal decoded, layer, composite, output, and temporal render targets are 16-bit floating point, so gradients and LUT work have more headroom than packed byte video I/O formats.
- Do not add extra Rec.709 or linear conversions unless the shader intentionally documents that behavior.

## Helper Functions

The wrapper provides:

```slang
float4 sampleVideo(float2 uv);
float4 sampleSourceHistory(int framesAgo, float2 uv);
float4 sampleTemporalHistory(int framesAgo, float2 uv);
```

`sampleVideo` samples the live decoded source video.

`sampleSourceHistory` samples previous decoded source frames. `framesAgo` is clamped into the available range. If no history is available, it falls back to `sampleVideo`.

`sampleTemporalHistory` samples previous pre-layer input frames for temporal shaders that request `preLayerInput` history. `framesAgo` is clamped into the available range. If no temporal history is available, it falls back to `sampleVideo`.

Example:

```slang
float4 shadeVideo(ShaderContext context)
{
    float4 previous = sampleSourceHistory(1, context.uv);
    return lerp(context.sourceColor, previous, 0.35);
}
```

## Parameters

Manifest parameters are exposed to Slang as global values with the same `id`.

Supported types:

| Manifest type | Slang type | JSON value |
| --- | --- | --- |
| `float` | `float` | number |
| `vec2` | `float2` | `[x, y]` |
| `color` | `float4` | `[r, g, b, a]` |
| `bool` | `bool` | `true` or `false` |
| `enum` | `int` | selected option index |
| `text` | generated texture/helper | string |
| `trigger` | `int <id>`, `float <id>Time` | pulse/count |

Float example:

```json
{
  "id": "brightness",
  "label": "Brightness",
  "type": "float",
  "default": 1.0,
  "min": 0.0,
  "max": 2.0,
  "step": 0.01
}
```

```slang
color.rgb *= brightness;
```

Vector example:

```json
{
  "id": "offset",
  "label": "Offset",
  "type": "vec2",
  "default": [0.0, 0.0],
  "min": [-0.2, -0.2],
  "max": [0.2, 0.2],
  "step": [0.001, 0.001]
}
```

```slang
float2 uv = clamp(context.uv + offset, float2(0.0), float2(1.0));
```

Color example:

```json
{
  "id": "tint",
  "label": "Tint",
  "type": "color",
  "default": [1.0, 1.0, 1.0, 1.0]
}
```

```slang
color *= tint;
```

Boolean example:

```json
{
  "id": "invert",
  "label": "Invert",
  "type": "bool",
  "default": false
}
```

```slang
if (invert)
    color.rgb = 1.0 - color.rgb;
```

Enum example:

```json
{
  "id": "mode",
  "label": "Mode",
  "type": "enum",
  "default": "normal",
  "options": [
    { "value": "normal", "label": "Normal" },
    { "value": "luma", "label": "Luma" },
    { "value": "posterize", "label": "Posterize" }
  ]
}
```

Enums are stored in presets/state by their string `value`, but exposed to Slang as a zero-based integer index in option order:

```slang
if (mode == 1)
{
    float luma = dot(color.rgb, float3(0.2126, 0.7152, 0.0722));
    color.rgb = float3(luma);
}
else if (mode == 2)
{
    color.rgb = floor(color.rgb * 4.0) / 4.0;
}
```

Text example:

```json
{
  "fonts": [
    { "id": "inter", "path": "fonts/Inter-Regular.ttf" }
  ],
  "parameters": [
    {
      "id": "titleText",
      "label": "Title",
      "type": "text",
      "default": "LIVE",
      "font": "inter",
      "maxLength": 64
    }
  ]
}
```

Text parameters are runtime-owned strings. They are not emitted as uniform values. Instead, the runtime renders the current string into a single-line SDF mask texture and the shader wrapper exposes helpers based on the parameter id:

```slang
float mask = sampleTitleText(textUv);
float4 premultipliedText = drawTitleText(textUv, float4(1.0, 1.0, 1.0, 1.0));
```

Text is currently limited to printable ASCII. `maxLength` defaults to `64` and is clamped to `1..256`. The optional `font` field references a packaged font declared in `fonts`; if no font is specified, the runtime uses its fallback sans-serif renderer.

Trigger example:

```json
{
  "id": "flash",
  "label": "Flash",
  "type": "trigger"
}
```

A trigger appears as a button in the control UI. Pressing it increments the shader-visible integer `flash` and records the runtime time in `flashTime`:

```slang
float age = context.time - flashTime;
float intensity = flash > 0 ? exp(-age * 5.0) : 0.0;
color.rgb += intensity;
```

Triggers are useful for one-shot shader reactions such as flashes, ripples, cuts, or randomized looks. They do not execute arbitrary CPU code; they only update uniforms consumed by the shader.

Parameter validation:

- Float values are clamped to `min`/`max` if provided.
- `vec2` must have exactly 2 numbers.
- `color` must have exactly 4 numbers.
- Enum defaults must match one of the declared option values.
- Text defaults must be strings. Non-printable characters are dropped and values are clamped to `maxLength`.
- Trigger values are incremented by the host when triggered. The shader sees the trigger count and last trigger time.
- Non-finite numeric values are rejected.

## Texture Assets

Declare texture assets in the manifest:

```json
{
  "textures": [
    {
      "id": "logoTexture",
      "path": "logo.png"
    }
  ]
}
```

Rules:

- `id` must be a valid shader identifier.
- `path` is relative to the shader package directory.
- The file must exist when the manifest is loaded.
- Texture asset changes trigger shader reload.

Texture IDs become `Sampler2D<float4>` globals:

```slang
float4 logo = logoTexture.Sample(logoUv);
```

For sprite or overlay shaders, return premultiplied-looking output if you want clean composition:

```slang
float alpha = logo.a;
return float4(logo.rgb * alpha, alpha);
```

See `shaders/dvd-bounce/` for a complete texture-driven example.

## Font Assets

Declare packaged font assets in the manifest:

```json
{
  "fonts": [
    {
      "id": "inter",
      "path": "fonts/Inter-Regular.ttf"
    }
  ]
}
```

Rules:

- `id` must be a valid shader identifier.
- `path` is relative to the shader package directory.
- The file must exist when the manifest is loaded.
- Font asset changes trigger shader reload.
- V1 text layout is single-line; shaders position and scale the generated text texture themselves.

See `shaders/text-overlay/` for a complete live text example. The sample bundles Roboto Regular and includes its OFL license beside the font file.

## Temporal Shaders

Temporal shaders can request access to previous frames.

Manifest example:

```json
{
  "temporal": {
    "enabled": true,
    "historySource": "preLayerInput",
    "historyLength": 12
  }
}
```

Supported `historySource` values:

- `source`: decoded source-video history from previous frames.
- `preLayerInput`: history of the input arriving at this layer before the shader runs.

`historyLength` is the requested frame count. The runtime clamps it by `maxTemporalHistoryFrames` in `config/runtime-host.json`.

Temporal history resets when:

- layers are added, removed, or reordered
- a layer bypass state changes
- a layer changes shader
- a shader is reloaded or recompiled
- render dimensions change

Use the available history lengths to avoid assuming history is ready on the first frame:

```slang
float4 shadeVideo(ShaderContext context)
{
    if (context.temporalHistoryLength <= 0)
        return context.sourceColor;

    float4 oldFrame = sampleTemporalHistory(3, context.uv);
    return lerp(context.sourceColor, oldFrame, 0.4);
}
```

See `shaders/temporal-ghost-trail/` and `shaders/temporal-low-fps/` for examples.

## Coordinate And Color Notes

- `uv` is normalized.
- Use `context.outputResolution` for pixel-sized effects.
- Use `context.inputResolution` when sampling source video by input pixel size.
- `sourceColor` and `sampleVideo` return RGBA values in normalized `0..1` range.
- Prefer `saturate(color)` or explicit `clamp` before returning if your math can overshoot.
- For generated calibration charts, test patterns, gradients, and exposure ramps, state whether patch values are linear-light, display-referred gamma encoded, Rec.709 encoded, or intentionally artistic.
- For one-stop exposure patches, each patch should normally be `baseLevel * 2^patchIndex` before any display/tone encoding.
- For Rec.709 OETF encoding, use:

```slang
float rec709Oetf(float linearLevel)
{
    float value = saturate(linearLevel);
    if (value < 0.018)
        return 4.5 * value;
    return 1.099 * pow(value, 0.45) - 0.099;
}
```

Pixel-size example:

```slang
float2 pixel = 1.0 / max(context.outputResolution, float2(1.0));
float4 right = sampleVideo(context.uv + float2(pixel.x, 0.0));
```

## Animation And Timing Notes

- `context.time` is elapsed runtime time in seconds and is the default animation source for generative shaders.
- `context.frameCount` increments once per rendered output frame and is useful when an effect must be frame-locked.
- Avoid expensive CPU-like timing logic in the shader; animation should usually be a simple function of `context.time`, `context.frameCount`, trigger uniforms, or parameters.
- If a shader appears to judder only while animated, first test whether freezing its time removes the issue. That usually separates animation cadence issues from rendering or transfer issues.
- Do not add custom timer uniforms to the wrapper. Use the fields already in `ShaderContext`.

## Performance Notes

The app has to meet a fixed video frame cadence, so avoid shader code that only looks good in unconstrained browser demos.

Guidelines:

- Keep loops bounded with compile-time constants where possible.
- Avoid very high per-pixel raymarch counts by default. If a heavy loop is needed, expose a quality/steps control with a safe default.
- Prefer early exits only when they are simple; highly divergent branches can be expensive across a full frame.
- Avoid repeated texture sampling in large loops unless the effect really needs it.
- Use `context.outputResolution` carefully. A 1080p frame is over 2 million fragments; a tiny extra loop can become expensive.
- The UI render time may measure CPU command submission rather than true GPU execution time, so visual frame issues can still be GPU-related even when reported render time is small.
- Do not write debug files, allocate resources, or assume CPU-side work can happen from `shader.slang`. Shader code is GPU-only.

## Reload And Generated Files

When a shader compiles, the runtime writes generated files under `runtime/shader_cache/`:

- `active_shader_wrapper.slang`
- `active_shader.raw.frag`
- `active_shader.frag`

These files are ignored by git and are useful for debugging compiler output. If a shader fails to compile, inspect the wrapper first; it shows the exact generated Slang code including your included shader.

## Common Pitfalls

- Do not use hyphens in parameter IDs, texture IDs, or entry point names.
- Do not declare your own `ShaderContext`, `GlobalParams`, `sampleVideo`, `sampleSourceHistory`, or `sampleTemporalHistory`.
- Do not write a `[shader("fragment")]` entry point in `shader.slang`; the runtime provides it.
- Remember enum globals are integer indexes, not strings.
- Declare every texture in `shader.json`; undeclared texture samplers will not be bound.
- Declare packaged fonts in `shader.json` when text parameters should use a specific font.
- Keep temporal history requests modest. They consume texture units and memory and are capped by runtime config.
- If a parameter appears in the UI but not in Slang, the shader may still compile, but the control has no effect.
- If a Slang name collides with a generated global, rename your parameter or local symbol.

## Minimal Package Checklist

Before committing a new shader package:

- `shader.json` is valid JSON.
- `id` is unique across `shaders/`.
- `entryPoint`, parameter IDs, and texture IDs are valid identifiers.
- `shader.slang` implements the configured entry point.
- Texture files referenced by `textures` exist.
- Font files referenced by `fonts` exist.
- Enum defaults are present in their `options`.
- Temporal shaders handle short or empty history gracefully.
- The app can reload and compile the shader without errors.