aiden/video-shader-toys
1
0
Fork 1
Code Issues Pull Requests Actions Releases 5 Wiki Activity

added optional web component UI control
All checks were successful
CI / React UI Build (push) Successful in 12s
Details
CI / Native Windows Build And Tests (push) Successful in 2m20s
Details
CI / Windows Release Package (push) Successful in 2m56s
Details

Browse Source
This commit is contained in:
Aiden
2026-05-30 22:57:59 +10:00
parent a6d2ee385e
commit 27690c3afa
26 changed files with 804 additions and 76 deletions
Download Patch File Download Diff File Expand all files Collapse all files

77
README.md
View File

@@ -13,7 +13,8 @@ The app loads shader packages from `shaders/`, compiles Slang to GLSL at runtime
- `runtime/templates/`: tracked shader wrapper templates.
- `runtime/`: ignored generated runtime cache/state output. See `runtime/README.md`.
- `tests/`: focused native tests for pure runtime logic.
- `docs/FORKING_RENDER_CADENCE_BASE.md`: notes for forking the cadence/video I/O base while replacing the GPU-rendered content.
- `docs/CURRENT_SYSTEM_ARCHITECTURE.md`: current architecture notes for the cadence/video I/O host.
- `docs/RENDER_CADENCE_GOLDEN_RULES.md`: guardrails for changes that touch render cadence, runtime work, and video I/O.
- `.gitea/workflows/ci.yml`: Gitea Actions CI for Windows native tests and Ubuntu UI build.
Native app internals are grouped by boundary:
@@ -128,6 +129,7 @@ The control UI provides:
- A searchable shader library for adding layers.
- Compact parameter rows with inline descriptions and intended OSC route copy controls.
- Shader-declared custom Web Component control panels with default-control fallback.
- Manual shader reload.
- Host config editing, save, restart request, and NDI input source discovery.
- Compact video I/O and render cadence status.
@@ -313,7 +315,19 @@ shaders/<id>/
optional-font-or-texture-assets
```
See `SHADER_CONTRACT.md` for the manifest schema, parameter types, texture assets, font/text assets, temporal history support, optional render-pass declarations, and the Slang entry point contract. `shaders/text-overlay/` is the reference live text package and bundles Roboto Regular with its OFL license. Broken shader packages are shown as unavailable in the selector with their error text instead of preventing the app from launching.
See `SHADER_CONTRACT.md` for the manifest schema, parameter types, texture assets, font/text assets, temporal history support, optional render-pass declarations, optional custom UI metadata, and the Slang entry point contract. `shaders/text-overlay/` is the reference live text package and bundles Roboto Regular with its OFL license. Broken shader packages are shown as unavailable in the selector with their error text instead of preventing the app from launching.
Shader packages can optionally declare a custom control panel:
```json
"ui": {
"type": "webComponent",
"entry": "ui/controls.js",
"tag": "my-shader-controls"
}
```
The host validates that metadata, exposes it in `/api/state`, and serves the module from `/shader-assets/{shaderId}/...`. Custom controls receive the current layer, declared parameters, `setParameter(id, value)`, and `requestReset()`. They still update the same manifest-declared parameters as the default React controls.
## Generated Files
@@ -374,62 +388,3 @@ If these variables are not set, CMake first looks under the private `video-io-3r
- Add Aja input and output (Assuming i can get a hold of an aja card)
- Add bluefish input and output (Assuming again card acess)
- Endpoint to show OSC paths seperatly instead of a part of the control UI
## Custom shader UI
Extend the shader manifest contract
Add optional UI metadata:
"ui": {
"type": "webComponent",
"entry": "ui/controls.js",
"tag": "my-shader-controls"
}
Keep this optional. No custom UI means current default controls.
Expose UI metadata in /api/state
Add the parsed ui block to each shader/layer summary so the React app knows whether a layer has a custom control panel.
Serve shader package UI assets safely
Add a route like:
/shader-assets/{shaderId}/ui/controls.js
It should only serve files inside that shader package folder.
Add a React host component
Create something like ShaderCustomPanel.jsx that:
dynamically imports/registers the custom element
passes layer, parameters, and setParameter
catches load/render failures
falls back to the normal ParameterField grid
Define the custom element API
Keep it small and stable:
element.layer = layer;
element.parameters = layer.parameters;
element.setParameter = (id, value) => {};
element.requestReset = () => {};
Custom UI should never bypass manifest validation.
Add fallback and escape hatch
Even if custom UI loads, provide a “Default controls” toggle per layer. That is the life raft.
Add tests
Backend:
manifest parser accepts valid UI blocks
rejects unsafe paths like ../
/api/state includes UI metadata
asset route refuses files outside the shader package
Frontend:
falls back when custom component fails
calls /api/layers/update-parameter through the same path as default controls
Document the contract
Add a section to shaders/SHADER_CONTRACT.md with:
manifest example
custom element lifecycle
available properties/functions
rule: all controls must map to declared parameters

3
docs/CURRENT_SYSTEM_ARCHITECTURE.md
View File

@@ -77,6 +77,7 @@ The checked-in implementation is `ShaderRuntimeContentController`. It wraps `Run
- build state and message
- bypass state
- manifest parameter definitions
- optional shader-declared custom UI metadata
- current parameter values
- render-ready artifacts
@@ -111,6 +112,7 @@ The host configuration editor is separate from runtime layer persistence. The UI
- rescan `shaders/`
- re-read manifests
- rebuild the supported shader catalog
- refresh optional shader custom UI metadata
- update active layer metadata and parameter definitions from changed manifests
- preserve compatible parameter values
- default new or incompatible parameter values
@@ -155,6 +157,7 @@ Screenshot routes are present in the UI/OpenAPI surface but are not implemented
The HTTP server runs on its own thread. `HttpControlServer` owns socket lifetime, HTTP parsing, static asset helpers, OpenAPI/Swagger helper serving, and WebSocket state transport. `RenderCadenceHttpRoutes` owns this app's current endpoint map:
- UI assets
- shader package custom UI assets under `/shader-assets/{shaderId}/...`
- OpenAPI/Swagger docs
- `GET /api/state`
- `/ws` state updates

71
docs/openapi.yaml
View File

@@ -106,6 +106,51 @@ paths:
text/plain:
schema:
type: string
/shader-assets/{shaderId}/{assetPath}:
get:
tags: [Static]
summary: Serve a shader package UI asset
description: Serves custom shader UI files declared by `shader.json`. Assets are resolved through the active runtime-content controller and must stay inside that shader package's `ui/` directory.
operationId: getShaderPackageAsset
parameters:
- name: shaderId
in: path
required: true
description: Shader package id.
schema:
type: string
- name: assetPath
in: path
required: true
description: Relative asset path below the shader package, usually `ui/controls.js`.
schema:
type: string
responses:
"200":
description: Shader package UI asset.
content:
text/javascript:
schema:
type: string
text/css:
schema:
type: string
image/svg+xml:
schema:
type: string
image/png:
schema:
type: string
format: binary
text/plain:
schema:
type: string
"404":
description: Asset was not found, the shader did not declare custom UI, or the path was unsafe.
content:
text/plain:
schema:
type: string
/docs:
get:
tags: [Docs]
@@ -1200,6 +1245,29 @@ components:
$ref: "#/components/schemas/TemporalState"
feedback:
$ref: "#/components/schemas/FeedbackState"
ui:
$ref: "#/components/schemas/ShaderUiDefinition"
nullable: true
ShaderUiDefinition:
type: object
nullable: true
properties:
type:
type: string
enum: [webComponent]
entry:
type: string
description: Package-relative JavaScript module path from the shader manifest.
example: ui/controls.js
tag:
type: string
description: Custom element tag registered by the module.
example: my-shader-controls
assetUrl:
type: string
description: HTTP URL for loading the JavaScript module from the local control server.
example: /shader-assets/my-shader/ui/controls.js
additionalProperties: false
TemporalState:
type: object
properties:
@@ -1232,6 +1300,9 @@ components:
type: boolean
temporal:
$ref: "#/components/schemas/TemporalState"
ui:
$ref: "#/components/schemas/ShaderUiDefinition"
nullable: true
parameters:
type: array
items:

74
shaders/SHADER_CONTRACT.md
View File

@@ -102,6 +102,7 @@ Optional fields:
- `fonts`: packaged font assets for live text parameters.
- `temporal`: history-buffer requirements.
- `feedback`: optional previous-frame shader-local feedback surface.
- `ui`: optional custom control UI module for this shader.
Parameter objects may also include an optional `description` string. The control UI displays it as one-line helper text with the full text available on hover, so use it for short operational guidance rather than long documentation.
@@ -121,6 +122,78 @@ Shader-visible identifiers must be valid Slang-style identifiers:
Use letters, numbers, and underscores only, and start with a letter or underscore. For example, `logoTexture` is valid; `logo-texture` is not valid as a shader-visible texture ID.
## Custom Control UI
Shaders may optionally declare a custom browser control panel implemented as a standard Web Component. This only changes the bundled React control surface; it does not change shader validation, parameter storage, render cadence, or video I/O behavior.
Manifest example:
```json
{
"ui": {
"type": "webComponent",
"entry": "ui/controls.js",
"tag": "my-shader-controls"
}
}
```
The `entry` file is loaded as a JavaScript module from:
```text
/shader-assets/{shaderId}/ui/controls.js
```
Rules:
- `type` must be `webComponent`.
- `entry` must be a safe relative `.js` or `.mjs` path inside the shader package. Use the package `ui/` directory for UI assets.
- `tag` must be a valid custom element name with a hyphen, for example `my-shader-controls`.
- The `entry` file must exist when the manifest is loaded.
- Custom UI controls must update declared manifest parameters. They cannot create hidden runtime parameters or bypass host validation.
- The React UI still provides a **Default controls** fallback for each layer.
Custom element API:
```js
class MyShaderControls extends HTMLElement {
set layer(value) {}
set parameters(value) {}
set values(value) {}
connectedCallback() {}
}
customElements.define("my-shader-controls", MyShaderControls);
```
The host sets these properties whenever layer state changes:
- `layer`: full layer object from `/api/state`.
- `parameters`: array of manifest parameter definitions with current `value`.
- `values`: object keyed by parameter id.
- `setParameter(id, value)`: updates a declared layer parameter through the same route used by the default controls.
- `requestReset()`: resets the layer parameters through the normal reset route.
A custom element may either call the functions directly:
```js
this.setParameter("strength", 0.75);
this.requestReset();
```
or dispatch events:
```js
this.dispatchEvent(new CustomEvent("shader-parameter-change", {
detail: { parameterId: "strength", value: 0.75 }
}));
this.dispatchEvent(new CustomEvent("shader-reset-parameters"));
```
When the host updates the element's properties, it also dispatches `shader-layer-update` with `{ layer, parameters, values }` in `event.detail`.
## Render Passes
Most shaders should omit `passes`. The runtime then creates one implicit pass:
@@ -852,5 +925,6 @@ Before committing a new shader package:
- Font files referenced by `fonts` exist.
- Enum defaults are present in their `options`.
- Text `fontParameter` selectors reference valid font assets through their enum options.
- Custom UI entries, when present, load from a safe package-relative JavaScript module and register the declared web component tag.
- Temporal shaders handle short or empty history gracefully.
- The app can reload and compile the shader without errors.

5
src/README.md
View File

@@ -252,6 +252,7 @@ The app starts a local HTTP control server on `http://127.0.0.1:8080` by default
Current endpoints:
- `GET /` and UI asset paths: serve the bundled control UI from `ui/dist`
- `GET /shader-assets/{shaderId}/...`: serves validated shader package custom UI assets from the active runtime-content controller
- `GET /api/state`: returns OpenAPI-shaped display data with cadence telemetry, supported shaders, output status, and a read-only current runtime layer
- `GET /api/config`: returns active and saved startup host config
- `GET /api/ndi/sources`: returns currently discoverable NDI source names for host-config input selection
@@ -270,7 +271,7 @@ The HTTP server runs on its own thread. `control/http/HttpControlServer` owns so
`app/RenderCadenceApp` owns startup, video output, preview, telemetry, OSC status, and HTTP server lifetime. It does not own the Slang shader stack directly. Runtime content is supplied through `IRuntimeContentController` in `app/RuntimeContentController.h`.
The default implementation is `ShaderRuntimeContentController`. It wraps `RuntimeLayerController`, exposes shader catalog/layer state to `/api/state`, handles the shader layer POST commands, and publishes render-ready shader layers to the current render thread. A fork that uses a D3D engine should replace this controller with a D3D engine controller and decide separately whether to keep the current GL render/readback thread as a bridge or replace it with a D3D-native frame publisher.
The default implementation is `ShaderRuntimeContentController`. It wraps `RuntimeLayerController`, exposes shader catalog/layer state and optional shader custom UI metadata to `/api/state`, resolves shader package UI assets for `/shader-assets/{shaderId}/...`, handles the shader layer POST commands, and publishes render-ready shader layers to the current render thread. A fork that uses a D3D engine should replace this controller with a D3D engine controller and decide separately whether to keep the current GL render/readback thread as a bridge or replace it with a D3D-native frame publisher.
## Optional DeckLink Output
@@ -407,7 +408,7 @@ Shader source semantics:
The `/api/state` shader list uses the same support rules as runtime shader compilation and reports only packages this app can run today. Unsupported manifest feature sets such as temporal, feedback, and texture-backed shaders are hidden from the control UI for now.
Runtime shaders are exposed through `RuntimeLayerModel` as display layers with manifest parameter definitions, current parameter values, build status, and render-ready artifacts. POST controls mutate this app-owned model and may start background shader builds when the selected shader changes or when `/api/reload` is requested. Durable UI/API layer changes request a debounced background write to `runtime/runtime_state.json`. OSC ingress is not wired yet; when it is added, OSC-driven changes should stay out of autosave unless an explicit persistence policy is introduced.
Runtime shaders are exposed through `RuntimeLayerModel` as display layers with manifest parameter definitions, optional custom UI metadata, current parameter values, build status, and render-ready artifacts. POST controls mutate this app-owned model and may start background shader builds when the selected shader changes or when `/api/reload` is requested. Durable UI/API layer changes request a debounced background write to `runtime/runtime_state.json`. OSC ingress is not wired yet; when it is added, OSC-driven changes should stay out of autosave unless an explicit persistence policy is introduced.
When a layer becomes render-ready, the app publishes the ready render-layer snapshot to the render thread. The render thread forwards those snapshots to `RuntimeShaderRenderContent`, the default `IRenderContent` implementation. That adapter owns the GL-side `RuntimeRenderScene`, diffs snapshots at a frame boundary, queues new or changed pass programs to the shared-context prepare worker, swaps in a full prepared render plan only after every pass is ready, removes obsolete GL programs, and renders ready layers in order. Stacked stateless full-frame shaders render through internal ping-pong targets so each layer can sample the previous layer through `gLayerInput`; multipass shaders route named intermediate outputs through their manifest-declared pass inputs, and the final ready layer renders to the output target.

12
src/app/RenderCadenceApp.h
View File

@@ -259,6 +259,18 @@ private:
routeCallbacks.getNdiSourcesJson = [this]() {
return BuildNdiSourcesJson();
};
routeCallbacks.resolveShaderAssetPath = [this](
const std::string& shaderId,
const std::string& assetPath,
std::filesystem::path& resolvedPath,
std::string& assetError) {
if (!mRuntimeContent)
{
assetError = "No runtime content controller is active.";
return false;
}
return mRuntimeContent->ResolveAssetPath(shaderId, assetPath, resolvedPath, assetError);
};
routeCallbacks.executePost = [this](const std::string& path, const std::string& body) {
if (path == "/api/config/save")
return HandleConfigSave(body);

32
src/app/RenderCadenceHttpRoutes.cpp
View File

@@ -35,11 +35,43 @@ std::string ActionResponse(bool ok, const std::string& error = std::string())
return writer.StringValue();
}
bool SplitShaderAssetPath(const std::string& path, std::string& shaderId, std::string& assetPath)
{
const std::string prefix = "/shader-assets/";
if (path.rfind(prefix, 0) != 0)
return false;
const std::string remaining = path.substr(prefix.size());
const std::size_t separator = remaining.find('/');
if (separator == std::string::npos || separator == 0 || separator + 1 >= remaining.size())
return false;
shaderId = remaining.substr(0, separator);
assetPath = remaining.substr(separator + 1);
return true;
}
HttpResponse ServeRenderCadenceGet(
const HttpRequest& request,
const HttpControlServer& server,
const RenderCadenceHttpRouteCallbacks& callbacks)
{
std::string shaderId;
std::string shaderAssetPath;
if (SplitShaderAssetPath(request.path, shaderId, shaderAssetPath))
{
if (!callbacks.resolveShaderAssetPath)
return HttpControlServer::TextResponse("404 Not Found", "Not Found");
std::filesystem::path resolvedPath;
std::string error;
if (!callbacks.resolveShaderAssetPath(shaderId, shaderAssetPath, resolvedPath, error))
return HttpControlServer::TextResponse("404 Not Found", "Not Found");
return server.ServeFile(resolvedPath);
}
if (request.path.rfind("/shader-assets/", 0) == 0)
return HttpControlServer::TextResponse("404 Not Found", "Not Found");
if (request.path == "/api/state")
return HttpControlServer::JsonResponse("200 OK", callbacks.getStateJson ? callbacks.getStateJson() : "{}");
if (request.path == "/api/config")

2
src/app/RenderCadenceHttpRoutes.h
View File

@@ -3,6 +3,7 @@
#include "../control/ControlActionResult.h"
#include "../control/http/HttpControlServer.h"
#include <filesystem>
#include <functional>
#include <string>
@@ -13,6 +14,7 @@ struct RenderCadenceHttpRouteCallbacks
std::function<std::string()> getStateJson;
std::function<std::string()> getConfigJson;
std::function<std::string()> getNdiSourcesJson;
std::function<bool(const std::string&, const std::string&, std::filesystem::path&, std::string&)> resolveShaderAssetPath;
std::function<ControlActionResult(const std::string&)> addLayer;
std::function<ControlActionResult(const std::string&)> removeLayer;
std::function<ControlActionResult(const std::string&, const std::string&)> executePost;

13
src/app/RuntimeContentController.h
View File

@@ -5,6 +5,7 @@
#include "../json/JsonWriter.h"
#include "../telemetry/CadenceTelemetry.h"
#include <filesystem>
#include <string>
namespace RenderCadenceCompositor
@@ -22,5 +23,17 @@ public:
virtual void WriteRuntimeJson(JsonWriter& writer, const CadenceTelemetrySnapshot& telemetry) const = 0;
virtual void WriteCatalogJson(JsonWriter& writer) const = 0;
virtual void WriteLayersJson(JsonWriter& writer, const CadenceTelemetrySnapshot& telemetry) const = 0;
virtual bool ResolveAssetPath(
const std::string& shaderId,
const std::string& assetPath,
std::filesystem::path& resolvedPath,
std::string& error) const
{
(void)shaderId;
(void)assetPath;
resolvedPath.clear();
error = "Runtime content assets are not available.";
return false;
}
};
}

103
src/app/ShaderRuntimeContentController.cpp
View File

@@ -7,6 +7,68 @@
namespace RenderCadenceCompositor
{
namespace
{
bool IsSafeShaderAssetPath(const std::string& assetPath, std::filesystem::path& normalizedPath)
{
if (assetPath.empty() || assetPath.find('\\') != std::string::npos ||
assetPath.find(':') != std::string::npos || assetPath.find('?') != std::string::npos ||
assetPath.find('#') != std::string::npos)
return false;
const std::filesystem::path path(assetPath);
if (path.empty() || path.is_absolute())
return false;
bool firstPart = true;
bool startsInUiDirectory = false;
for (const std::filesystem::path& part : path)
{
if (part.empty() || part == "." || part == "..")
return false;
if (firstPart)
{
startsInUiDirectory = part == "ui";
firstPart = false;
}
}
if (!startsInUiDirectory)
return false;
normalizedPath = path.lexically_normal();
if (normalizedPath.empty() || normalizedPath.is_absolute())
return false;
for (const std::filesystem::path& part : normalizedPath)
{
if (part.empty() || part == "." || part == "..")
return false;
}
return true;
}
bool IsPathUnderRoot(const std::filesystem::path& root, const std::filesystem::path& path)
{
std::error_code errorCode;
const std::filesystem::path canonicalRoot = std::filesystem::weakly_canonical(root, errorCode);
if (errorCode)
return false;
const std::filesystem::path canonicalPath = std::filesystem::weakly_canonical(path, errorCode);
if (errorCode)
return false;
const std::filesystem::path relative = canonicalPath.lexically_relative(canonicalRoot);
if (relative.empty() || relative.is_absolute())
return false;
for (const std::filesystem::path& part : relative)
{
if (part == "..")
return false;
}
return true;
}
}
ShaderRuntimeContentController::ShaderRuntimeContentController(RenderLayerPublisher publisher) :
mRuntimeLayers(std::move(publisher))
{
@@ -64,4 +126,45 @@ void ShaderRuntimeContentController::WriteLayersJson(JsonWriter& writer, const C
{
WriteRuntimeShaderLayersJson(writer, mRuntimeLayers.ShaderCatalog(), mRuntimeLayers.Snapshot(telemetry));
}
bool ShaderRuntimeContentController::ResolveAssetPath(
const std::string& shaderId,
const std::string& assetPath,
std::filesystem::path& resolvedPath,
std::string& error) const
{
const ShaderPackage* shaderPackage = mRuntimeLayers.ShaderCatalog().FindPackage(shaderId);
if (!shaderPackage)
{
error = "Shader package not found.";
return false;
}
if (!shaderPackage->ui.enabled)
{
error = "Shader package does not declare a custom UI.";
return false;
}
std::filesystem::path normalizedPath;
if (!IsSafeShaderAssetPath(assetPath, normalizedPath))
{
error = "Shader asset path is not safe.";
return false;
}
const std::filesystem::path candidatePath = shaderPackage->directoryPath / normalizedPath;
if (!IsPathUnderRoot(shaderPackage->directoryPath, candidatePath))
{
error = "Shader asset path escaped the package directory.";
return false;
}
if (!std::filesystem::exists(candidatePath) || !std::filesystem::is_regular_file(candidatePath))
{
error = "Shader asset was not found.";
return false;
}
resolvedPath = candidatePath;
return true;
}
}

5
src/app/ShaderRuntimeContentController.h
View File

@@ -26,6 +26,11 @@ public:
void WriteRuntimeJson(JsonWriter& writer, const CadenceTelemetrySnapshot& telemetry) const override;
void WriteCatalogJson(JsonWriter& writer) const override;
void WriteLayersJson(JsonWriter& writer, const CadenceTelemetrySnapshot& telemetry) const override;
bool ResolveAssetPath(
const std::string& shaderId,
const std::string& assetPath,
std::filesystem::path& resolvedPath,
std::string& error) const override;
private:
RuntimeLayerController mRuntimeLayers;

28
src/control/RuntimeStateJson.h
View File

@@ -181,6 +181,27 @@ inline void WriteFeedbackJson(JsonWriter& writer, const FeedbackSettings& feedba
writer.EndObject();
}
inline std::string ShaderAssetUrl(const std::string& shaderId, const std::string& assetPath)
{
return "/shader-assets/" + shaderId + "/" + assetPath;
}
inline void WriteShaderUiJson(JsonWriter& writer, const std::string& shaderId, const ShaderUiDefinition& ui)
{
if (!ui.enabled)
{
writer.Null();
return;
}
writer.BeginObject();
writer.KeyString("type", ui.type);
writer.KeyString("entry", ui.entryPath);
writer.KeyString("tag", ui.customElementTag);
writer.KeyString("assetUrl", ShaderAssetUrl(shaderId, ui.entryPath));
writer.EndObject();
}
inline const char* RuntimeLayerBuildStateName(RuntimeLayerBuildState state)
{
switch (state)
@@ -276,6 +297,8 @@ inline void WriteShaderCatalogJson(JsonWriter& writer, const SupportedShaderCata
writer.KeyString("category", shader.category);
writer.KeyBool("available", true);
writer.KeyNull("error");
writer.Key("ui");
WriteShaderUiJson(writer, shader.id, shader.ui);
writer.EndObject();
}
writer.EndArray();
@@ -304,6 +327,11 @@ inline void WriteRuntimeShaderLayersJson(
writer.KeyString("buildState", RuntimeLayerBuildStateName(layer.buildState));
writer.KeyBool("renderReady", layer.renderReady);
writer.KeyString("message", layer.message);
writer.Key("ui");
if (shaderPackage)
WriteShaderUiJson(writer, shaderPackage->id, shaderPackage->ui);
else
writer.Null();
writer.Key("temporal");
if (shaderPackage)
WriteTemporalJson(writer, shaderPackage->temporal);

1
src/control/http/HttpControlServer.h
View File

@@ -95,6 +95,7 @@ public:
HttpResponse ServeOpenApiSpec() const;
HttpResponse ServeSwaggerDocs() const;
HttpResponse ServeUiAsset(const std::string& relativePath) const;
HttpResponse ServeFile(const std::filesystem::path& path) const;
static HttpResponse JsonResponse(const std::string& status, const std::string& body);
static HttpResponse TextResponse(const std::string& status, const std::string& body);

8
src/control/http/HttpControlServerRoutes.cpp
View File

@@ -45,6 +45,14 @@ HttpControlServer::HttpResponse HttpControlServer::ServeUiAsset(const std::strin
return { "200 OK", GuessContentType(path), body };
}
HttpControlServer::HttpResponse HttpControlServer::ServeFile(const std::filesystem::path& path) const
{
const std::string body = LoadTextFile(path);
if (body.empty())
return TextResponse("404 Not Found", "Not Found");
return { "200 OK", GuessContentType(path), body };
}
std::string HttpControlServer::LoadTextFile(const std::filesystem::path& path) const
{
std::ifstream input(path, std::ios::binary);

1
src/runtime/catalog/SupportedShaderCatalog.cpp
View File

@@ -193,6 +193,7 @@ bool SupportedShaderCatalog::Load(const std::filesystem::path& shaderRoot, unsig
summary.name = shaderPackage.displayName.empty() ? shaderPackage.id : shaderPackage.displayName;
summary.description = shaderPackage.description;
summary.category = shaderPackage.category;
summary.ui = shaderPackage.ui;
mShaders.push_back(std::move(summary));
mPackagesById[shaderPackage.id] = shaderPackage;
}

1
src/runtime/catalog/SupportedShaderCatalog.h
View File

@@ -16,6 +16,7 @@ struct SupportedShaderSummary
std::string name;
std::string description;
std::string category;
ShaderUiDefinition ui;
};
struct ShaderSupportResult

106
src/shader/ShaderManifestParser.cpp
View File

@@ -46,6 +46,66 @@ bool ParseTemporalHistorySource(const std::string& sourceName, TemporalHistorySo
}
return false;
}
bool IsSafeUiEntryPath(const std::string& entryPath)
{
if (Trim(entryPath).empty())
return false;
if (entryPath.find('\\') != std::string::npos || entryPath.find(':') != std::string::npos ||
entryPath.find('?') != std::string::npos || entryPath.find('#') != std::string::npos)
return false;
const std::filesystem::path path(entryPath);
if (path.empty() || path.is_absolute())
return false;
bool firstPart = true;
bool startsInUiDirectory = false;
for (const std::filesystem::path& part : path)
{
if (part.empty() || part == "." || part == "..")
return false;
if (firstPart)
{
startsInUiDirectory = part == "ui";
firstPart = false;
}
}
if (!startsInUiDirectory)
return false;
const std::filesystem::path normalized = path.lexically_normal();
if (normalized.empty() || normalized.is_absolute())
return false;
for (const std::filesystem::path& part : normalized)
{
if (part.empty() || part == "." || part == "..")
return false;
}
const std::string extension = normalized.extension().string();
return extension == ".js" || extension == ".mjs";
}
bool IsValidCustomElementTag(const std::string& tag)
{
if (tag.empty() || tag.find('-') == std::string::npos || tag.front() == '-' || tag.back() == '-')
return false;
const unsigned char first = static_cast<unsigned char>(tag.front());
if (first < 'a' || first > 'z')
return false;
for (char ch : tag)
{
const unsigned char value = static_cast<unsigned char>(ch);
if ((value >= 'a' && value <= 'z') || (value >= '0' && value <= '9') || value == '-')
continue;
return false;
}
return true;
}
}
std::string ManifestPathMessage(const std::filesystem::path& manifestPath)
@@ -367,4 +427,50 @@ bool ParseFeedbackSettings(const JsonValue& manifestJson, ShaderPackage& shaderP
return true;
}
bool ParseUiDefinition(const JsonValue& manifestJson, ShaderPackage& shaderPackage, const std::filesystem::path& manifestPath, std::string& error)
{
const JsonValue* uiValue = nullptr;
if (!OptionalObjectField(manifestJson, "ui", uiValue, manifestPath, error))
return false;
if (!uiValue)
return true;
ShaderUiDefinition ui;
if (!RequireNonEmptyStringField(*uiValue, "type", ui.type, manifestPath, error) ||
!RequireNonEmptyStringField(*uiValue, "entry", ui.entryPath, manifestPath, error) ||
!RequireNonEmptyStringField(*uiValue, "tag", ui.customElementTag, manifestPath, error))
{
error = "Shader UI definition is missing required 'type', 'entry', or 'tag' in: " + ManifestPathMessage(manifestPath);
return false;
}
if (ui.type != "webComponent")
{
error = "Shader UI type must be 'webComponent' in: " + ManifestPathMessage(manifestPath);
return false;
}
if (!IsSafeUiEntryPath(ui.entryPath))
{
error = "Shader UI entry must be a safe relative .js or .mjs path under ui/ in: " + ManifestPathMessage(manifestPath);
return false;
}
if (!IsValidCustomElementTag(ui.customElementTag))
{
error = "Shader UI tag must be a valid custom element name with a hyphen in: " + ManifestPathMessage(manifestPath);
return false;
}
const std::filesystem::path entryPath = shaderPackage.directoryPath / std::filesystem::path(ui.entryPath);
if (!std::filesystem::exists(entryPath))
{
error = "Shader UI entry not found for package " + shaderPackage.id + ": " + entryPath.string();
return false;
}
ui.entryPath = std::filesystem::path(ui.entryPath).lexically_normal().generic_string();
ui.enabled = true;
shaderPackage.ui = ui;
return true;
}
}

1
src/shader/ShaderManifestParser.h
View File

@@ -25,6 +25,7 @@ bool ParseShaderMetadata(const JsonValue& manifestJson, ShaderPackage& shaderPac
bool ParsePassDefinitions(const JsonValue& manifestJson, ShaderPackage& shaderPackage, const std::filesystem::path& manifestPath, std::string& error);
bool ParseTemporalSettings(const JsonValue& manifestJson, ShaderPackage& shaderPackage, unsigned maxTemporalHistoryFrames, const std::filesystem::path& manifestPath, std::string& error);
bool ParseFeedbackSettings(const JsonValue& manifestJson, ShaderPackage& shaderPackage, const std::filesystem::path& manifestPath, std::string& error);
bool ParseUiDefinition(const JsonValue& manifestJson, ShaderPackage& shaderPackage, const std::filesystem::path& manifestPath, std::string& error);
bool ParseTextureAssets(const JsonValue& manifestJson, ShaderPackage& shaderPackage, const std::filesystem::path& manifestPath, std::string& error);
bool ParseFontAssets(const JsonValue& manifestJson, ShaderPackage& shaderPackage, const std::filesystem::path& manifestPath, std::string& error);
bool ParseParameterDefinitions(const JsonValue& manifestJson, ShaderPackage& shaderPackage, const std::filesystem::path& manifestPath, std::string& error);

1
src/shader/ShaderPackageRegistry.cpp
View File

@@ -146,5 +146,6 @@ bool ShaderPackageRegistry::ParseManifest(const std::filesystem::path& manifestP
ParseFontAssets(manifestJson, shaderPackage, manifestPath, error) &&
ParseTemporalSettings(manifestJson, shaderPackage, mMaxTemporalHistoryFrames, manifestPath, error) &&
ParseFeedbackSettings(manifestJson, shaderPackage, manifestPath, error) &&
ParseUiDefinition(manifestJson, shaderPackage, manifestPath, error) &&
ParseParameterDefinitions(manifestJson, shaderPackage, manifestPath, error);
}

9
src/shader/ShaderTypes.h
View File

@@ -70,6 +70,14 @@ struct FeedbackSettings
std::string writePassId;
};
struct ShaderUiDefinition
{
bool enabled = false;
std::string type;
std::string entryPath;
std::string customElementTag;
};
struct ShaderTextureAsset
{
std::string id;
@@ -118,6 +126,7 @@ struct ShaderPackage
std::vector<ShaderFontAsset> fontAssets;
TemporalSettings temporal;
FeedbackSettings feedback;
ShaderUiDefinition ui;
std::filesystem::file_time_type shaderWriteTime;
std::filesystem::file_time_type manifestWriteTime;
};

49
tests/RenderCadenceCompositorHttpControlServerTests.cpp
View File

@@ -157,6 +157,53 @@ void TestRootServesUiIndex()
std::filesystem::remove_all(root);
}
void TestShaderAssetEndpointUsesCallback()
{
using namespace RenderCadenceCompositor;
const std::filesystem::path root = std::filesystem::temp_directory_path() / "render-cadence-compositor-shader-asset-test";
std::filesystem::create_directories(root / "ui");
const std::filesystem::path assetPath = root / "ui" / "controls.js";
{
std::ofstream output(assetPath, std::ios::binary);
output << "customElements.define('solid-controls', class extends HTMLElement {});";
}
HttpControlServer server;
RenderCadenceHttpRouteCallbacks callbacks;
callbacks.resolveShaderAssetPath = [&assetPath](const std::string& shaderId, const std::string& relativePath, std::filesystem::path& resolvedPath, std::string&) {
ExpectEquals(shaderId, "solid", "shader asset callback receives shader id");
ExpectEquals(relativePath, "ui/controls.js", "shader asset callback receives package-relative asset path");
resolvedPath = assetPath;
return true;
};
HttpControlServer::HttpRequest request;
request.method = "GET";
request.path = "/shader-assets/solid/ui/controls.js";
const HttpControlServer::HttpResponse response = RouteRenderCadenceHttpRequest(request, server, callbacks);
ExpectEquals(response.status, "200 OK", "shader asset endpoint serves resolved asset");
ExpectEquals(response.contentType, "text/javascript", "shader asset endpoint guesses javascript content type");
Expect(response.body.find("solid-controls") != std::string::npos, "shader asset endpoint returns asset body");
std::filesystem::remove_all(root);
}
void TestIncompleteShaderAssetEndpointReturns404()
{
using namespace RenderCadenceCompositor;
HttpControlServer server;
HttpControlServer::HttpRequest request;
request.method = "GET";
request.path = "/shader-assets/solid";
RenderCadenceHttpRouteCallbacks callbacks;
const HttpControlServer::HttpResponse response = RouteRenderCadenceHttpRequest(request, server, callbacks);
ExpectEquals(response.status, "404 Not Found", "incomplete shader asset endpoint returns 404");
}
void TestKnownPostEndpointReturnsActionError()
{
using namespace RenderCadenceCompositor;
@@ -285,6 +332,8 @@ int main()
TestNdiSourcesEndpointUsesCallback();
TestWebSocketAcceptKey();
TestRootServesUiIndex();
TestShaderAssetEndpointUsesCallback();
TestIncompleteShaderAssetEndpointReturns404();
TestKnownPostEndpointReturnsActionError();
TestLayerPostEndpointsUseCallbacks();
TestGenericPostCallbackHandlesControlRoutes();

3
tests/RenderCadenceCompositorRuntimeStateJsonTests.cpp
View File

@@ -66,12 +66,14 @@ int main()
const std::filesystem::path root = MakeTestRoot();
WriteFile(root / "solid-color" / "shader.slang", "float4 shadeVideo(float2 uv) { return float4(uv, 0.0, 1.0); }\n");
WriteFile(root / "solid-color" / "ui" / "controls.js", "customElements.define('solid-color-controls', class extends HTMLElement {});\n");
WriteFile(root / "solid-color" / "shader.json", R"({
"id": "solid-color",
"name": "Solid Color",
"description": "A single color shader.",
"category": "Generator",
"entryPoint": "shadeVideo",
"ui": { "type": "webComponent", "entry": "ui/controls.js", "tag": "solid-color-controls" },
"parameters": [
{
"id": "color",
@@ -120,6 +122,7 @@ int main()
const std::string json = RenderCadenceCompositor::RuntimeStateToJson(stateInput);
ExpectContains(json, "\"shaders\":[{\"id\":\"solid-color\"", "state JSON should include supported shaders");
ExpectContains(json, "\"ui\":{\"type\":\"webComponent\",\"entry\":\"ui/controls.js\",\"tag\":\"solid-color-controls\",\"assetUrl\":\"/shader-assets/solid-color/ui/controls.js\"}", "state JSON should expose shader custom UI metadata");
ExpectContains(json, "\"layerCount\":1", "state JSON should expose the display layer count");
ExpectContains(json, "\"layers\":[{\"id\":\"runtime-layer-1\"", "state JSON should expose the active display layer");
ExpectContains(json, "\"parameters\":[{\"id\":\"color\"", "state JSON should expose active shader parameters");

46
tests/ShaderPackageRegistryTests.cpp
View File

@@ -50,12 +50,14 @@ void TestValidManifest()
WriteFile(root / "look" / "mask.png", "not a real png, but enough for existence checks");
WriteFile(root / "look" / "Inter.ttf", "not a real font, but enough for existence checks");
WriteFile(root / "look" / "Mono.ttf", "not a real font, but enough for existence checks");
WriteFile(root / "look" / "ui" / "controls.js", "customElements.define('look-controls', class extends HTMLElement {});\n");
WriteShaderPackage(root, "look", R"({
"id": "look-01",
"name": "Look 01",
"description": "Test package",
"category": "Tests",
"entryPoint": "shadeVideo",
"ui": { "type": "webComponent", "entry": "ui/controls.js", "tag": "look-controls" },
"textures": [{ "id": "maskTex", "path": "mask.png" }],
"fonts": [
{ "id": "inter", "path": "Inter.ttf" },
@@ -83,6 +85,8 @@ void TestValidManifest()
Expect(package.fontAssets.size() == 2 && package.fontAssets[0].id == "inter", "font assets parse");
Expect(package.temporal.enabled && package.temporal.effectiveHistoryLength == 4, "temporal history is capped");
Expect(package.feedback.enabled && package.feedback.writePassId == "main", "feedback defaults to the implicit main pass");
Expect(package.ui.enabled && package.ui.entryPath == "ui/controls.js", "custom UI entry parses");
Expect(package.ui.customElementTag == "look-controls", "custom UI tag parses");
Expect(package.parameters.size() == 4, "parameters parse");
Expect(package.parameters[0].description == "Scales the output intensity.", "parameter descriptions parse");
Expect(package.parameters[1].type == ShaderParameterType::Text && package.parameters[1].defaultTextValue == "LIVE", "text parameter parses");
@@ -93,6 +97,47 @@ void TestValidManifest()
std::filesystem::remove_all(root);
}
void TestInvalidUiDefinition()
{
struct Case
{
const char* directoryName;
const char* uiJson;
const char* expectedError;
bool writeEntry;
};
const Case cases[] = {
{ "bad-type", R"({ "type": "react", "entry": "ui/controls.js", "tag": "bad-type-controls" })", "webComponent", true },
{ "bad-path", R"({ "type": "webComponent", "entry": "../controls.js", "tag": "bad-path-controls" })", "safe relative", true },
{ "wrong-dir", R"({ "type": "webComponent", "entry": "controls.js", "tag": "wrong-dir-controls" })", "safe relative", true },
{ "bad-extension", R"({ "type": "webComponent", "entry": "ui/controls.txt", "tag": "bad-extension-controls" })", ".js or .mjs", true },
{ "bad-tag", R"({ "type": "webComponent", "entry": "ui/controls.js", "tag": "BadTag" })", "custom element", true },
{ "missing-entry", R"({ "type": "webComponent", "entry": "ui/missing.js", "tag": "missing-entry-controls" })", "UI entry not found", false },
};
const std::filesystem::path root = MakeTestRoot();
for (const Case& testCase : cases)
{
WriteShaderPackage(root, testCase.directoryName, std::string(R"({
"id": ")") + testCase.directoryName + R"(",
"name": "Bad UI",
"ui": )" + testCase.uiJson + R"(,
"parameters": []
})");
if (testCase.writeEntry)
WriteFile(root / testCase.directoryName / "ui" / "controls.js", "customElements.define('bad-controls', class extends HTMLElement {});\n");
ShaderPackageRegistry registry(4);
ShaderPackage package;
std::string error;
Expect(!registry.ParseManifest(root / testCase.directoryName / "shader.json", package, error), "invalid custom UI manifest is rejected");
Expect(error.find(testCase.expectedError) != std::string::npos, "invalid custom UI error explains the rejected field");
}
std::filesystem::remove_all(root);
}
void TestExplicitPassManifest()
{
const std::filesystem::path root = MakeTestRoot();
@@ -289,6 +334,7 @@ int main()
{
TestValidManifest();
TestExplicitPassManifest();
TestInvalidUiDefinition();
TestMissingFontAsset();
TestInvalidManifest();
TestInvalidTemporalSettings();

41
ui/src/components/LayerCard.jsx
View File

@@ -2,6 +2,7 @@ import { EyeOff, GripVertical, RotateCcw, SlidersHorizontal, Trash2 } from "luci
import { postJson } from "../api/controlApi";
import { ParameterField } from "./ParameterField";
import { ShaderCustomPanel } from "./ShaderCustomPanel";
export function LayerCard({
layer,
@@ -19,6 +20,24 @@ export function LayerCard({
onLayerParameterChange,
}) {
const selectedShader = shaders.find((shader) => shader.id === layer.shaderId);
const customUi = layer.ui?.type === "webComponent" ? layer.ui : selectedShader?.ui;
const hasCustomUi = customUi?.type === "webComponent" && customUi.assetUrl && customUi.tag;
const updateParameter = (parameterId, value) => onLayerParameterChange(layer.id, parameterId, value);
const resetParameters = () => postJson("/api/layers/reset-parameters", { layerId: layer.id });
const parameterControls = layer.parameters.length > 0 ? (
<div className="parameter-grid">
{layer.parameters.map((parameter) => (
<ParameterField
key={`${layer.id}:${parameter.id}`}
layer={layer}
parameter={parameter}
onParameterChange={updateParameter}
/>
))}
</div>
) : (
<p className="muted">This shader does not expose any user parameters.</p>
);
return (
<div
@@ -120,26 +139,24 @@ export function LayerCard({
type="button"
className="button-with-icon"
disabled={layer.parameters.length === 0}
onClick={() => postJson("/api/layers/reset-parameters", { layerId: layer.id })}
onClick={resetParameters}
>
<RotateCcw size={16} strokeWidth={1.9} aria-hidden="true" />
<span>Reset</span>
</button>
</div>
{layer.parameters.length > 0 ? (
<div className="parameter-grid">
{layer.parameters.map((parameter) => (
<ParameterField
key={`${layer.id}:${parameter.id}`}
{hasCustomUi ? (
<ShaderCustomPanel
layer={layer}
parameter={parameter}
onParameterChange={(parameterId, value) => onLayerParameterChange(layer.id, parameterId, value)}
/>
))}
</div>
ui={customUi}
onParameterChange={updateParameter}
onResetParameters={resetParameters}
>
{parameterControls}
</ShaderCustomPanel>
) : (
<p className="muted">This shader does not expose any user parameters.</p>
parameterControls
)}
</div>
) : null}

151
ui/src/components/ShaderCustomPanel.jsx Normal file
View File

@@ -0,0 +1,151 @@
import { useEffect, useMemo, useState } from "react";
const moduleLoadCache = new Map();
function loadCustomElement(ui) {
if (!ui?.assetUrl || !ui?.tag) {
return Promise.reject(new Error("Custom UI metadata is incomplete."));
}
if (customElements.get(ui.tag)) {
return Promise.resolve();
}
if (!moduleLoadCache.has(ui.assetUrl)) {
moduleLoadCache.set(ui.assetUrl, import(/* @vite-ignore */ ui.assetUrl));
}
return moduleLoadCache.get(ui.assetUrl).then(() => {
if (!customElements.get(ui.tag)) {
throw new Error(`Custom UI module did not register ${ui.tag}.`);
}
});
}
function parameterMap(parameters) {
return Object.fromEntries((parameters ?? []).map((parameter) => [parameter.id, parameter.value]));
}
export function ShaderCustomPanel({ layer, ui, onParameterChange, onResetParameters, children }) {
const [useDefaultControls, setUseDefaultControls] = useState(false);
const [loadError, setLoadError] = useState("");
const [element, setElement] = useState(null);
const values = useMemo(() => parameterMap(layer.parameters), [layer.parameters]);
useEffect(() => {
if (!ui?.assetUrl || !ui?.tag) {
setElement(null);
return undefined;
}
let cancelled = false;
let customElement = null;
setLoadError("");
setElement(null);
setUseDefaultControls(false);
loadCustomElement(ui)
.then(() => {
if (cancelled) {
return;
}
customElement = document.createElement(ui.tag);
customElement.className = "shader-custom-ui__element";
setElement(customElement);
})
.catch((error) => {
if (!cancelled) {
setLoadError(error instanceof Error ? error.message : "Custom controls failed to load.");
}
});
return () => {
cancelled = true;
customElement?.remove();
};
}, [ui?.assetUrl, ui?.tag]);
useEffect(() => {
if (!element) {
return undefined;
}
function handleParameterChange(event) {
const detail = event.detail ?? {};
const parameterId = detail.parameterId ?? detail.id;
if (parameterId) {
onParameterChange(parameterId, detail.value);
}
}
function handleReset() {
onResetParameters();
}
element.addEventListener("shader-parameter-change", handleParameterChange);
element.addEventListener("shader-reset-parameters", handleReset);
return () => {
element.removeEventListener("shader-parameter-change", handleParameterChange);
element.removeEventListener("shader-reset-parameters", handleReset);
};
}, [element, onParameterChange, onResetParameters]);
useEffect(() => {
if (!element) {
return;
}
element.layer = layer;
element.parameters = layer.parameters ?? [];
element.values = values;
element.setParameter = onParameterChange;
element.requestReset = onResetParameters;
element.dispatchEvent(new CustomEvent("shader-layer-update", {
detail: {
layer,
parameters: layer.parameters ?? [],
values,
},
}));
}, [element, layer, onParameterChange, onResetParameters, values]);
if (!ui?.assetUrl || !ui?.tag) {
return children;
}
if (useDefaultControls || loadError) {
return (
<div className="shader-custom-ui">
{!loadError ? (
<div className="shader-custom-ui__toolbar">
<button type="button" className="button-with-icon" onClick={() => setUseDefaultControls(false)}>
<span>Custom UI</span>
</button>
</div>
) : null}
{loadError ? <p className="shader-custom-ui__status">Custom controls unavailable; default controls shown.</p> : null}
{children}
</div>
);
}
return (
<div className="shader-custom-ui">
<div className="shader-custom-ui__toolbar">
<button type="button" className="button-with-icon" onClick={() => setUseDefaultControls(true)}>
<span>Default controls</span>
</button>
</div>
<div
className="shader-custom-ui__host"
ref={(node) => {
if (!node || !element) {
return;
}
if (element.parentNode !== node) {
node.replaceChildren(element);
}
}}
>
{!element ? <p className="shader-custom-ui__status">Loading custom controls.</p> : null}
</div>
</div>
);
}

35
ui/src/styles.css
View File

@@ -907,6 +907,41 @@ pre {
overflow-wrap: anywhere;
}
.shader-custom-ui {
display: grid;
gap: 0.65rem;
}
.shader-custom-ui__toolbar {
display: flex;
justify-content: flex-end;
}
.shader-custom-ui__toolbar button {
width: auto;
min-width: var(--button-min-width);
}
.shader-custom-ui__host {
min-width: 0;
min-height: 3rem;
padding: 0.65rem;
border: 1px solid var(--app-border);
border-radius: var(--app-radius);
background: #141a23;
}
.shader-custom-ui__element {
display: block;
width: 100%;
}
.shader-custom-ui__status {
margin: 0;
color: var(--app-muted);
font-size: 0.84rem;
}
.shader-picker__topline {
display: flex;
align-items: baseline;