aiden/video-shader-toys
1
0
Fork 0
Code Issues Pull Requests Actions Releases 5 Wiki Activity

Input GPU decoding
All checks were successful
CI / React UI Build (push) Successful in 11s
Details
CI / Native Windows Build And Tests (push) Successful in 3m4s
Details
CI / Windows Release Package (push) Has been skipped
Details

Browse Source
This commit is contained in:
Aiden
2026-05-12 20:26:03 +10:00
parent ce28904891
commit fd4b70ec9c
6 changed files with 278 additions and 66 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
apps/RenderCadenceCompositor/README.md
View File

@@ -40,7 +40,7 @@ Startup warms up real rendered frames before DeckLink scheduled playback starts.
Included now:
- output-only DeckLink
- optional DeckLink input edge with BGRA8 capture or UYVY8-to-BGRA8 CPU conversion
- optional DeckLink input edge with BGRA8 capture or raw UYVY8 capture decoded on the render thread
- non-blocking startup when DeckLink output is unavailable
- hidden render-thread-owned OpenGL context
- simple smooth-motion renderer
@@ -74,7 +74,7 @@ Included now:
Intentionally not included yet:
- input format conversion
- additional input format conversion/scaling
- temporal/history/feedback shader storage
- texture/LUT asset upload
- text-parameter rasterization
@@ -118,7 +118,7 @@ This tracks parity with `apps/LoopThroughWithOpenGLCompositing`.
- [x] Layer reorder/bypass/set-shader/update-parameter/reset-parameter HTTP controls
- [x] Trigger parameter pulse count/time for latest trigger events
- [x] Optional DeckLink input capture
- [x] UYVY8-to-BGRA8 input conversion
- [x] UYVY8 input capture with render-thread GPU decode to shader input texture
- [x] Latest-frame CPU input mailbox
- [x] Render-owned input texture upload
- [x] Runtime shaders receive input through `gVideoInput`
@@ -247,11 +247,11 @@ Startup order is:
1. create `InputFrameMailbox`
2. try to attach DeckLink input for the configured input mode
3. prefer BGRA8 capture, otherwise accept UYVY8 capture and convert to BGRA8 before the mailbox
3. prefer BGRA8 capture, otherwise accept raw UYVY8 capture and configure the mailbox for UYVY8 bytes
4. start `DeckLinkInputThread`
5. leave input absent if discovery, setup, format support, or stream startup fails
`DeckLinkInput` and `DeckLinkInputThread` are deliberately narrow. They capture BGRA8 frames directly or convert UYVY8 frames to BGRA8 before submitting to `InputFrameMailbox`; they do not call GL, render, preview, screenshot, shader, or output scheduling code. Unsupported input modes or formats outside BGRA8/UYVY8 are reported explicitly and treated as an unavailable edge rather than silently converted.
`DeckLinkInput` and `DeckLinkInputThread` are deliberately narrow. They capture BGRA8 frames directly or raw UYVY8 frames into `InputFrameMailbox`; they do not call GL, render, preview, screenshot, shader, or output scheduling code. UYVY8-to-RGBA decode happens later inside the render-thread-owned input texture upload path, so the DeckLink callback stays a capture/copy edge only. Unsupported input modes or formats outside BGRA8/UYVY8 are reported explicitly and treated as an unavailable edge rather than silently converted.
The app samples telemetry once per second.
@@ -266,11 +266,11 @@ Input telemetry:
- `inputFramesReceived`: frames accepted into `InputFrameMailbox`
- `inputFramesDropped`: ready input frames dropped or missed because the mailbox was full
- `inputLatestAgeMs`: age of the newest submitted input frame
- `inputUploadMs`: render-thread GL upload time for the latest uploaded input frame
- `inputFormatSupported`: whether the latest frame reaching the render upload path was BGRA8-compatible
- `inputUploadMs`: render-thread GL upload/decode submission time for the latest uploaded input frame
- `inputFormatSupported`: whether the latest frame reaching the render upload path was BGRA8 or UYVY8 compatible
- `inputSignalPresent`: whether any input frame has reached the mailbox
- `inputCaptureFps`: DeckLink input callback capture rate
- `inputConvertMs`: input-edge UYVY8-to-BGRA8 conversion time for the latest converted frame
- `inputConvertMs`: input-edge CPU conversion time; expected to remain `0` for BGRA8 and raw UYVY8 capture because UYVY8 decode is render-thread GPU work
- `inputSubmitMs`: time spent submitting the latest captured/converted input frame to `InputFrameMailbox`
- `inputCaptureFormat`: selected DeckLink input format (`BGRA8`, `UYVY8`, or `none`)
- `inputNoSignalFrames`: DeckLink callbacks reporting no input source
@@ -380,7 +380,7 @@ This app keeps the same core behavior but splits it into modules that can grow:
- `platform/`: COM/Win32/hidden GL context support
- `render/`: cadence thread, clock, and simple renderer
- `frames/InputFrameMailbox`: non-blocking latest-frame CPU input handoff
- `render/InputFrameTexture`: render-thread-owned upload of the latest CPU input frame into GL
- `render/InputFrameTexture`: render-thread-owned upload of the latest CPU input frame into GL, including raw UYVY8 decode into the shader-visible input texture
- `render/readback/`: PBO-backed BGRA8 readback and completed-frame publication
- `render/runtime/RuntimeRenderScene`: render-thread-owned GL scene for ready runtime shader layers
- `render/runtime/RuntimeShaderPrepareWorker`: shared-context runtime shader program compile/link worker
@@ -403,4 +403,4 @@ Only after this app matches the probe's smooth output:
3. port runtime snapshots/live state
4. add control services
5. add preview/screenshot from system-memory frames
6. add scaling and additional input format support after the BGRA8/UYVY8 input edge is stable
6. add scaling and additional input format support after the BGRA8/raw-UYVY8 input edge is stable