# New Render Cadence App Plan This plan describes a new application folder that rebuilds the output path from the proven `DeckLinkRenderCadenceProbe` architecture, but as a maintainable app foundation rather than a monolithic probe file. The first goal is not to port the current compositor feature set. The first goal is to reproduce the probe's smooth 59.94/60 fps DeckLink output with clean module boundaries, tests where possible, and a structure that can later accept the shader/runtime/control systems without compromising timing. ## Working Name Suggested folder: ```text apps/RenderCadenceCompositor ``` Suggested executable: ```text RenderCadenceCompositor ``` The existing app remains intact: ```text apps/LoopThroughWithOpenGLCompositing ``` The probe remains the control sample: ```text apps/DeckLinkRenderCadenceProbe ``` ## Design Principle The app is built around one spine: ```text Render cadence thread -> owns GL context -> renders at selected frame cadence -> performs async BGRA8 readback -> publishes completed system-memory frames System frame exchange -> owns Free / Rendering / Completed / Scheduled slots -> latest-N semantics for completed unscheduled frames -> protects scheduled frames until DeckLink completion DeckLink output thread -> consumes completed frames -> schedules to target buffer depth -> releases scheduled frames on completion -> never renders ``` Everything else must fit around that spine. ## Non-Negotiable Rules - The render thread owns its GL context from initialization to shutdown. - The render thread is driven by selected render cadence, not DeckLink demand. - DeckLink scheduling never calls render code. - Completion callbacks never render. - No synchronous render request exists in the output path. - Preview, screenshot, input upload, shader rebuild, and runtime control cannot run ahead of a due output frame. - Completed unscheduled frames are latest-N and disposable. - Scheduled frames are protected until DeckLink completion. - Startup warms up real rendered frames before scheduled playback starts. ## Borrow From The Probe Keep these behaviors from `DeckLinkRenderCadenceProbe`: - hidden OpenGL context owned by the render thread - simple render loop with `nextRenderTime` - BGRA8 render target - PBO ring readback - non-blocking fence polling with zero timeout - system-memory slots with `Free`, `Rendering`, `Completed`, `Scheduled` - drop oldest completed unscheduled frame if render needs space - DeckLink playout thread only schedules completed frames - warmup completed frames before `StartScheduledPlayback()` - one-line-per-second timing telemetry ## Do Not Borrow Directly The probe is deliberately compact. Do not carry over these probe limitations into the new app: - one huge `.cpp` file - hard-coded output mode as permanent behavior - render pattern, frame store, PBO logic, DeckLink playout, COM setup, and telemetry mixed together - no reusable interfaces - no unit-testable non-GL core ## Proposed Folder Structure ```text apps/RenderCadenceCompositor/ README.md RenderCadenceCompositor.cpp app/ RenderCadenceApp.cpp RenderCadenceApp.h AppConfig.cpp AppConfig.h AppConfigProvider.cpp AppConfigProvider.h control/ HttpControlServer.cpp HttpControlServer.h RuntimeStateJson.h platform/ ComInit.cpp ComInit.h HiddenGlWindow.cpp HiddenGlWindow.h Win32Console.cpp Win32Console.h render/ RenderThread.cpp RenderThread.h RenderCadenceClock.cpp RenderCadenceClock.h SimpleMotionRenderer.cpp SimpleMotionRenderer.h Bgra8ReadbackPipeline.cpp Bgra8ReadbackPipeline.h PboReadbackRing.cpp PboReadbackRing.h frames/ SystemFrameExchange.cpp SystemFrameExchange.h SystemFrameTypes.h video/ DeckLinkOutput.cpp DeckLinkOutput.h DeckLinkOutputThread.cpp DeckLinkOutputThread.h telemetry/ CadenceTelemetry.cpp CadenceTelemetry.h CadenceTelemetryJson.h TelemetryHealthMonitor.h logging/ Logger.cpp Logger.h json/ JsonWriter.cpp JsonWriter.h ``` The new app can reuse selected existing source files from the current app at first: - `videoio/decklink/DeckLinkSession.*` - `videoio/decklink/DeckLinkDisplayMode.*` - `videoio/decklink/DeckLinkVideoIOFormat.*` - `videoio/decklink/DeckLinkFrameTransfer.*` - `videoio/VideoIOFormat.*` - `videoio/VideoIOTypes.h` - `videoio/VideoPlayoutScheduler.*` - `gl/renderer/GLExtensions.*` Longer term, shared code should move into common libraries, but the first version can link these files directly to avoid a big build-system refactor. ## Module Responsibilities ### `RenderCadenceApp` Owns top-level startup/shutdown sequencing. Responsibilities: - initialize COM - discover/select DeckLink output - create frame exchange - start render thread - wait for completed-frame warmup - start DeckLink output thread - wait for scheduled buffer warmup - start DeckLink scheduled playback - start telemetry printer - stop in reverse order It should not contain OpenGL drawing code, frame slot policy, or DeckLink scheduling loops. ### `AppConfig` Owns runtime settings for the initial app. Initial settings: - output mode preference - output width/height validation - frame buffer capacity - PBO depth - warmup completed-frame count - target DeckLink scheduled depth - telemetry interval Initial values should match the successful probe: ```text systemFrameSlots = 12 pboDepth = 6 warmupFrames = 4 targetDeckLinkBufferedFrames = 4 pixelFormat = BGRA8 ``` ### `HiddenGlWindow` Owns hidden Win32 window, device context, and OpenGL context creation. Responsibilities: - create hidden window with `CS_OWNDC` - choose/set pixel format - create `HGLRC` - expose `MakeCurrent()` and `ClearCurrent()` - destroy context/window safely Only `RenderThread` should call `MakeCurrent()` after startup. ### `RenderThread` Owns the render loop and GL context for its full lifetime. Responsibilities: - create/bind hidden GL context - resolve GL extensions - initialize renderer/readback pipeline - run cadence loop - render one frame when due - queue PBO readback - consume completed PBOs into `SystemFrameExchange` - record telemetry - destroy GL resources on the render thread It must not: - wait for DeckLink - schedule DeckLink frames - block on a system frame slot if only completed unscheduled frames can be dropped - accept arbitrary GL tasks ahead of output frames ### `RenderCadenceClock` Small, testable cadence helper. Responsibilities: - track target frame duration - return whether a render is due - compute sleep duration - detect overrun/skipped ticks - never speed up to fill buffers This should be unit tested without GL. ### `SimpleMotionRenderer` First renderer only. Responsibilities: - render obvious smooth motion and color changes - produce BGRA8-compatible framebuffer content - make dropped/repeated frames visually obvious This intentionally avoids shader-package/runtime complexity. ### `Bgra8ReadbackPipeline` Owns output framebuffer and BGRA8 readback orchestration. Responsibilities: - configure render target dimensions - render into an RGBA8/BGRA-compatible texture - coordinate `PboReadbackRing` - publish completed frames into `SystemFrameExchange` ### `PboReadbackRing` Owns PBO/fence state. Responsibilities: - queue readback into the next free PBO slot - poll completed fences with zero timeout - map/copy completed PBOs into provided system-memory slots - count PBO misses - clean up fences/PBOs on render thread This is GL-backed, but the state model should be small and easy to reason about. ### `SystemFrameExchange` The central handoff between render and video. Responsibilities: - own system-memory frame buffers - track slot states: `Free`, `Rendering`, `Completed`, `Scheduled` - provide `AcquireForRender()` - provide `PublishCompleted()` - provide `ConsumeCompletedForSchedule()` - provide `ReleaseScheduledByBytes()` - drop oldest completed unscheduled frame when render needs a slot - expose metrics This should be unit tested heavily. ### `DeckLinkOutput` Thin wrapper around `DeckLinkSession` for output-only use. Responsibilities: - discover/select output mode - configure output callback - prepare output schedule - schedule app-owned system-memory frames - start scheduled playback - stop/release resources - expose actual DeckLink buffered count No input support in the first version. ### `DeckLinkOutputThread` Owns playout scheduling loop. Responsibilities: - keep scheduled depth near target - consume completed frames from `SystemFrameExchange` - schedule them through `DeckLinkOutput` - release frame if scheduling fails - sleep briefly when scheduled buffer is full or no completed frame exists It must not render. ### `CadenceTelemetry` Owns counters, not policy. Initial counters: - rendered frames - completed readback frames - scheduled frames - completion count - completed-frame drops - acquire misses - schedule underruns - PBO queue misses - DeckLink late count - DeckLink dropped count - free/rendering/completed/scheduled slot counts - actual DeckLink buffered frames ### `TelemetryHealthMonitor` Samples cadence telemetry once per interval and logs only health events. Normal telemetry is available through the HTTP state endpoint. The console should not receive a healthy once-per-second cadence line. Health events: - warning when DeckLink late/dropped-frame counters increase - warning when schedule failures increase - error when app/DeckLink output buffering is starved ## Startup Sequence Target first-version startup: ```text main -> load AppConfig through AppConfigProvider -> initialize COM -> create SystemFrameExchange -> start RenderThread -> wait for completed frame warmup -> optionally discover/select/configure DeckLink output -> if DeckLink is available: -> start DeckLinkOutputThread -> wait for scheduled depth warmup -> DeckLinkOutput start scheduled playback -> if DeckLink is unavailable: -> continue without video output -> start TelemetryHealthMonitor -> start HttpControlServer -> wait for Enter ``` Shutdown: ```text stop HttpControlServer stop TelemetryHealthMonitor stop DeckLinkOutputThread DeckLinkOutput stop playback stop RenderThread DeckLinkOutput release resources release COM ``` ## First Milestone: Modular Probe Equivalent This is the only goal for the initial implementation. Feature set: - console app - output-only DeckLink - no input - hidden GL context - simple motion renderer - BGRA8 only - PBO async readback - latest-N system-memory frame exchange - warmup before playback - one-line telemetry Acceptance: - visible DeckLink output is smooth - `renderFps` near selected cadence - `scheduleFps` near selected cadence - scheduled count/decklink buffered count stable around 4 - no continuous late/drop count - no continuous PBO misses - behavior matches or exceeds `DeckLinkRenderCadenceProbe` ## Second Milestone: Testable Core Before porting compositor features, add tests for non-GL/non-DeckLink pieces. Test targets: - `SystemFrameExchangeTests` - `RenderCadenceClockTests` - `CadenceTelemetryTests` Important cases: - slot lifecycle transitions - scheduled slots are protected - completed unscheduled frames can be dropped - stale handles/generations are rejected - cadence does not speed up to refill buffers - cadence records overrun/skipped ticks ## Third Milestone: Replace Simple Renderer With Render Interface Add an interface around frame rendering: ```text IRenderScene -> InitializeGl() -> RenderFrame(frameIndex, time) -> ShutdownGl() ``` The first implementation remains `SimpleMotionRenderer`. This creates the insertion point for shader-package rendering later without changing timing/scheduling. ## Fourth Milestone: Begin Porting Current App Features Port only after the modular probe equivalent is stable. Suggested order: 1. shader package compile/load 2. render pass/layer stack drawing 3. runtime snapshot input to renderer 4. live state overlays 5. control services 6. persistence/runtime store 7. preview from system-memory frames 8. screenshot from system-memory frames 9. input capture via CPU latest-frame mailbox Each port must preserve the rule that the render thread cadence is primary. ## What Not To Port Early Do not port these until the output spine is proven: - DeckLink input - preview GL presentation - screenshot GL readback - HTTP/OSC control services - shader hot reload - persistence - runtime state JSON/open API - complex telemetry/event dispatch These are useful, but they are exactly the kinds of features that can accidentally reintroduce timing coupling. ## Build Plan Initial CMake can follow the probe pattern: ```cmake set(RENDER_CADENCE_APP_DIR "${CMAKE_CURRENT_SOURCE_DIR}/apps/RenderCadenceCompositor") add_executable(RenderCadenceCompositor # selected shared DeckLink/video/gl support files # new modular app files ) ``` Later, shared source should be split into libraries: ```text video_shader_decklink video_shader_videoio video_shader_gl_support render_cadence_core ``` Avoid doing that library split before the first modular app works. ## VS Code Launch Add a separate launch profile: ```text Debug RenderCadenceCompositor ``` Run it as a console app so telemetry remains visible. ## Documentation Add: ```text apps/RenderCadenceCompositor/README.md ``` The README should record: - intended architecture - build/run instructions - expected telemetry - test result notes - differences from the old app - differences from the probe ## Success Criteria Before Porting More Features Do not start feature porting until the new app can run with: - stable smooth DeckLink output - stable target scheduled depth - stable actual DeckLink buffered count - no regular visible freezes - no steady PBO misses - no steadily increasing late/dropped completions - focus/minimize changes do not affect output cadence - clean shutdown without hangs This gives us a clean foundation. Once this is true, every feature added later has to prove it does not damage the spine.