video-shader-toys/docs/NEW_RENDER_CADENCE_APP_PLAN.md

# New Render Cadence App Plan

This plan describes a new application folder that rebuilds the output path from the proven `DeckLinkRenderCadenceProbe` architecture, but as a maintainable app foundation rather than a monolithic probe file.

The first goal is not to port the current compositor feature set. The first goal is to reproduce the probe's smooth 59.94/60 fps DeckLink output with clean module boundaries, tests where possible, and a structure that can later accept the shader/runtime/control systems without compromising timing.

## Working Name

Suggested folder:

```text
apps/RenderCadenceCompositor
```

Suggested executable:

```text
RenderCadenceCompositor
```

The existing app remains intact:

```text
apps/LoopThroughWithOpenGLCompositing
```

The probe remains the control sample:

```text
apps/DeckLinkRenderCadenceProbe
```

## Design Principle

The app is built around one spine:

```text
Render cadence thread
  -> owns GL context
  -> renders at selected frame cadence
  -> performs async BGRA8 readback
  -> publishes completed system-memory frames

System frame exchange
  -> owns Free / Rendering / Completed / Scheduled slots
  -> latest-N semantics for completed unscheduled frames
  -> protects scheduled frames until DeckLink completion

DeckLink output thread
  -> consumes completed frames
  -> schedules to target buffer depth
  -> releases scheduled frames on completion
  -> never renders
```

Everything else must fit around that spine.

## Non-Negotiable Rules

- The render thread owns its GL context from initialization to shutdown.
- The render thread is driven by selected render cadence, not DeckLink demand.
- DeckLink scheduling never calls render code.
- Completion callbacks never render.
- No synchronous render request exists in the output path.
- Preview, screenshot, input upload, shader rebuild, and runtime control cannot run ahead of a due output frame.
- Completed unscheduled frames are latest-N and disposable.
- Scheduled frames are protected until DeckLink completion.
- Startup warms up real rendered frames before scheduled playback starts.

## Borrow From The Probe

Keep these behaviors from `DeckLinkRenderCadenceProbe`:

- hidden OpenGL context owned by the render thread
- simple render loop with `nextRenderTime`
- BGRA8 render target
- PBO ring readback
- non-blocking fence polling with zero timeout
- system-memory slots with `Free`, `Rendering`, `Completed`, `Scheduled`
- drop oldest completed unscheduled frame if render needs space
- DeckLink playout thread only schedules completed frames
- warmup completed frames before `StartScheduledPlayback()`
- one-line-per-second timing telemetry

## Do Not Borrow Directly

The probe is deliberately compact. Do not carry over these probe limitations into the new app:

- one huge `.cpp` file
- hard-coded output mode as permanent behavior
- render pattern, frame store, PBO logic, DeckLink playout, COM setup, and telemetry mixed together
- no reusable interfaces
- no unit-testable non-GL core

## Proposed Folder Structure

```text
apps/RenderCadenceCompositor/
  README.md
  RenderCadenceCompositor.cpp

  app/
    RenderCadenceApp.cpp
    RenderCadenceApp.h
    AppConfig.cpp
    AppConfig.h
    AppConfigProvider.cpp
    AppConfigProvider.h

  control/
    HttpControlServer.cpp
    HttpControlServer.h
    RuntimeStateJson.h

  platform/
    ComInit.cpp
    ComInit.h
    HiddenGlWindow.cpp
    HiddenGlWindow.h
    Win32Console.cpp
    Win32Console.h

  render/
    RenderThread.cpp
    RenderThread.h
    RenderCadenceClock.cpp
    RenderCadenceClock.h
    SimpleMotionRenderer.cpp
    SimpleMotionRenderer.h
    Bgra8ReadbackPipeline.cpp
    Bgra8ReadbackPipeline.h
    PboReadbackRing.cpp
    PboReadbackRing.h

  frames/
    SystemFrameExchange.cpp
    SystemFrameExchange.h
    SystemFrameTypes.h

  video/
    DeckLinkOutput.cpp
    DeckLinkOutput.h
    DeckLinkOutputThread.cpp
    DeckLinkOutputThread.h

  telemetry/
    CadenceTelemetry.cpp
    CadenceTelemetry.h
    CadenceTelemetryJson.h
    TelemetryHealthMonitor.h

  logging/
    Logger.cpp
    Logger.h

  json/
    JsonWriter.cpp
    JsonWriter.h
```

The new app can reuse selected existing source files from the current app at first:

- `videoio/decklink/DeckLinkSession.*`
- `videoio/decklink/DeckLinkDisplayMode.*`
- `videoio/decklink/DeckLinkVideoIOFormat.*`
- `videoio/decklink/DeckLinkFrameTransfer.*`
- `videoio/VideoIOFormat.*`
- `videoio/VideoIOTypes.h`
- `videoio/VideoPlayoutScheduler.*`
- `gl/renderer/GLExtensions.*`

Longer term, shared code should move into common libraries, but the first version can link these files directly to avoid a big build-system refactor.

## Module Responsibilities

### `RenderCadenceApp`

Owns top-level startup/shutdown sequencing.

Responsibilities:

- initialize COM
- discover/select DeckLink output
- create frame exchange
- start render thread
- wait for completed-frame warmup
- start DeckLink output thread
- wait for scheduled buffer warmup
- start DeckLink scheduled playback
- start telemetry printer
- stop in reverse order

It should not contain OpenGL drawing code, frame slot policy, or DeckLink scheduling loops.

### `AppConfig`

Owns runtime settings for the initial app.

Initial settings:

- output mode preference
- output width/height validation
- frame buffer capacity
- PBO depth
- warmup completed-frame count
- target DeckLink scheduled depth
- telemetry interval

Initial values should match the successful probe:

```text
systemFrameSlots = 12
pboDepth = 6
warmupFrames = 4
targetDeckLinkBufferedFrames = 4
pixelFormat = BGRA8
```

### `HiddenGlWindow`

Owns hidden Win32 window, device context, and OpenGL context creation.

Responsibilities:

- create hidden window with `CS_OWNDC`
- choose/set pixel format
- create `HGLRC`
- expose `MakeCurrent()` and `ClearCurrent()`
- destroy context/window safely

Only `RenderThread` should call `MakeCurrent()` after startup.

### `RenderThread`

Owns the render loop and GL context for its full lifetime.

Responsibilities:

- create/bind hidden GL context
- resolve GL extensions
- initialize renderer/readback pipeline
- run cadence loop
- render one frame when due
- queue PBO readback
- consume completed PBOs into `SystemFrameExchange`
- record telemetry
- destroy GL resources on the render thread

It must not:

- wait for DeckLink
- schedule DeckLink frames
- block on a system frame slot if only completed unscheduled frames can be dropped
- accept arbitrary GL tasks ahead of output frames

### `RenderCadenceClock`

Small, testable cadence helper.

Responsibilities:

- track target frame duration
- return whether a render is due
- compute sleep duration
- detect overrun/skipped ticks
- never speed up to fill buffers

This should be unit tested without GL.

### `SimpleMotionRenderer`

First renderer only.

Responsibilities:

- render obvious smooth motion and color changes
- produce BGRA8-compatible framebuffer content
- make dropped/repeated frames visually obvious

This intentionally avoids shader-package/runtime complexity.

### `Bgra8ReadbackPipeline`

Owns output framebuffer and BGRA8 readback orchestration.

Responsibilities:

- configure render target dimensions
- render into an RGBA8/BGRA-compatible texture
- coordinate `PboReadbackRing`
- publish completed frames into `SystemFrameExchange`

### `PboReadbackRing`

Owns PBO/fence state.

Responsibilities:

- queue readback into the next free PBO slot
- poll completed fences with zero timeout
- map/copy completed PBOs into provided system-memory slots
- count PBO misses
- clean up fences/PBOs on render thread

This is GL-backed, but the state model should be small and easy to reason about.

### `SystemFrameExchange`

The central handoff between render and video.

Responsibilities:

- own system-memory frame buffers
- track slot states: `Free`, `Rendering`, `Completed`, `Scheduled`
- provide `AcquireForRender()`
- provide `PublishCompleted()`
- provide `ConsumeCompletedForSchedule()`
- provide `ReleaseScheduledByBytes()`
- drop oldest completed unscheduled frame when render needs a slot
- expose metrics

This should be unit tested heavily.

### `DeckLinkOutput`

Thin wrapper around `DeckLinkSession` for output-only use.

Responsibilities:

- discover/select output mode
- configure output callback
- prepare output schedule
- schedule app-owned system-memory frames
- start scheduled playback
- stop/release resources
- expose actual DeckLink buffered count

No input support in the first version.

### `DeckLinkOutputThread`

Owns playout scheduling loop.

Responsibilities:

- keep scheduled depth near target
- consume completed frames from `SystemFrameExchange`
- schedule them through `DeckLinkOutput`
- release frame if scheduling fails
- sleep briefly when scheduled buffer is full or no completed frame exists

It must not render.

### `CadenceTelemetry`

Owns counters, not policy.

Initial counters:

- rendered frames
- completed readback frames
- scheduled frames
- completion count
- completed-frame drops
- acquire misses
- schedule underruns
- PBO queue misses
- DeckLink late count
- DeckLink dropped count
- free/rendering/completed/scheduled slot counts
- actual DeckLink buffered frames

### `TelemetryHealthMonitor`

Samples cadence telemetry once per interval and logs only health events.

Normal telemetry is available through the HTTP state endpoint. The console should not receive a healthy once-per-second cadence line.

Health events:

- warning when DeckLink late/dropped-frame counters increase
- warning when schedule failures increase
- error when app/DeckLink output buffering is starved

## Startup Sequence

Target first-version startup:

```text
main
  -> load AppConfig through AppConfigProvider
  -> initialize COM
  -> create SystemFrameExchange
  -> start RenderThread
  -> wait for completed frame warmup
  -> optionally discover/select/configure DeckLink output
  -> if DeckLink is available:
       -> start DeckLinkOutputThread
       -> wait for scheduled depth warmup
       -> DeckLinkOutput start scheduled playback
  -> if DeckLink is unavailable:
       -> continue without video output
  -> start TelemetryHealthMonitor
  -> start HttpControlServer
  -> wait for Enter
```

Shutdown:

```text
stop HttpControlServer
stop TelemetryHealthMonitor
stop DeckLinkOutputThread
DeckLinkOutput stop playback
stop RenderThread
DeckLinkOutput release resources
release COM
```

## First Milestone: Modular Probe Equivalent

This is the only goal for the initial implementation.

Feature set:

- console app
- output-only DeckLink
- no input
- hidden GL context
- simple motion renderer
- BGRA8 only
- PBO async readback
- latest-N system-memory frame exchange
- warmup before playback
- one-line telemetry

Acceptance:

- visible DeckLink output is smooth
- `renderFps` near selected cadence
- `scheduleFps` near selected cadence
- scheduled count/decklink buffered count stable around 4
- no continuous late/drop count
- no continuous PBO misses
- behavior matches or exceeds `DeckLinkRenderCadenceProbe`

## Second Milestone: Testable Core

Before porting compositor features, add tests for non-GL/non-DeckLink pieces.

Test targets:

- `SystemFrameExchangeTests`
- `RenderCadenceClockTests`
- `CadenceTelemetryTests`

Important cases:

- slot lifecycle transitions
- scheduled slots are protected
- completed unscheduled frames can be dropped
- stale handles/generations are rejected
- cadence does not speed up to refill buffers
- cadence records overrun/skipped ticks

## Third Milestone: Replace Simple Renderer With Render Interface

Add an interface around frame rendering:

```text
IRenderScene
  -> InitializeGl()
  -> RenderFrame(frameIndex, time)
  -> ShutdownGl()
```

The first implementation remains `SimpleMotionRenderer`.

This creates the insertion point for shader-package rendering later without changing timing/scheduling.

## Fourth Milestone: Begin Porting Current App Features

Port only after the modular probe equivalent is stable.

Suggested order:

1. shader package compile/load
2. render pass/layer stack drawing
3. runtime snapshot input to renderer
4. live state overlays
5. control services
6. persistence/runtime store
7. preview from system-memory frames
8. screenshot from system-memory frames
9. input capture via CPU latest-frame mailbox

Each port must preserve the rule that the render thread cadence is primary.

## What Not To Port Early

Do not port these until the output spine is proven:

- DeckLink input
- preview GL presentation
- screenshot GL readback
- HTTP/OSC control services
- shader hot reload
- persistence
- runtime state JSON/open API
- complex telemetry/event dispatch

These are useful, but they are exactly the kinds of features that can accidentally reintroduce timing coupling.

## Build Plan

Initial CMake can follow the probe pattern:

```cmake
set(RENDER_CADENCE_APP_DIR "${CMAKE_CURRENT_SOURCE_DIR}/apps/RenderCadenceCompositor")

add_executable(RenderCadenceCompositor
  # selected shared DeckLink/video/gl support files
  # new modular app files
)
```

Later, shared source should be split into libraries:

```text
video_shader_decklink
video_shader_videoio
video_shader_gl_support
render_cadence_core
```

Avoid doing that library split before the first modular app works.

## VS Code Launch

Add a separate launch profile:

```text
Debug RenderCadenceCompositor
```

Run it as a console app so telemetry remains visible.

## Documentation

Add:

```text
apps/RenderCadenceCompositor/README.md
```

The README should record:

- intended architecture
- build/run instructions
- expected telemetry
- test result notes
- differences from the old app
- differences from the probe

## Success Criteria Before Porting More Features

Do not start feature porting until the new app can run with:

- stable smooth DeckLink output
- stable target scheduled depth
- stable actual DeckLink buffered count
- no regular visible freezes
- no steady PBO misses
- no steadily increasing late/dropped completions
- focus/minimize changes do not affect output cadence
- clean shutdown without hangs

This gives us a clean foundation. Once this is true, every feature added later has to prove it does not damage the spine.