# Audio / SDI Tearing Investigation

Date: 2026-05-05

## Problem

After adding DeckLink audio pass-through, the SDI output intermittently shows a torn/corrupted frame. The preview window does not show the artifact.

Observed artifact:

- Bottom portion of the SDI image can show an offset mix of current/previous frame.
- Looks like a frame-buffer or output-transfer issue rather than shader rendering.
- Occurs even with all shaders bypassed.
- Main branch is known good with no tearing.

Later tests also showed audio tearing/stutter when non-silent audio was scheduled.

## Known Good Baseline

- `main` branch has no SDI tearing.
- Current branch with `audioEnabled: false` ran for several minutes with no visible tearing.

This strongly suggests the issue is tied to DeckLink audio output/scheduling rather than the shader stack.

## SDK References Checked

### `InputLoopThrough`

Location:

`3rdParty/Blackmagic DeckLink SDK 16.0/Win/Samples/InputLoopThrough`

Findings:

- This is the SDK loop-through sample that keeps audio.
- It preserves DeckLink audio packet timestamps using `GetPacketTime(..., m_frameTimescale)`.
- It schedules audio packets with `ScheduleAudioSamples(..., packetTime, m_frameTimescale, ...)`.
- It uses 16-channel 32-bit embedded audio by default.
- It has separate scheduling threads for video/audio.
- It waits for both video and audio preroll before `StartScheduledPlayback`.

### `LoopThroughWithOpenGLCompositing`

Location:

`3rdParty/Blackmagic DeckLink SDK 16.0/Win/Samples/LoopThroughWithOpenGLCompositing`

Findings:

- This sample is the base for this app.
- It ignores `IDeckLinkAudioInputPacket`.
- It does not demonstrate audio pass-through.

### `SignalGenerator`

Location:

`3rdParty/Blackmagic DeckLink SDK 16.0/Win/Samples/SignalGenerator`

Findings:

- Uses `RenderAudioSamples()` callback to top up audio when DeckLink requests samples.
- Uses `GetBufferedAudioSampleFrameCount()` and a water level before scheduling more audio.

## Tests Tried And Results

### 1. Initial audio pass-through with FIFO and sample-time accumulator

Implementation:

- Copied incoming audio into a stereo FIFO.
- Scheduled audio with a generated `mNextAudioSampleFrame` clock in 48 kHz timescale.
- Matched delay to video preroll.

Result:

- Audio eventually worked.
- SDI video tearing appeared.

Conclusion:

- Basic audio output path triggered SDI instability.

### 2. Reworked audio toward SDK `InputLoopThrough` packet-timestamp model

Implementation:

- Preserved incoming packet time via `GetPacketTime(..., mFrameTimescale)`.
- Queued timestamped audio packets.
- Scheduled packets with `ScheduleAudioSamples(..., packet.streamTime, mFrameTimescale, ...)`.

Result:

- Tearing persisted.

Conclusion:

- Simply matching SDK timestamp domain did not fix the issue.

### 3. Restored video callback closer to `main`

Implementation:

- Removed extra `glFinish()` calls.
- Restored preview/readback ordering closer to `main`.
- Re-enabled fast transfer path after earlier tests disabled it.
- Removed audio texture upload from video playout callback.
- Removed audio analysis and audio locks from video playout callback.
- Removed DeckLink scheduling mutex around `ScheduleVideoFrame`.

Result:

- Tearing frequency seemed reduced at one point, but tearing persisted.

Conclusion:

- Extra work in the playout callback may have made timing worse, but was not the root cause.

### 4. Disabled audio completely

Config:

```json
"audioEnabled": false
```

Result:

- Ran for several minutes with no visible tearing.

Conclusion:

- The tearing is tied to audio being enabled.

### 5. Enabled audio input/analysis but disabled DeckLink audio output

Config:

```json
"audioEnabled": true,
"audioOutputEnabled": false
```

Result:

- No tearing appeared.

Conclusion:

- DeckLink audio input and CPU analysis are not the trigger.
- The problem is on the DeckLink audio output side.

### 6. Enabled DeckLink audio output but disabled scheduling

Config:

```json
"audioEnabled": true,
"audioOutputEnabled": true,
"audioScheduleEnabled": false
```

Result:

- No video tearing.
- Slight stutter appeared.

Conclusion:

- `EnableAudioOutput()` alone did not produce the tearing.
- Stutter was likely from enabling an audio output stream without feeding it samples.

### 7. Enabled audio scheduling but skipped audio preroll

Config:

```json
"audioEnabled": true,
"audioOutputEnabled": true,
"audioScheduleEnabled": true,
"audioPrerollEnabled": false
```

Result:

- Video tearing returned.
- Stutter also present.

Conclusion:

- `BeginAudioPreroll()` / `EndAudioPreroll()` are not required to trigger the tear.
- `ScheduleAudioSamples()` is strongly implicated.

### 8. Retained scheduled audio packet memory after `ScheduleAudioSamples`

Implementation:

- Kept scheduled packet buffers alive in a retain queue after scheduling.
- Avoided passing DeckLink pointers to vectors that immediately went out of scope.

Result:

- Video tearing and stutter persisted.

Conclusion:

- Buffer lifetime after `ScheduleAudioSamples()` was not the root cause.

### 9. Added audio water-level cap

Implementation:

- Restored SDK-style `GetBufferedAudioSampleFrameCount()` check.
- Only scheduled more audio if DeckLink buffer was below the target water level.

Result:

- Stutter was reduced.
- Video tearing persisted.

Conclusion:

- Overscheduling contributed to stutter/timing pressure.
- It did not explain the tearing.

### 10. Removed standalone audio scheduler thread

Implementation:

- Stopped starting the dedicated audio scheduler thread.
- Audio top-up occurred from input packet arrival and `RenderAudioSamples()` callback.

Result:

- No meaningful change.

Conclusion:

- The polling thread itself was not the cause.

### 11. Switched from timestamped audio output to continuous audio output

Implementation:

- Changed audio output to `bmdAudioOutputStreamContinuous`.
- Scheduled audio using a monotonic 48 kHz sample clock.

Result:

- Video tearing and stutter persisted.

Conclusion:

- The issue was not specific to timestamped output mode.

### 12. Rendered into the actual `completedFrame`

Implementation:

- Changed `PlayoutFrameCompleted()` to reuse the exact `completedFrame` passed by DeckLink rather than rotating an independent output-frame queue.

Result:

- No change.

Conclusion:

- The app was probably not overwriting a still-in-use frame from its output queue.

### 13. Scheduled generated silence instead of captured audio

Config:

```json
"audioScheduleSilence": true
```

Result:

- Occasional stutter.
- No video tearing.

Conclusion:

- Scheduling audio buffers itself can be stable if the audio data is zero.
- Non-zero audio data appears to be important.

### 14. Flattened captured audio into PCM FIFO and scheduled fixed chunks

Implementation:

- Captured packets were flattened into a PCM FIFO.
- DeckLink received fixed 10 ms chunks rather than original packet boundaries.
- Missing audio was padded with silence.

Result:

- Video tearing returned.
- Audio stutter/tearing returned.

Conclusion:

- Packet boundaries/timestamps were not the whole cause.
- Non-zero captured audio data still triggered instability.

### 15. Scheduled generated 440 Hz tone

Config:

```json
"audioScheduleTone": true
```

Result:

- Video tearing occurred.
- Tone/audio also tore.

Conclusion:

- The issue is not specific to captured input data.
- Non-zero scheduled audio, even generated tone, triggers the problem.

### 16. Changed DeckLink output to 16 embedded audio channels

Implementation:

- Enabled DeckLink audio output with 16 channels instead of 2.
- Mapped stereo to channels 1/2.
- Filled channels 3-16 with silence.

Result:

- Video tearing and audio tearing still occurred.

Conclusion:

- The issue is not simply caused by 2-channel embedded audio output.

### 17. Used DeckLink-owned output video frames with audio enabled

Implementation:

- When audio output is enabled:
  - disabled fast transfer path
  - created output frames with `CreateVideoFrame()`
  - avoided `CreateVideoFrameWithBuffer()` and the custom pinned playout allocator

Result:

- Video tearing and audio tearing still occurred.

Conclusion:

- The custom pinned output video buffers are likely not the root cause.

## Current Strong Conclusions

- Shader stack is not the cause.
- Preview/render output is not showing the issue, so the artifact is SDI/output-side.
- DeckLink audio input is not the cause.
- DeckLink audio output enabled but unscheduled does not cause tearing.
- `ScheduleAudioSamples()` with zero/silent buffers does not cause tearing.
- `ScheduleAudioSamples()` with non-zero audio causes both video tearing and audio tearing.
- The problem persists across:
  - timestamped audio output
  - continuous audio output
  - captured audio
  - generated tone
  - 2-channel output
  - 16-channel embedded output
  - app-owned/pinned output video buffers
  - DeckLink-owned output video frames

## Current Hypothesis

The issue appears to be a DeckLink output interaction where non-zero embedded audio samples disturb SDI video/audio output in this app’s scheduling model.

Since silence is stable but tone is not, the next likely areas to investigate are:

- Audio sample format/range/endian expectations.
- Whether DeckLink expects 32-bit audio samples to be in a different effective range than we are providing.
- Whether the scheduled audio buffer layout for the selected hardware/output mode differs from our assumptions.
- Whether the selected output mode/keyer/SDI configuration has constraints when non-zero embedded audio is present.
- Whether the SDK sample behaves correctly on the same hardware with a generated tone and same video mode.

## Suggested Next Tests

1. Schedule very low amplitude non-zero audio, e.g. constant `1`, then `256`, then a very quiet sine.
2. Try 16-bit audio output instead of 32-bit if supported.
3. Try `bmdAudioOutputStreamContinuousDontResample`.
4. Disable external keying and test with non-zero audio.
5. Build/run the SDK `SignalGenerator` or `InputLoopThrough` sample on the same DeckLink device, video mode, and SDI output path with non-zero embedded audio.
6. Add instrumentation for DeckLink status/errors around scheduled video/audio completion.
7. Confirm Desktop Video setup panel audio/SDI settings for the selected output.

## Current Config At Time Of Note

```json
"audioEnabled": true,
"audioOutputEnabled": true,
"audioScheduleEnabled": true,
"audioPrerollEnabled": true,
"audioScheduleSilence": false,
"audioScheduleTone": false
```