UYVY backend
This commit is contained in:
@@ -12,8 +12,8 @@
|
||||
"frameRate": "59.94"
|
||||
},
|
||||
"output": {
|
||||
"backend": "decklink",
|
||||
"device": "default",
|
||||
"backend": "ndi",
|
||||
"device": "Shader",
|
||||
"resolution": "1080p",
|
||||
"frameRate": "59.94",
|
||||
"keying": {
|
||||
|
||||
@@ -137,11 +137,11 @@ Video input and output are optional edges. `input.backend` and `output.backend`
|
||||
|
||||
The input edge writes CPU frames into `InputFrameMailbox`. The current DeckLink backend captures BGRA8 directly where possible, or raw UYVY8 for render-thread GPU decode. The input edge does not call GL, render, preview, screenshot, shader, or output scheduling code.
|
||||
|
||||
The output edge consumes completed system-memory frames from `SystemFrameExchange`. The current DeckLink backend schedules those frames to DeckLink. If video output is unavailable, the app continues running render cadence, control, preview, telemetry, and logging.
|
||||
The output edge consumes completed system-memory frames from `SystemFrameExchange`. The render thread owns output pixel packing before readback: BGRA8 is read directly, and UYVY8 is packed on the GPU into a half-width RGBA8 target before async PBO readback. DeckLink and NDI output then schedule/send those completed CPU frames without invoking GL or converting pixels. If video output is unavailable, the app continues running render cadence, control, preview, telemetry, and logging.
|
||||
|
||||
Runtime state exposes backend-neutral output telemetry through `videoOutput`. Portable fields such as `enabled`, `backend`, and `scheduleFailures` stay at that level; backend-specific counters live under `videoOutput.backendMetrics`.
|
||||
|
||||
`PreviewWindowThread` is optional and uses a non-consuming system-memory tap. It paints with Win32/GDI on its own thread and skips preview ticks instead of blocking the frame exchange.
|
||||
`PreviewWindowThread` is optional and uses a non-consuming system-memory tap. It paints BGRA8 directly, decodes UYVY8 only for preview presentation, and skips preview ticks instead of blocking the frame exchange.
|
||||
|
||||
Screenshot routes are present in the UI/OpenAPI surface but are not implemented in the current native command path yet.
|
||||
|
||||
|
||||
@@ -18,7 +18,7 @@ These parts are the useful base for the fork:
|
||||
- `src/video/core`: backend-neutral input/output contracts.
|
||||
- `src/video/decklink`, `src/video/ndi`, and `src/video/playout`: concrete video I/O edges and scheduling support.
|
||||
- `src/render/thread`: render cadence ownership, readback pumping, runtime render-layer commit point, and metrics.
|
||||
- `src/render/readback`: BGRA8 PBO readback and completed-frame publication.
|
||||
- `src/render/readback`: BGRA8/UYVY8 PBO readback and completed-frame publication.
|
||||
- `src/platform`: hidden GL window/context support.
|
||||
- `src/app`: startup, config, video backend factory, runtime layer orchestration, preview, telemetry, and HTTP server hookup.
|
||||
- `src/control`, `src/telemetry`, `src/logging`, and `ui`: useful if the new repo still wants a local control surface.
|
||||
|
||||
@@ -60,6 +60,8 @@ It must not:
|
||||
|
||||
If no completed frame is available, record the miss and keep the ownership boundary intact.
|
||||
|
||||
Output pixel-format packing that requires GL belongs in the render-owned readback path before frame publication. Video I/O edges may select supported system-memory formats, but they must not invoke GL or become hidden pixel-conversion renderers.
|
||||
|
||||
DeckLink input is also an edge, not a renderer. It may capture/copy the latest supported CPU frame into an input mailbox and update input telemetry, but it must not call GL, schedule output, compile shaders, or drive render cadence. Format decode that requires GL belongs to the render-owned input upload path.
|
||||
|
||||
## 4. Runtime Build Work Produces Artifacts
|
||||
|
||||
@@ -566,7 +566,7 @@ components:
|
||||
properties:
|
||||
backend:
|
||||
type: string
|
||||
enum: [decklink, none]
|
||||
enum: [decklink, ndi, none]
|
||||
device:
|
||||
type: string
|
||||
resolution:
|
||||
@@ -578,13 +578,18 @@ components:
|
||||
properties:
|
||||
backend:
|
||||
type: string
|
||||
enum: [decklink, none]
|
||||
enum: [decklink, ndi, none]
|
||||
device:
|
||||
type: string
|
||||
resolution:
|
||||
type: string
|
||||
frameRate:
|
||||
type: string
|
||||
pixelFormat:
|
||||
type: string
|
||||
enum: [auto, bgra8, uyvy8]
|
||||
systemFramePixelFormat:
|
||||
type: string
|
||||
keying:
|
||||
type: object
|
||||
properties:
|
||||
@@ -752,10 +757,10 @@ components:
|
||||
description: Maximum observed render-thread frame draw duration in milliseconds for this process.
|
||||
readbackQueueMs:
|
||||
type: number
|
||||
description: Most recent duration spent queueing BGRA8 async PBO readback after rendering.
|
||||
description: Most recent duration spent queueing async PBO readback for the selected system-frame pixel format after rendering.
|
||||
completedReadbackCopyMs:
|
||||
type: number
|
||||
description: Most recent duration spent mapping and copying a completed BGRA8 readback into system-memory frame storage.
|
||||
description: Most recent duration spent mapping and copying a completed readback into system-memory frame storage.
|
||||
completedDrops:
|
||||
type: number
|
||||
description: Number of completed unscheduled system-memory frames dropped so render could reuse the slot.
|
||||
|
||||
@@ -13,8 +13,9 @@ RenderThread
|
||||
owns a hidden OpenGL context
|
||||
polls the oldest ready input frame without waiting
|
||||
uploads input frames into a render-owned GL texture
|
||||
renders simple BGRA8 motion at selected cadence
|
||||
queues async PBO readback
|
||||
renders at selected cadence
|
||||
optionally GPU-packs the output target into UYVY8
|
||||
queues async PBO readback for the selected system-frame format
|
||||
publishes completed frames into SystemFrameExchange
|
||||
|
||||
InputFrameMailbox
|
||||
@@ -47,17 +48,17 @@ Startup builds a small output preroll reserve before DeckLink scheduled playback
|
||||
|
||||
Included now:
|
||||
|
||||
- generic video output edge contract with DeckLink as the current implementation
|
||||
- generic video output edge contract with DeckLink and NDI output implementations
|
||||
- optional DeckLink input edge with BGRA8 capture or raw UYVY8 capture decoded on the render thread
|
||||
- non-blocking startup when DeckLink output is unavailable
|
||||
- hidden render-thread-owned OpenGL context
|
||||
- simple smooth-motion renderer
|
||||
- BGRA8-only output
|
||||
- BGRA8 and render-thread-packed UYVY8 system-memory output
|
||||
- non-blocking three-frame FIFO input mailbox for render
|
||||
- fast contiguous mailbox copy path for matching input row strides
|
||||
- bounded three-frame input warmup before render cadence starts
|
||||
- render-thread-owned input texture upload
|
||||
- async PBO readback
|
||||
- async PBO readback for the configured system-frame format
|
||||
- bounded FIFO system-memory frame exchange
|
||||
- bounded completed-frame output preroll reserve before DeckLink playback, with DeckLink scheduled depth still targeted at four
|
||||
- conservative DeckLink schedule-lead telemetry and recovery
|
||||
@@ -103,7 +104,7 @@ Those features should be ported only after the cadence spine is stable.
|
||||
This tracks parity with `apps/LoopThroughWithOpenGLCompositing`.
|
||||
|
||||
- [x] Stable DeckLink output cadence
|
||||
- [x] BGRA8 system-memory output path
|
||||
- [x] BGRA8 and UYVY8 system-memory output paths
|
||||
- [x] Render thread owns its primary GL context
|
||||
- [x] Output startup warmup before scheduled playback
|
||||
- [x] Non-blocking startup when DeckLink output is unavailable
|
||||
@@ -208,6 +209,7 @@ Currently consumed fields:
|
||||
- `output.device`
|
||||
- `output.resolution`
|
||||
- `output.frameRate`
|
||||
- `output.pixelFormat` (`auto`, `bgra8`, or `uyvy8`)
|
||||
- `output.keying.external`
|
||||
- `output.keying.alphaRequired`
|
||||
- `autoReload`
|
||||
@@ -215,9 +217,11 @@ Currently consumed fields:
|
||||
- `previewEnabled`
|
||||
- `previewFps`
|
||||
|
||||
`input.backend` and `output.backend` currently support `decklink` and `none`. Backend creation is routed through the app-side video backend factory, so new concrete backends can be added without making `main` or the render cadence path own their startup details.
|
||||
`input.backend` and `output.backend` currently support `decklink`, `ndi`, and `none`. Backend creation is routed through the app-side video backend factory, so new concrete backends can be added without making `main` or the render cadence path own their startup details.
|
||||
|
||||
When `previewEnabled` is true, the preview window runs on `PreviewWindowThread`. It paints BGRA8 system-memory frames with Win32/GDI after render readback has already completed, so it does not bind GL and does not consume frames from DeckLink output. `previewFps` controls the preview repaint cadence; the default is 60 fps and `config/runtime-host.json` tracks the shipped 59.94 output cadence.
|
||||
`output.pixelFormat=auto` chooses UYVY8 for DeckLink/NDI output unless alpha output is required, in which case it uses BGRA8. Explicit `uyvy8` requests are rejected back to BGRA8 when alpha is required. V210/YUVA remain explicit unsupported render-readback states until matching render-thread packers exist.
|
||||
|
||||
When `previewEnabled` is true, the preview window runs on `PreviewWindowThread`. It paints BGRA8 system-memory frames directly and decodes UYVY8 frames to BGRA for Win32/GDI after render readback has already completed, so it does not bind GL and does not consume frames from video output. `previewFps` controls the preview repaint cadence; the default is 60 fps and `config/runtime-host.json` tracks the shipped 59.94 output cadence.
|
||||
|
||||
The loaded config is treated as a read-only startup snapshot. Subsystems that need config should receive this snapshot or a narrowed config struct from app orchestration; they should not reload files independently.
|
||||
|
||||
@@ -305,7 +309,7 @@ Input telemetry:
|
||||
- `renderFrameMs`: most recent render-thread draw duration, excluding completed-readback copy and readback queue work
|
||||
- `renderFrameBudgetUsedPercent`: most recent render draw time as a percentage of the selected frame budget
|
||||
- `renderFrameMaxMs`: maximum observed render-thread draw duration for this process
|
||||
- `readbackQueueMs`: time spent queueing the most recent async BGRA8 PBO readback
|
||||
- `readbackQueueMs`: time spent queueing the most recent async PBO readback for the selected system-frame format
|
||||
- `completedReadbackCopyMs`: time spent mapping/copying the most recent completed readback into system-memory frame storage
|
||||
- `completedDrops`: oldest completed unscheduled system-memory frames dropped because the bounded completed reserve overflowed; this is an app-side reserve drop, not a DeckLink dropped-frame report
|
||||
- `acquireMisses`: times render/readback could not acquire a writable system-memory frame slot; completed frames waiting for playout are preserved instead of being displaced
|
||||
@@ -440,7 +444,7 @@ This app keeps the same core behavior but splits it into modules that can grow:
|
||||
- `render/`: cadence thread, clock, and simple renderer
|
||||
- `frames/InputFrameMailbox`: non-blocking bounded FIFO CPU input handoff with contiguous-copy fast path for matching row strides
|
||||
- `render/InputFrameTexture`: render-thread-owned upload of the currently acquired CPU input frame into GL, including raw UYVY8 decode into the shader-visible input texture
|
||||
- `render/readback/`: PBO-backed BGRA8 readback and completed-frame publication
|
||||
- `render/readback/`: PBO-backed BGRA8/UYVY8 readback and completed-frame publication
|
||||
- `render/thread/`: render thread lifecycle, GL startup/cadence loop, metrics, and runtime layer commit mailbox
|
||||
- `render/runtime/RuntimeRenderScene`: render-thread-owned GL scene for ready runtime shader layers
|
||||
- `render/runtime/RuntimeShaderPrepareWorker`: shared-context runtime shader program compile/link worker
|
||||
|
||||
@@ -11,7 +11,9 @@
|
||||
|
||||
#include <windows.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <chrono>
|
||||
#include <cctype>
|
||||
#include <iostream>
|
||||
#include <sstream>
|
||||
#include <string>
|
||||
@@ -46,6 +48,46 @@ private:
|
||||
HRESULT mResult = S_OK;
|
||||
};
|
||||
|
||||
std::string LowerAscii(std::string value)
|
||||
{
|
||||
std::transform(value.begin(), value.end(), value.begin(), [](unsigned char ch) {
|
||||
return static_cast<char>(std::tolower(ch));
|
||||
});
|
||||
return value;
|
||||
}
|
||||
|
||||
VideoIOPixelFormat SelectSystemFramePixelFormat(const RenderCadenceCompositor::AppConfig& config)
|
||||
{
|
||||
const std::string requestedFormat = LowerAscii(config.output.pixelFormat);
|
||||
if (requestedFormat == "bgra" || requestedFormat == "bgra8" || requestedFormat == "8-bit bgra")
|
||||
return VideoIOPixelFormat::Bgra8;
|
||||
|
||||
if (requestedFormat == "uyvy" || requestedFormat == "uyvy8" || requestedFormat == "8-bit yuv uyvy")
|
||||
{
|
||||
if (config.output.outputAlphaRequired)
|
||||
{
|
||||
RenderCadenceCompositor::LogWarning("app", "output.pixelFormat=uyvy8 was requested, but alpha output requires BGRA8 until a YUVA packer exists.");
|
||||
return VideoIOPixelFormat::Bgra8;
|
||||
}
|
||||
return VideoIOPixelFormat::Uyvy8;
|
||||
}
|
||||
|
||||
if (!requestedFormat.empty() && requestedFormat != "auto")
|
||||
{
|
||||
RenderCadenceCompositor::LogWarning(
|
||||
"app",
|
||||
"Unsupported output.pixelFormat '" + config.output.pixelFormat + "'; falling back to automatic BGRA8/UYVY8 selection.");
|
||||
}
|
||||
|
||||
if (config.output.outputAlphaRequired)
|
||||
return VideoIOPixelFormat::Bgra8;
|
||||
|
||||
const std::string backend = LowerAscii(config.output.backend);
|
||||
if (backend == "decklink" || backend == "ndi")
|
||||
return VideoIOPixelFormat::Uyvy8;
|
||||
return VideoIOPixelFormat::Bgra8;
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
int main(int argc, char** argv)
|
||||
@@ -63,10 +105,14 @@ int main(int argc, char** argv)
|
||||
|
||||
RenderCadenceCompositor::AppConfig appConfig = configProvider.Config();
|
||||
RenderCadenceCompositor::Logger::Instance().Start(appConfig.logging);
|
||||
appConfig.output.systemFramePixelFormat = SelectSystemFramePixelFormat(appConfig);
|
||||
RenderCadenceCompositor::Log(
|
||||
"app",
|
||||
"RenderCadenceCompositor starting. Starts render cadence, configured video I/O backends, and telemetry. Press Enter to stop.");
|
||||
RenderCadenceCompositor::Log("app", "Loaded config from " + configProvider.SourcePath().string());
|
||||
RenderCadenceCompositor::Log(
|
||||
"app",
|
||||
std::string("System frame output format: ") + VideoIOPixelFormatName(appConfig.output.systemFramePixelFormat) + ".");
|
||||
|
||||
ComInitGuard com;
|
||||
if (RenderCadenceCompositor::VideoBackendsRequireCom(appConfig) && !com.Initialize())
|
||||
@@ -83,7 +129,7 @@ int main(int argc, char** argv)
|
||||
appConfig.output.resolution,
|
||||
frameExchangeConfig.width,
|
||||
frameExchangeConfig.height);
|
||||
frameExchangeConfig.pixelFormat = VideoIOPixelFormat::Bgra8;
|
||||
frameExchangeConfig.pixelFormat = appConfig.output.systemFramePixelFormat;
|
||||
frameExchangeConfig.rowBytes = VideoIORowBytes(frameExchangeConfig.pixelFormat, frameExchangeConfig.width);
|
||||
frameExchangeConfig.capacity =
|
||||
appConfig.warmupCompletedFrames +
|
||||
@@ -137,6 +183,7 @@ int main(int argc, char** argv)
|
||||
RenderThread::Config renderConfig;
|
||||
renderConfig.width = frameExchangeConfig.width;
|
||||
renderConfig.height = frameExchangeConfig.height;
|
||||
renderConfig.outputPixelFormat = frameExchangeConfig.pixelFormat;
|
||||
const double fallbackFrameDurationMilliseconds = FrameDurationMillisecondsFromRateString(appConfig.output.frameRate);
|
||||
renderConfig.frameDurationMilliseconds = outputVideoModeResolved
|
||||
? outputVideoMode.frameDurationMilliseconds
|
||||
|
||||
@@ -13,8 +13,10 @@ AppConfig DefaultAppConfig()
|
||||
config.output.device = "default";
|
||||
config.output.resolution = "1080p";
|
||||
config.output.frameRate = "59.94";
|
||||
config.output.pixelFormat = "auto";
|
||||
config.output.externalKeyingEnabled = false;
|
||||
config.output.outputAlphaRequired = false;
|
||||
config.output.systemFramePixelFormat = VideoIOPixelFormat::Bgra8;
|
||||
config.outputThread.targetBufferedFrames = 4;
|
||||
config.telemetry.interval = std::chrono::seconds(1);
|
||||
config.logging.minimumLevel = LogLevel::Log;
|
||||
|
||||
@@ -27,8 +27,10 @@ struct VideoOutputAppConfig
|
||||
std::string device = "default";
|
||||
std::string resolution = "1080p";
|
||||
std::string frameRate = "59.94";
|
||||
std::string pixelFormat = "auto";
|
||||
bool externalKeyingEnabled = false;
|
||||
bool outputAlphaRequired = false;
|
||||
VideoIOPixelFormat systemFramePixelFormat = VideoIOPixelFormat::Bgra8;
|
||||
VideoFormat videoMode;
|
||||
};
|
||||
|
||||
|
||||
@@ -102,6 +102,7 @@ void ApplyOutputConfig(const JsonValue& root, AppConfig& config)
|
||||
ApplyString(*output, "device", config.output.device);
|
||||
ApplyString(*output, "resolution", config.output.resolution);
|
||||
ApplyString(*output, "frameRate", config.output.frameRate);
|
||||
ApplyString(*output, "pixelFormat", config.output.pixelFormat);
|
||||
|
||||
const JsonValue* keying = Find(*output, "keying");
|
||||
if (keying && keying->isObject())
|
||||
|
||||
@@ -149,6 +149,7 @@ private:
|
||||
Log("app", "Initializing optional video output backend: " + mConfig.output.backend + ".");
|
||||
VideoOutputEdgeConfig outputConfig;
|
||||
outputConfig.outputVideoMode = mConfig.output.videoMode;
|
||||
outputConfig.systemFramePixelFormat = mConfig.output.systemFramePixelFormat;
|
||||
outputConfig.externalKeyingEnabled = mConfig.output.externalKeyingEnabled;
|
||||
outputConfig.outputAlphaRequired = mConfig.output.outputAlphaRequired;
|
||||
if (!mOutput->Initialize(
|
||||
|
||||
@@ -306,6 +306,8 @@ inline std::string RuntimeStateToJson(const RuntimeStateJsonInput& input)
|
||||
writer.KeyString("device", input.config.output.device);
|
||||
writer.KeyString("resolution", input.config.output.resolution);
|
||||
writer.KeyString("frameRate", input.config.output.frameRate);
|
||||
writer.KeyString("pixelFormat", input.config.output.pixelFormat);
|
||||
writer.KeyString("systemFramePixelFormat", VideoIOPixelFormatName(input.config.output.systemFramePixelFormat));
|
||||
writer.Key("keying");
|
||||
writer.BeginObject();
|
||||
writer.KeyBool("external", input.config.output.externalKeyingEnabled);
|
||||
|
||||
@@ -14,6 +14,26 @@ const char* kPreviewWindowClassName = "RenderCadencePreviewWindow";
|
||||
constexpr UINT_PTR kPreviewTimerId = 1;
|
||||
constexpr UINT kPreviewStopMessage = WM_APP + 1;
|
||||
|
||||
unsigned char ClampByte(float value)
|
||||
{
|
||||
value = std::max(0.0f, std::min(1.0f, value));
|
||||
return static_cast<unsigned char>(value * 255.0f + 0.5f);
|
||||
}
|
||||
|
||||
void StoreRec709LegalUyvyAsBgra(unsigned char yByte, unsigned char uByte, unsigned char vByte, unsigned char* destinationPixel)
|
||||
{
|
||||
const float y = (static_cast<float>(yByte) - 16.0f) / 219.0f;
|
||||
const float cb = (static_cast<float>(uByte) - 16.0f) / 224.0f - 0.5f;
|
||||
const float cr = (static_cast<float>(vByte) - 16.0f) / 224.0f - 0.5f;
|
||||
const float red = y + 1.5748f * cr;
|
||||
const float green = y - 0.1873f * cb - 0.4681f * cr;
|
||||
const float blue = y + 1.8556f * cb;
|
||||
destinationPixel[0] = ClampByte(blue);
|
||||
destinationPixel[1] = ClampByte(green);
|
||||
destinationPixel[2] = ClampByte(red);
|
||||
destinationPixel[3] = 255;
|
||||
}
|
||||
|
||||
void RegisterPreviewWindowClass()
|
||||
{
|
||||
static bool registered = false;
|
||||
@@ -206,12 +226,12 @@ void PreviewWindowThread::Paint(HWND window)
|
||||
const bool canPaintFrame =
|
||||
frameAcquired &&
|
||||
frame.bytes != nullptr &&
|
||||
frame.pixelFormat == VideoIOPixelFormat::Bgra8 &&
|
||||
(frame.pixelFormat == VideoIOPixelFormat::Bgra8 || frame.pixelFormat == VideoIOPixelFormat::Uyvy8) &&
|
||||
frame.width > 0 &&
|
||||
frame.height > 0;
|
||||
if (canPaintFrame)
|
||||
{
|
||||
if (CopyPreviewOrientedBgra8Frame(frame))
|
||||
if (CopyPreviewOrientedFrame(frame))
|
||||
{
|
||||
const int previousStretchMode = SetStretchBltMode(dc, HALFTONE);
|
||||
POINT previousBrushOrigin = {};
|
||||
@@ -265,6 +285,15 @@ void PreviewWindowThread::Paint(HWND window)
|
||||
EndPaint(window, &paint);
|
||||
}
|
||||
|
||||
bool PreviewWindowThread::CopyPreviewOrientedFrame(const SystemFrame& frame)
|
||||
{
|
||||
if (frame.pixelFormat == VideoIOPixelFormat::Bgra8)
|
||||
return CopyPreviewOrientedBgra8Frame(frame);
|
||||
if (frame.pixelFormat == VideoIOPixelFormat::Uyvy8)
|
||||
return CopyPreviewOrientedUyvy8Frame(frame);
|
||||
return false;
|
||||
}
|
||||
|
||||
bool PreviewWindowThread::CopyPreviewOrientedBgra8Frame(const SystemFrame& frame)
|
||||
{
|
||||
constexpr std::size_t kBgraBytesPerPixel = 4;
|
||||
@@ -294,4 +323,40 @@ bool PreviewWindowThread::CopyPreviewOrientedBgra8Frame(const SystemFrame& frame
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
bool PreviewWindowThread::CopyPreviewOrientedUyvy8Frame(const SystemFrame& frame)
|
||||
{
|
||||
constexpr std::size_t kBgraBytesPerPixel = 4;
|
||||
if (frame.bytes == nullptr || frame.width == 0 || frame.height == 0 || frame.rowBytes <= 0)
|
||||
return false;
|
||||
|
||||
const std::size_t width = static_cast<std::size_t>(frame.width);
|
||||
const std::size_t height = static_cast<std::size_t>(frame.height);
|
||||
const std::size_t sourceRowBytes = static_cast<std::size_t>(frame.rowBytes);
|
||||
const std::size_t minimumSourceRowBytes = width * 2u;
|
||||
const std::size_t destinationRowBytes = width * kBgraBytesPerPixel;
|
||||
if (sourceRowBytes < minimumSourceRowBytes)
|
||||
return false;
|
||||
|
||||
mPaintPixels.resize(destinationRowBytes * height);
|
||||
const unsigned char* sourceBytes = static_cast<const unsigned char*>(frame.bytes);
|
||||
for (std::size_t y = 0; y < height; ++y)
|
||||
{
|
||||
const unsigned char* sourceRow = sourceBytes + (height - 1u - y) * sourceRowBytes;
|
||||
unsigned char* destinationRow = mPaintPixels.data() + y * destinationRowBytes;
|
||||
for (std::size_t x = 0; x < width; x += 2u)
|
||||
{
|
||||
const unsigned char* sourceMacroPixel = sourceRow + x * 2u;
|
||||
unsigned char* destinationPixel0 = destinationRow + x * kBgraBytesPerPixel;
|
||||
StoreRec709LegalUyvyAsBgra(sourceMacroPixel[1], sourceMacroPixel[0], sourceMacroPixel[2], destinationPixel0);
|
||||
if (x + 1u < width)
|
||||
{
|
||||
unsigned char* destinationPixel1 = destinationPixel0 + kBgraBytesPerPixel;
|
||||
StoreRec709LegalUyvyAsBgra(sourceMacroPixel[3], sourceMacroPixel[0], sourceMacroPixel[2], destinationPixel1);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -36,7 +36,9 @@ private:
|
||||
void ThreadMain();
|
||||
bool CreatePreviewWindow(std::string& error);
|
||||
void Paint(HWND window);
|
||||
bool CopyPreviewOrientedFrame(const SystemFrame& frame);
|
||||
bool CopyPreviewOrientedBgra8Frame(const SystemFrame& frame);
|
||||
bool CopyPreviewOrientedUyvy8Frame(const SystemFrame& frame);
|
||||
|
||||
SystemFrameExchange* mExchange = nullptr;
|
||||
PreviewWindowConfig mConfig;
|
||||
|
||||
@@ -1,160 +0,0 @@
|
||||
#include "Bgra8ReadbackPipeline.h"
|
||||
|
||||
#include "../frames/SystemFrameTypes.h"
|
||||
|
||||
#include <chrono>
|
||||
#include <cstring>
|
||||
|
||||
namespace
|
||||
{
|
||||
double MillisecondsSince(std::chrono::steady_clock::time_point start)
|
||||
{
|
||||
return std::chrono::duration_cast<std::chrono::duration<double, std::milli>>(
|
||||
std::chrono::steady_clock::now() - start).count();
|
||||
}
|
||||
}
|
||||
|
||||
Bgra8ReadbackPipeline::~Bgra8ReadbackPipeline()
|
||||
{
|
||||
Shutdown();
|
||||
}
|
||||
|
||||
bool Bgra8ReadbackPipeline::Initialize(unsigned width, unsigned height, std::size_t pboDepth)
|
||||
{
|
||||
Shutdown();
|
||||
|
||||
mWidth = width;
|
||||
mHeight = height;
|
||||
mRowBytes = VideoIORowBytes(VideoIOPixelFormat::Bgra8, width);
|
||||
if (mWidth == 0 || mHeight == 0 || mRowBytes == 0)
|
||||
return false;
|
||||
|
||||
if (!CreateRenderTarget())
|
||||
{
|
||||
Shutdown();
|
||||
return false;
|
||||
}
|
||||
|
||||
const std::size_t byteCount = static_cast<std::size_t>(mRowBytes) * static_cast<std::size_t>(mHeight);
|
||||
if (!mPboRing.Initialize(pboDepth, byteCount))
|
||||
{
|
||||
Shutdown();
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
void Bgra8ReadbackPipeline::Shutdown()
|
||||
{
|
||||
mPboRing.Shutdown();
|
||||
DestroyRenderTarget();
|
||||
mWidth = 0;
|
||||
mHeight = 0;
|
||||
mRowBytes = 0;
|
||||
}
|
||||
|
||||
bool Bgra8ReadbackPipeline::RenderAndQueue(uint64_t frameIndex, const RenderCallback& renderFrame)
|
||||
{
|
||||
if (mFramebuffer == 0 || !renderFrame)
|
||||
return false;
|
||||
|
||||
glBindFramebuffer(GL_FRAMEBUFFER, mFramebuffer);
|
||||
const auto renderStart = std::chrono::steady_clock::now();
|
||||
renderFrame(frameIndex);
|
||||
mLastRenderFrameMilliseconds = MillisecondsSince(renderStart);
|
||||
glBindFramebuffer(GL_FRAMEBUFFER, 0);
|
||||
|
||||
const auto queueStart = std::chrono::steady_clock::now();
|
||||
const bool queued = mPboRing.QueueReadback(mFramebuffer, mWidth, mHeight, frameIndex);
|
||||
mLastReadbackQueueMilliseconds = MillisecondsSince(queueStart);
|
||||
return queued;
|
||||
}
|
||||
|
||||
void Bgra8ReadbackPipeline::ConsumeCompleted(
|
||||
const AcquireFrameCallback& acquireFrame,
|
||||
const PublishFrameCallback& publishFrame,
|
||||
const CounterCallback& onAcquireMiss,
|
||||
const CounterCallback& onCompleted)
|
||||
{
|
||||
if (!acquireFrame || !publishFrame)
|
||||
return;
|
||||
|
||||
PboReadbackRing::CompletedReadback readback;
|
||||
while (mPboRing.TryAcquireCompleted(readback))
|
||||
{
|
||||
const auto copyStart = std::chrono::steady_clock::now();
|
||||
glBindBuffer(GL_PIXEL_PACK_BUFFER, readback.pbo);
|
||||
void* mapped = glMapBuffer(GL_PIXEL_PACK_BUFFER, GL_READ_ONLY);
|
||||
if (!mapped)
|
||||
{
|
||||
glBindBuffer(GL_PIXEL_PACK_BUFFER, 0);
|
||||
mPboRing.ReleaseCompleted(readback);
|
||||
mLastCompletedReadbackCopyMilliseconds = MillisecondsSince(copyStart);
|
||||
continue;
|
||||
}
|
||||
|
||||
SystemFrame frame;
|
||||
if (acquireFrame(frame))
|
||||
{
|
||||
const std::size_t byteCount = static_cast<std::size_t>(frame.rowBytes) * static_cast<std::size_t>(frame.height);
|
||||
if (frame.bytes != nullptr && byteCount <= readback.byteCount)
|
||||
{
|
||||
std::memcpy(frame.bytes, mapped, byteCount);
|
||||
frame.frameIndex = readback.frameIndex;
|
||||
frame.pixelFormat = VideoIOPixelFormat::Bgra8;
|
||||
publishFrame(frame);
|
||||
if (onCompleted)
|
||||
onCompleted();
|
||||
}
|
||||
}
|
||||
else if (onAcquireMiss)
|
||||
{
|
||||
onAcquireMiss();
|
||||
}
|
||||
|
||||
glUnmapBuffer(GL_PIXEL_PACK_BUFFER);
|
||||
glBindBuffer(GL_PIXEL_PACK_BUFFER, 0);
|
||||
mPboRing.ReleaseCompleted(readback);
|
||||
mLastCompletedReadbackCopyMilliseconds = MillisecondsSince(copyStart);
|
||||
}
|
||||
}
|
||||
|
||||
bool Bgra8ReadbackPipeline::CreateRenderTarget()
|
||||
{
|
||||
glGenFramebuffers(1, &mFramebuffer);
|
||||
glBindFramebuffer(GL_FRAMEBUFFER, mFramebuffer);
|
||||
|
||||
glGenTextures(1, &mTexture);
|
||||
glBindTexture(GL_TEXTURE_2D, mTexture);
|
||||
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
|
||||
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
|
||||
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
|
||||
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
|
||||
glTexImage2D(
|
||||
GL_TEXTURE_2D,
|
||||
0,
|
||||
GL_RGBA8,
|
||||
static_cast<GLsizei>(mWidth),
|
||||
static_cast<GLsizei>(mHeight),
|
||||
0,
|
||||
GL_BGRA,
|
||||
GL_UNSIGNED_INT_8_8_8_8_REV,
|
||||
nullptr);
|
||||
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, mTexture, 0);
|
||||
|
||||
const bool complete = glCheckFramebufferStatus(GL_FRAMEBUFFER) == GL_FRAMEBUFFER_COMPLETE;
|
||||
glBindTexture(GL_TEXTURE_2D, 0);
|
||||
glBindFramebuffer(GL_FRAMEBUFFER, 0);
|
||||
return complete;
|
||||
}
|
||||
|
||||
void Bgra8ReadbackPipeline::DestroyRenderTarget()
|
||||
{
|
||||
if (mFramebuffer != 0)
|
||||
glDeleteFramebuffers(1, &mFramebuffer);
|
||||
if (mTexture != 0)
|
||||
glDeleteTextures(1, &mTexture);
|
||||
mFramebuffer = 0;
|
||||
mTexture = 0;
|
||||
}
|
||||
419
src/render/readback/OutputReadbackPipeline.cpp
Normal file
419
src/render/readback/OutputReadbackPipeline.cpp
Normal file
@@ -0,0 +1,419 @@
|
||||
#include "OutputReadbackPipeline.h"
|
||||
|
||||
#include "../frames/SystemFrameTypes.h"
|
||||
|
||||
#include <chrono>
|
||||
#include <cstring>
|
||||
|
||||
namespace
|
||||
{
|
||||
double MillisecondsSince(std::chrono::steady_clock::time_point start)
|
||||
{
|
||||
return std::chrono::duration_cast<std::chrono::duration<double, std::milli>>(
|
||||
std::chrono::steady_clock::now() - start).count();
|
||||
}
|
||||
|
||||
constexpr GLuint kSceneTextureUnit = 0;
|
||||
|
||||
const char* kPackVertexShader = R"GLSL(
|
||||
#version 430 core
|
||||
void main()
|
||||
{
|
||||
vec2 positions[3] = vec2[3](
|
||||
vec2(-1.0, -1.0),
|
||||
vec2( 3.0, -1.0),
|
||||
vec2(-1.0, 3.0));
|
||||
gl_Position = vec4(positions[gl_VertexID], 0.0, 1.0);
|
||||
}
|
||||
)GLSL";
|
||||
|
||||
const char* kUyvyPackFragmentShader = R"GLSL(
|
||||
#version 430 core
|
||||
layout(binding = 0) uniform sampler2D uSceneTexture;
|
||||
uniform vec2 uSceneSize;
|
||||
out vec4 fragColor;
|
||||
|
||||
float rec709Luma(vec3 rgb)
|
||||
{
|
||||
return dot(clamp(rgb, vec3(0.0), vec3(1.0)), vec3(0.2126, 0.7152, 0.0722));
|
||||
}
|
||||
|
||||
float legalY(vec3 rgb)
|
||||
{
|
||||
return clamp((16.0 + rec709Luma(rgb) * 219.0) / 255.0, 16.0 / 255.0, 235.0 / 255.0);
|
||||
}
|
||||
|
||||
vec2 legalCbCr(vec3 rgb)
|
||||
{
|
||||
float y = rec709Luma(rgb);
|
||||
float cb = (rgb.b - y) / 1.8556;
|
||||
float cr = (rgb.r - y) / 1.5748;
|
||||
return clamp((16.0 + (vec2(cb, cr) + vec2(0.5)) * 224.0) / 255.0, vec2(16.0 / 255.0), vec2(240.0 / 255.0));
|
||||
}
|
||||
|
||||
void main()
|
||||
{
|
||||
ivec2 sceneSize = ivec2(uSceneSize);
|
||||
ivec2 packedCoord = ivec2(gl_FragCoord.xy);
|
||||
int sourceX0 = clamp(packedCoord.x * 2, 0, max(sceneSize.x - 1, 0));
|
||||
int sourceX1 = clamp(sourceX0 + 1, 0, max(sceneSize.x - 1, 0));
|
||||
int sourceY = clamp(packedCoord.y, 0, max(sceneSize.y - 1, 0));
|
||||
vec3 rgb0 = texelFetch(uSceneTexture, ivec2(sourceX0, sourceY), 0).rgb;
|
||||
vec3 rgb1 = texelFetch(uSceneTexture, ivec2(sourceX1, sourceY), 0).rgb;
|
||||
vec2 cbcr = legalCbCr((rgb0 + rgb1) * 0.5);
|
||||
fragColor = vec4(cbcr.x, legalY(rgb0), cbcr.y, legalY(rgb1));
|
||||
}
|
||||
)GLSL";
|
||||
}
|
||||
|
||||
OutputReadbackPipeline::~OutputReadbackPipeline()
|
||||
{
|
||||
Shutdown();
|
||||
}
|
||||
|
||||
bool OutputReadbackPipeline::SupportsPixelFormat(VideoIOPixelFormat pixelFormat)
|
||||
{
|
||||
return pixelFormat == VideoIOPixelFormat::Bgra8 || pixelFormat == VideoIOPixelFormat::Uyvy8;
|
||||
}
|
||||
|
||||
bool OutputReadbackPipeline::Initialize(unsigned width, unsigned height, VideoIOPixelFormat pixelFormat, std::size_t pboDepth)
|
||||
{
|
||||
Shutdown();
|
||||
|
||||
mWidth = width;
|
||||
mHeight = height;
|
||||
mPixelFormat = pixelFormat;
|
||||
mRowBytes = VideoIORowBytes(mPixelFormat, width);
|
||||
mUyvyPackWidth = mWidth / 2u;
|
||||
if (mWidth == 0 || mHeight == 0 || mRowBytes == 0 || !SupportsPixelFormat(mPixelFormat))
|
||||
return false;
|
||||
if (mPixelFormat == VideoIOPixelFormat::Uyvy8 && (mWidth % 2u) != 0)
|
||||
return false;
|
||||
|
||||
if (!CreateRenderTargets())
|
||||
{
|
||||
Shutdown();
|
||||
return false;
|
||||
}
|
||||
|
||||
const std::size_t byteCount = static_cast<std::size_t>(mRowBytes) * static_cast<std::size_t>(mHeight);
|
||||
if (!mPboRing.Initialize(pboDepth, byteCount))
|
||||
{
|
||||
Shutdown();
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
void OutputReadbackPipeline::Shutdown()
|
||||
{
|
||||
mPboRing.Shutdown();
|
||||
DestroyRenderTargets();
|
||||
DestroyUyvyPackProgram();
|
||||
mWidth = 0;
|
||||
mHeight = 0;
|
||||
mRowBytes = 0;
|
||||
mUyvyPackWidth = 0;
|
||||
mPixelFormat = VideoIOPixelFormat::Bgra8;
|
||||
mLastRenderFrameMilliseconds = 0.0;
|
||||
mLastReadbackQueueMilliseconds = 0.0;
|
||||
mLastCompletedReadbackCopyMilliseconds = 0.0;
|
||||
}
|
||||
|
||||
bool OutputReadbackPipeline::RenderAndQueue(uint64_t frameIndex, const RenderCallback& renderFrame)
|
||||
{
|
||||
if (mSceneFramebuffer == 0 || !renderFrame)
|
||||
return false;
|
||||
|
||||
const auto renderStart = std::chrono::steady_clock::now();
|
||||
glBindFramebuffer(GL_FRAMEBUFFER, mSceneFramebuffer);
|
||||
renderFrame(frameIndex);
|
||||
|
||||
ReadbackTarget target;
|
||||
const bool targetSelected = SelectReadbackTarget(target);
|
||||
glBindFramebuffer(GL_FRAMEBUFFER, 0);
|
||||
mLastRenderFrameMilliseconds = MillisecondsSince(renderStart);
|
||||
if (!targetSelected)
|
||||
return false;
|
||||
|
||||
const auto queueStart = std::chrono::steady_clock::now();
|
||||
const bool queued = mPboRing.QueueReadback(PboReadbackRing::ReadbackRequest{
|
||||
target.framebuffer,
|
||||
target.readWidth,
|
||||
target.readHeight,
|
||||
target.readFormat,
|
||||
target.readType,
|
||||
frameIndex
|
||||
});
|
||||
mLastReadbackQueueMilliseconds = MillisecondsSince(queueStart);
|
||||
return queued;
|
||||
}
|
||||
|
||||
void OutputReadbackPipeline::ConsumeCompleted(
|
||||
const AcquireFrameCallback& acquireFrame,
|
||||
const PublishFrameCallback& publishFrame,
|
||||
const CounterCallback& onAcquireMiss,
|
||||
const CounterCallback& onCompleted)
|
||||
{
|
||||
if (!acquireFrame || !publishFrame)
|
||||
return;
|
||||
|
||||
PboReadbackRing::CompletedReadback readback;
|
||||
while (mPboRing.TryAcquireCompleted(readback))
|
||||
{
|
||||
const auto copyStart = std::chrono::steady_clock::now();
|
||||
glBindBuffer(GL_PIXEL_PACK_BUFFER, readback.pbo);
|
||||
void* mapped = glMapBuffer(GL_PIXEL_PACK_BUFFER, GL_READ_ONLY);
|
||||
if (!mapped)
|
||||
{
|
||||
glBindBuffer(GL_PIXEL_PACK_BUFFER, 0);
|
||||
mPboRing.ReleaseCompleted(readback);
|
||||
mLastCompletedReadbackCopyMilliseconds = MillisecondsSince(copyStart);
|
||||
continue;
|
||||
}
|
||||
|
||||
SystemFrame frame;
|
||||
if (acquireFrame(frame))
|
||||
{
|
||||
const std::size_t byteCount = static_cast<std::size_t>(mRowBytes) * static_cast<std::size_t>(mHeight);
|
||||
const bool frameMatches =
|
||||
frame.bytes != nullptr &&
|
||||
frame.width == mWidth &&
|
||||
frame.height == mHeight &&
|
||||
frame.rowBytes == static_cast<long>(mRowBytes) &&
|
||||
frame.pixelFormat == mPixelFormat;
|
||||
if (frameMatches && byteCount <= readback.byteCount)
|
||||
{
|
||||
std::memcpy(frame.bytes, mapped, byteCount);
|
||||
frame.frameIndex = readback.frameIndex;
|
||||
frame.pixelFormat = mPixelFormat;
|
||||
publishFrame(frame);
|
||||
if (onCompleted)
|
||||
onCompleted();
|
||||
}
|
||||
}
|
||||
else if (onAcquireMiss)
|
||||
{
|
||||
onAcquireMiss();
|
||||
}
|
||||
|
||||
glUnmapBuffer(GL_PIXEL_PACK_BUFFER);
|
||||
glBindBuffer(GL_PIXEL_PACK_BUFFER, 0);
|
||||
mPboRing.ReleaseCompleted(readback);
|
||||
mLastCompletedReadbackCopyMilliseconds = MillisecondsSince(copyStart);
|
||||
}
|
||||
}
|
||||
|
||||
bool OutputReadbackPipeline::CreateRenderTargets()
|
||||
{
|
||||
if (!CreateSceneTarget())
|
||||
return false;
|
||||
if (mPixelFormat == VideoIOPixelFormat::Uyvy8 && !CreateUyvyTarget())
|
||||
return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool OutputReadbackPipeline::CreateSceneTarget()
|
||||
{
|
||||
glGenFramebuffers(1, &mSceneFramebuffer);
|
||||
glBindFramebuffer(GL_FRAMEBUFFER, mSceneFramebuffer);
|
||||
|
||||
glGenTextures(1, &mSceneTexture);
|
||||
glBindTexture(GL_TEXTURE_2D, mSceneTexture);
|
||||
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
|
||||
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
|
||||
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
|
||||
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
|
||||
glTexImage2D(
|
||||
GL_TEXTURE_2D,
|
||||
0,
|
||||
GL_RGBA8,
|
||||
static_cast<GLsizei>(mWidth),
|
||||
static_cast<GLsizei>(mHeight),
|
||||
0,
|
||||
GL_BGRA,
|
||||
GL_UNSIGNED_INT_8_8_8_8_REV,
|
||||
nullptr);
|
||||
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, mSceneTexture, 0);
|
||||
|
||||
const bool complete = glCheckFramebufferStatus(GL_FRAMEBUFFER) == GL_FRAMEBUFFER_COMPLETE;
|
||||
glBindTexture(GL_TEXTURE_2D, 0);
|
||||
glBindFramebuffer(GL_FRAMEBUFFER, 0);
|
||||
return complete;
|
||||
}
|
||||
|
||||
bool OutputReadbackPipeline::CreateUyvyTarget()
|
||||
{
|
||||
if (mUyvyPackWidth == 0)
|
||||
return false;
|
||||
|
||||
glGenFramebuffers(1, &mUyvyFramebuffer);
|
||||
glBindFramebuffer(GL_FRAMEBUFFER, mUyvyFramebuffer);
|
||||
|
||||
glGenTextures(1, &mUyvyTexture);
|
||||
glBindTexture(GL_TEXTURE_2D, mUyvyTexture);
|
||||
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
|
||||
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
|
||||
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
|
||||
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
|
||||
glTexImage2D(
|
||||
GL_TEXTURE_2D,
|
||||
0,
|
||||
GL_RGBA8,
|
||||
static_cast<GLsizei>(mUyvyPackWidth),
|
||||
static_cast<GLsizei>(mHeight),
|
||||
0,
|
||||
GL_RGBA,
|
||||
GL_UNSIGNED_BYTE,
|
||||
nullptr);
|
||||
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, mUyvyTexture, 0);
|
||||
|
||||
const bool complete = glCheckFramebufferStatus(GL_FRAMEBUFFER) == GL_FRAMEBUFFER_COMPLETE;
|
||||
glBindTexture(GL_TEXTURE_2D, 0);
|
||||
glBindFramebuffer(GL_FRAMEBUFFER, 0);
|
||||
return complete;
|
||||
}
|
||||
|
||||
bool OutputReadbackPipeline::SelectReadbackTarget(ReadbackTarget& target)
|
||||
{
|
||||
if (mPixelFormat == VideoIOPixelFormat::Bgra8)
|
||||
{
|
||||
target.framebuffer = mSceneFramebuffer;
|
||||
target.readWidth = mWidth;
|
||||
target.readHeight = mHeight;
|
||||
target.readFormat = GL_BGRA;
|
||||
target.readType = GL_UNSIGNED_INT_8_8_8_8_REV;
|
||||
return true;
|
||||
}
|
||||
|
||||
if (mPixelFormat == VideoIOPixelFormat::Uyvy8 && PackUyvy8())
|
||||
{
|
||||
target.framebuffer = mUyvyFramebuffer;
|
||||
target.readWidth = mUyvyPackWidth;
|
||||
target.readHeight = mHeight;
|
||||
target.readFormat = GL_RGBA;
|
||||
target.readType = GL_UNSIGNED_BYTE;
|
||||
return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
bool OutputReadbackPipeline::PackUyvy8()
|
||||
{
|
||||
if (mSceneTexture == 0 || mUyvyFramebuffer == 0 || !EnsureUyvyPackProgram())
|
||||
return false;
|
||||
|
||||
glBindFramebuffer(GL_FRAMEBUFFER, mUyvyFramebuffer);
|
||||
glViewport(0, 0, static_cast<GLsizei>(mUyvyPackWidth), static_cast<GLsizei>(mHeight));
|
||||
glDisable(GL_SCISSOR_TEST);
|
||||
glDisable(GL_DEPTH_TEST);
|
||||
glDisable(GL_BLEND);
|
||||
glUseProgram(mUyvyProgram);
|
||||
const GLint sceneSizeLocation = glGetUniformLocation(mUyvyProgram, "uSceneSize");
|
||||
if (sceneSizeLocation >= 0)
|
||||
glUniform2f(sceneSizeLocation, static_cast<GLfloat>(mWidth), static_cast<GLfloat>(mHeight));
|
||||
glActiveTexture(GL_TEXTURE0 + kSceneTextureUnit);
|
||||
glBindTexture(GL_TEXTURE_2D, mSceneTexture);
|
||||
glBindVertexArray(mUyvyVertexArray);
|
||||
glDrawArrays(GL_TRIANGLES, 0, 3);
|
||||
glBindVertexArray(0);
|
||||
glBindTexture(GL_TEXTURE_2D, 0);
|
||||
glActiveTexture(GL_TEXTURE0);
|
||||
glUseProgram(0);
|
||||
return true;
|
||||
}
|
||||
|
||||
bool OutputReadbackPipeline::EnsureUyvyPackProgram()
|
||||
{
|
||||
if (mUyvyProgram != 0)
|
||||
return true;
|
||||
|
||||
if (!CompileShader(GL_VERTEX_SHADER, kPackVertexShader, mUyvyVertexShader))
|
||||
return false;
|
||||
if (!CompileShader(GL_FRAGMENT_SHADER, kUyvyPackFragmentShader, mUyvyFragmentShader))
|
||||
{
|
||||
DestroyUyvyPackProgram();
|
||||
return false;
|
||||
}
|
||||
if (!LinkProgram(mUyvyVertexShader, mUyvyFragmentShader, mUyvyProgram))
|
||||
{
|
||||
DestroyUyvyPackProgram();
|
||||
return false;
|
||||
}
|
||||
|
||||
glUseProgram(mUyvyProgram);
|
||||
const GLint samplerLocation = glGetUniformLocation(mUyvyProgram, "uSceneTexture");
|
||||
if (samplerLocation >= 0)
|
||||
glUniform1i(samplerLocation, static_cast<GLint>(kSceneTextureUnit));
|
||||
glUseProgram(0);
|
||||
|
||||
glGenVertexArrays(1, &mUyvyVertexArray);
|
||||
return mUyvyVertexArray != 0;
|
||||
}
|
||||
|
||||
bool OutputReadbackPipeline::CompileShader(GLenum shaderType, const char* source, GLuint& shader)
|
||||
{
|
||||
shader = glCreateShader(shaderType);
|
||||
glShaderSource(shader, 1, &source, nullptr);
|
||||
glCompileShader(shader);
|
||||
|
||||
GLint compileResult = GL_FALSE;
|
||||
glGetShaderiv(shader, GL_COMPILE_STATUS, &compileResult);
|
||||
if (compileResult != GL_FALSE)
|
||||
return true;
|
||||
|
||||
glDeleteShader(shader);
|
||||
shader = 0;
|
||||
return false;
|
||||
}
|
||||
|
||||
bool OutputReadbackPipeline::LinkProgram(GLuint vertexShader, GLuint fragmentShader, GLuint& program)
|
||||
{
|
||||
program = glCreateProgram();
|
||||
glAttachShader(program, vertexShader);
|
||||
glAttachShader(program, fragmentShader);
|
||||
glLinkProgram(program);
|
||||
|
||||
GLint linkResult = GL_FALSE;
|
||||
glGetProgramiv(program, GL_LINK_STATUS, &linkResult);
|
||||
if (linkResult != GL_FALSE)
|
||||
return true;
|
||||
|
||||
glDeleteProgram(program);
|
||||
program = 0;
|
||||
return false;
|
||||
}
|
||||
|
||||
void OutputReadbackPipeline::DestroyRenderTargets()
|
||||
{
|
||||
if (mSceneFramebuffer != 0)
|
||||
glDeleteFramebuffers(1, &mSceneFramebuffer);
|
||||
if (mSceneTexture != 0)
|
||||
glDeleteTextures(1, &mSceneTexture);
|
||||
if (mUyvyFramebuffer != 0)
|
||||
glDeleteFramebuffers(1, &mUyvyFramebuffer);
|
||||
if (mUyvyTexture != 0)
|
||||
glDeleteTextures(1, &mUyvyTexture);
|
||||
mSceneFramebuffer = 0;
|
||||
mSceneTexture = 0;
|
||||
mUyvyFramebuffer = 0;
|
||||
mUyvyTexture = 0;
|
||||
}
|
||||
|
||||
void OutputReadbackPipeline::DestroyUyvyPackProgram()
|
||||
{
|
||||
if (mUyvyProgram != 0)
|
||||
glDeleteProgram(mUyvyProgram);
|
||||
if (mUyvyVertexShader != 0)
|
||||
glDeleteShader(mUyvyVertexShader);
|
||||
if (mUyvyFragmentShader != 0)
|
||||
glDeleteShader(mUyvyFragmentShader);
|
||||
if (mUyvyVertexArray != 0)
|
||||
glDeleteVertexArrays(1, &mUyvyVertexArray);
|
||||
mUyvyProgram = 0;
|
||||
mUyvyVertexShader = 0;
|
||||
mUyvyFragmentShader = 0;
|
||||
mUyvyVertexArray = 0;
|
||||
}
|
||||
@@ -9,7 +9,7 @@
|
||||
|
||||
struct SystemFrame;
|
||||
|
||||
class Bgra8ReadbackPipeline
|
||||
class OutputReadbackPipeline
|
||||
{
|
||||
public:
|
||||
using RenderCallback = std::function<void(uint64_t frameIndex)>;
|
||||
@@ -17,12 +17,12 @@ public:
|
||||
using PublishFrameCallback = std::function<bool(const SystemFrame& frame)>;
|
||||
using CounterCallback = std::function<void()>;
|
||||
|
||||
Bgra8ReadbackPipeline() = default;
|
||||
Bgra8ReadbackPipeline(const Bgra8ReadbackPipeline&) = delete;
|
||||
Bgra8ReadbackPipeline& operator=(const Bgra8ReadbackPipeline&) = delete;
|
||||
~Bgra8ReadbackPipeline();
|
||||
OutputReadbackPipeline() = default;
|
||||
OutputReadbackPipeline(const OutputReadbackPipeline&) = delete;
|
||||
OutputReadbackPipeline& operator=(const OutputReadbackPipeline&) = delete;
|
||||
~OutputReadbackPipeline();
|
||||
|
||||
bool Initialize(unsigned width, unsigned height, std::size_t pboDepth);
|
||||
bool Initialize(unsigned width, unsigned height, VideoIOPixelFormat pixelFormat, std::size_t pboDepth);
|
||||
void Shutdown();
|
||||
|
||||
bool RenderAndQueue(uint64_t frameIndex, const RenderCallback& renderFrame);
|
||||
@@ -32,25 +32,52 @@ public:
|
||||
const CounterCallback& onAcquireMiss = {},
|
||||
const CounterCallback& onCompleted = {});
|
||||
|
||||
GLuint Framebuffer() const { return mFramebuffer; }
|
||||
static bool SupportsPixelFormat(VideoIOPixelFormat pixelFormat);
|
||||
|
||||
GLuint Framebuffer() const { return mSceneFramebuffer; }
|
||||
unsigned Width() const { return mWidth; }
|
||||
unsigned Height() const { return mHeight; }
|
||||
unsigned RowBytes() const { return mRowBytes; }
|
||||
VideoIOPixelFormat PixelFormat() const { return VideoIOPixelFormat::Bgra8; }
|
||||
VideoIOPixelFormat PixelFormat() const { return mPixelFormat; }
|
||||
uint64_t PboQueueMisses() const { return mPboRing.QueueMisses(); }
|
||||
double LastRenderFrameMilliseconds() const { return mLastRenderFrameMilliseconds; }
|
||||
double LastReadbackQueueMilliseconds() const { return mLastReadbackQueueMilliseconds; }
|
||||
double LastCompletedReadbackCopyMilliseconds() const { return mLastCompletedReadbackCopyMilliseconds; }
|
||||
|
||||
private:
|
||||
bool CreateRenderTarget();
|
||||
void DestroyRenderTarget();
|
||||
struct ReadbackTarget
|
||||
{
|
||||
GLuint framebuffer = 0;
|
||||
unsigned readWidth = 0;
|
||||
unsigned readHeight = 0;
|
||||
GLenum readFormat = GL_BGRA;
|
||||
GLenum readType = GL_UNSIGNED_INT_8_8_8_8_REV;
|
||||
};
|
||||
|
||||
bool CreateSceneTarget();
|
||||
bool CreateUyvyTarget();
|
||||
bool CreateRenderTargets();
|
||||
bool SelectReadbackTarget(ReadbackTarget& target);
|
||||
bool PackUyvy8();
|
||||
bool EnsureUyvyPackProgram();
|
||||
bool CompileShader(GLenum shaderType, const char* source, GLuint& shader);
|
||||
bool LinkProgram(GLuint vertexShader, GLuint fragmentShader, GLuint& program);
|
||||
void DestroyRenderTargets();
|
||||
void DestroyUyvyPackProgram();
|
||||
|
||||
unsigned mWidth = 0;
|
||||
unsigned mHeight = 0;
|
||||
unsigned mRowBytes = 0;
|
||||
GLuint mFramebuffer = 0;
|
||||
GLuint mTexture = 0;
|
||||
unsigned mUyvyPackWidth = 0;
|
||||
VideoIOPixelFormat mPixelFormat = VideoIOPixelFormat::Bgra8;
|
||||
GLuint mSceneFramebuffer = 0;
|
||||
GLuint mSceneTexture = 0;
|
||||
GLuint mUyvyFramebuffer = 0;
|
||||
GLuint mUyvyTexture = 0;
|
||||
GLuint mUyvyVertexArray = 0;
|
||||
GLuint mUyvyProgram = 0;
|
||||
GLuint mUyvyVertexShader = 0;
|
||||
GLuint mUyvyFragmentShader = 0;
|
||||
double mLastRenderFrameMilliseconds = 0.0;
|
||||
double mLastReadbackQueueMilliseconds = 0.0;
|
||||
double mLastCompletedReadbackCopyMilliseconds = 0.0;
|
||||
@@ -46,9 +46,9 @@ void PboReadbackRing::Shutdown()
|
||||
mByteCount = 0;
|
||||
}
|
||||
|
||||
bool PboReadbackRing::QueueReadback(GLuint framebuffer, unsigned width, unsigned height, uint64_t frameIndex)
|
||||
bool PboReadbackRing::QueueReadback(const ReadbackRequest& request)
|
||||
{
|
||||
if (mSlots.empty())
|
||||
if (mSlots.empty() || request.framebuffer == 0 || request.width == 0 || request.height == 0)
|
||||
return false;
|
||||
|
||||
Slot& slot = mSlots[mWriteIndex];
|
||||
@@ -58,15 +58,22 @@ bool PboReadbackRing::QueueReadback(GLuint framebuffer, unsigned width, unsigned
|
||||
return false;
|
||||
}
|
||||
|
||||
glBindFramebuffer(GL_READ_FRAMEBUFFER, framebuffer);
|
||||
glBindFramebuffer(GL_READ_FRAMEBUFFER, request.framebuffer);
|
||||
glPixelStorei(GL_PACK_ALIGNMENT, 4);
|
||||
glPixelStorei(GL_PACK_ROW_LENGTH, 0);
|
||||
glBindBuffer(GL_PIXEL_PACK_BUFFER, slot.pbo);
|
||||
glBufferData(GL_PIXEL_PACK_BUFFER, static_cast<GLsizeiptr>(mByteCount), nullptr, GL_STREAM_READ);
|
||||
glReadPixels(0, 0, static_cast<GLsizei>(width), static_cast<GLsizei>(height), GL_BGRA, GL_UNSIGNED_INT_8_8_8_8_REV, nullptr);
|
||||
glReadPixels(
|
||||
0,
|
||||
0,
|
||||
static_cast<GLsizei>(request.width),
|
||||
static_cast<GLsizei>(request.height),
|
||||
request.format,
|
||||
request.type,
|
||||
nullptr);
|
||||
slot.fence = glFenceSync(GL_SYNC_GPU_COMMANDS_COMPLETE, 0);
|
||||
slot.inFlight = slot.fence != nullptr;
|
||||
slot.frameIndex = frameIndex;
|
||||
slot.frameIndex = request.frameIndex;
|
||||
glBindBuffer(GL_PIXEL_PACK_BUFFER, 0);
|
||||
|
||||
if (!slot.inFlight)
|
||||
|
||||
@@ -16,6 +16,16 @@ public:
|
||||
std::size_t byteCount = 0;
|
||||
};
|
||||
|
||||
struct ReadbackRequest
|
||||
{
|
||||
GLuint framebuffer = 0;
|
||||
unsigned width = 0;
|
||||
unsigned height = 0;
|
||||
GLenum format = GL_BGRA;
|
||||
GLenum type = GL_UNSIGNED_INT_8_8_8_8_REV;
|
||||
uint64_t frameIndex = 0;
|
||||
};
|
||||
|
||||
PboReadbackRing() = default;
|
||||
PboReadbackRing(const PboReadbackRing&) = delete;
|
||||
PboReadbackRing& operator=(const PboReadbackRing&) = delete;
|
||||
@@ -24,7 +34,7 @@ public:
|
||||
bool Initialize(std::size_t depth, std::size_t byteCount);
|
||||
void Shutdown();
|
||||
|
||||
bool QueueReadback(GLuint framebuffer, unsigned width, unsigned height, uint64_t frameIndex);
|
||||
bool QueueReadback(const ReadbackRequest& request);
|
||||
bool TryAcquireCompleted(CompletedReadback& readback);
|
||||
void ReleaseCompleted(const CompletedReadback& readback);
|
||||
void DrainCompleted();
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
#include "../logging/Logger.h"
|
||||
#include "../platform/HiddenGlWindow.h"
|
||||
#include "InputFrameTexture.h"
|
||||
#include "readback/Bgra8ReadbackPipeline.h"
|
||||
#include "readback/OutputReadbackPipeline.h"
|
||||
#include "GLExtensions.h"
|
||||
#include "RuntimeRenderScene.h"
|
||||
#include "SimpleMotionRenderer.h"
|
||||
@@ -43,14 +43,15 @@ void RenderThread::ThreadMain()
|
||||
|
||||
SimpleMotionRenderer renderer;
|
||||
RuntimeRenderScene runtimeRenderScene;
|
||||
Bgra8ReadbackPipeline readback;
|
||||
OutputReadbackPipeline readback;
|
||||
InputFrameTexture inputTexture;
|
||||
if (!runtimeRenderScene.StartPrepareWorker(std::move(prepareWindow), error))
|
||||
{
|
||||
SignalStartupFailure(error.empty() ? "Runtime shader prepare worker initialization failed." : error);
|
||||
return;
|
||||
}
|
||||
if (!renderer.InitializeGl(mConfig.width, mConfig.height) || !readback.Initialize(mConfig.width, mConfig.height, mConfig.pboDepth))
|
||||
if (!renderer.InitializeGl(mConfig.width, mConfig.height) ||
|
||||
!readback.Initialize(mConfig.width, mConfig.height, mConfig.outputPixelFormat, mConfig.pboDepth))
|
||||
{
|
||||
SignalStartupFailure("Render pipeline initialization failed.");
|
||||
return;
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
#include "RuntimeLayerModel.h"
|
||||
#include "RuntimeShaderArtifact.h"
|
||||
#include "RuntimeRenderScene.h"
|
||||
#include "VideoIOFormat.h"
|
||||
|
||||
#include <atomic>
|
||||
#include <condition_variable>
|
||||
@@ -16,7 +17,7 @@
|
||||
class SystemFrameExchange;
|
||||
class InputFrameMailbox;
|
||||
class InputFrameTexture;
|
||||
class Bgra8ReadbackPipeline;
|
||||
class OutputReadbackPipeline;
|
||||
|
||||
class RenderThread
|
||||
{
|
||||
@@ -25,6 +26,7 @@ public:
|
||||
{
|
||||
unsigned width = 1920;
|
||||
unsigned height = 1080;
|
||||
VideoIOPixelFormat outputPixelFormat = VideoIOPixelFormat::Bgra8;
|
||||
double frameDurationMilliseconds = 1000.0 / 59.94;
|
||||
std::size_t pboDepth = 6;
|
||||
};
|
||||
@@ -77,7 +79,7 @@ private:
|
||||
void CountRendered();
|
||||
void CountCompleted();
|
||||
void CountAcquireMiss();
|
||||
void PublishReadbackMetrics(const Bgra8ReadbackPipeline& readback);
|
||||
void PublishReadbackMetrics(const OutputReadbackPipeline& readback);
|
||||
void PublishInputMetrics(const InputFrameTexture& inputTexture);
|
||||
void TryCommitReadyRuntimeShader(RuntimeRenderScene& runtimeRenderScene);
|
||||
bool TryTakePendingRuntimeShaderArtifact(RuntimeShaderArtifact& artifact);
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
#include "../frames/InputFrameMailbox.h"
|
||||
#include "InputFrameTexture.h"
|
||||
#include "readback/Bgra8ReadbackPipeline.h"
|
||||
#include "readback/OutputReadbackPipeline.h"
|
||||
|
||||
RenderThread::Metrics RenderThread::GetMetrics() const
|
||||
{
|
||||
@@ -48,7 +48,7 @@ void RenderThread::CountAcquireMiss()
|
||||
mAcquireMisses.fetch_add(1, std::memory_order_relaxed);
|
||||
}
|
||||
|
||||
void RenderThread::PublishReadbackMetrics(const Bgra8ReadbackPipeline& readback)
|
||||
void RenderThread::PublishReadbackMetrics(const OutputReadbackPipeline& readback)
|
||||
{
|
||||
const double renderMilliseconds = readback.LastRenderFrameMilliseconds();
|
||||
mRenderFrameMilliseconds.store(renderMilliseconds, std::memory_order_relaxed);
|
||||
|
||||
@@ -55,6 +55,7 @@ struct VideoOutputEdgeMetrics
|
||||
struct VideoOutputEdgeConfig
|
||||
{
|
||||
VideoFormat outputVideoMode;
|
||||
VideoIOPixelFormat systemFramePixelFormat = VideoIOPixelFormat::Bgra8;
|
||||
bool externalKeyingEnabled = false;
|
||||
bool outputAlphaRequired = false;
|
||||
};
|
||||
|
||||
@@ -18,7 +18,7 @@ bool DeckLinkOutput::Initialize(const VideoOutputEdgeConfig& config, CompletionC
|
||||
formats.output = config.outputVideoMode;
|
||||
if (!mSession.DiscoverDevicesAndModes(formats, error))
|
||||
return false;
|
||||
if (!mSession.SelectPreferredFormats(formats, config.outputAlphaRequired, error))
|
||||
if (!mSession.SelectPreferredFormats(formats, config.systemFramePixelFormat, config.outputAlphaRequired, error))
|
||||
return false;
|
||||
if (!mSession.ConfigureOutput(
|
||||
[this](const VideoIOCompletion& completion) { HandleCompletion(completion); },
|
||||
|
||||
@@ -117,6 +117,11 @@ bool OutputSupportsFormat(IDeckLinkOutput* output, BMDDisplayMode displayMode, B
|
||||
&supported);
|
||||
return result == S_OK && supported != FALSE;
|
||||
}
|
||||
|
||||
bool RenderReadbackSupportsOutputFormat(VideoIOPixelFormat pixelFormat)
|
||||
{
|
||||
return pixelFormat == VideoIOPixelFormat::Bgra8 || pixelFormat == VideoIOPixelFormat::Uyvy8;
|
||||
}
|
||||
}
|
||||
|
||||
DeckLinkSession::~DeckLinkSession()
|
||||
@@ -254,7 +259,7 @@ bool DeckLinkSession::DiscoverDevicesAndModes(const VideoFormatSelection& videoM
|
||||
return true;
|
||||
}
|
||||
|
||||
bool DeckLinkSession::SelectPreferredFormats(const VideoFormatSelection& videoModes, bool outputAlphaRequired, std::string& error)
|
||||
bool DeckLinkSession::SelectPreferredFormats(const VideoFormatSelection& videoModes, VideoIOPixelFormat systemFramePixelFormat, bool outputAlphaRequired, std::string& error)
|
||||
{
|
||||
if (!output)
|
||||
{
|
||||
@@ -271,18 +276,30 @@ bool DeckLinkSession::SelectPreferredFormats(const VideoFormatSelection& videoMo
|
||||
}
|
||||
|
||||
mState.inputPixelFormat = VideoIOPixelFormat::Uyvy8;
|
||||
if (!RenderReadbackSupportsOutputFormat(systemFramePixelFormat))
|
||||
{
|
||||
error = "DeckLink output requested " + std::string(VideoIOPixelFormatName(systemFramePixelFormat)) +
|
||||
", but render readback currently supports only BGRA8 and UYVY8 system frames.";
|
||||
return false;
|
||||
}
|
||||
if (outputAlphaRequired && systemFramePixelFormat != VideoIOPixelFormat::Bgra8)
|
||||
{
|
||||
error = "DeckLink alpha output requires BGRA8 system frames until a YUVA render packer exists.";
|
||||
return false;
|
||||
}
|
||||
|
||||
const bool outputTenBitSupported = OutputSupportsFormat(output, outputDisplayMode, bmdFormat10BitYUV);
|
||||
const bool outputTenBitYuvaSupported = OutputSupportsFormat(output, outputDisplayMode, bmdFormat10BitYUVA);
|
||||
mState.outputPixelFormat = outputAlphaRequired
|
||||
? (outputTenBitYuvaSupported ? VideoIOPixelFormat::Yuva10 : VideoIOPixelFormat::Bgra8)
|
||||
: (outputTenBitSupported ? VideoIOPixelFormat::V210 : VideoIOPixelFormat::Bgra8);
|
||||
if (outputAlphaRequired && outputTenBitYuvaSupported)
|
||||
mState.formatStatusMessage += "External keying requires alpha; using 10-bit YUVA output. ";
|
||||
else if (outputAlphaRequired)
|
||||
mState.formatStatusMessage += "External keying requires alpha, but DeckLink output does not report 10-bit YUVA support for the configured mode; using 8-bit BGRA output. ";
|
||||
else if (!outputTenBitSupported)
|
||||
mState.formatStatusMessage += "DeckLink output does not report 10-bit YUV support for the configured mode; using 8-bit BGRA output. ";
|
||||
const BMDPixelFormat requestedOutputPixelFormat = DeckLinkPixelFormatForVideoIO(systemFramePixelFormat);
|
||||
if (!OutputSupportsFormat(output, outputDisplayMode, requestedOutputPixelFormat))
|
||||
{
|
||||
error = "DeckLink output does not report support for " +
|
||||
std::string(VideoIOPixelFormatName(systemFramePixelFormat)) +
|
||||
" in the configured display mode.";
|
||||
return false;
|
||||
}
|
||||
|
||||
mState.outputPixelFormat = systemFramePixelFormat;
|
||||
if (outputAlphaRequired)
|
||||
mState.formatStatusMessage += "External keying requires alpha; using BGRA8 system frames. ";
|
||||
|
||||
int deckLinkOutputRowBytes = 0;
|
||||
if (output->RowBytesForPixelFormat(DeckLinkPixelFormatForVideoIO(mState.outputPixelFormat), mState.outputFrameSize.width, &deckLinkOutputRowBytes) != S_OK)
|
||||
@@ -291,9 +308,12 @@ bool DeckLinkSession::SelectPreferredFormats(const VideoFormatSelection& videoMo
|
||||
return false;
|
||||
}
|
||||
mState.outputFrameRowBytes = static_cast<unsigned>(deckLinkOutputRowBytes);
|
||||
mState.outputPackTextureWidth = OutputIsTenBit()
|
||||
? PackedTextureWidthFromRowBytes(mState.outputFrameRowBytes)
|
||||
: mState.outputFrameSize.width;
|
||||
if (OutputIsTenBit())
|
||||
mState.outputPackTextureWidth = PackedTextureWidthFromRowBytes(mState.outputFrameRowBytes);
|
||||
else if (mState.outputPixelFormat == VideoIOPixelFormat::Uyvy8)
|
||||
mState.outputPackTextureWidth = mState.outputFrameSize.width / 2u;
|
||||
else
|
||||
mState.outputPackTextureWidth = mState.outputFrameSize.width;
|
||||
|
||||
if (InputIsTenBit())
|
||||
{
|
||||
|
||||
@@ -42,7 +42,7 @@ public:
|
||||
|
||||
void ReleaseResources();
|
||||
bool DiscoverDevicesAndModes(const VideoFormatSelection& videoModes, std::string& error);
|
||||
bool SelectPreferredFormats(const VideoFormatSelection& videoModes, bool outputAlphaRequired, std::string& error);
|
||||
bool SelectPreferredFormats(const VideoFormatSelection& videoModes, VideoIOPixelFormat systemFramePixelFormat, bool outputAlphaRequired, std::string& error);
|
||||
bool ConfigureOutput(OutputFrameCallback callback, const VideoFormat& outputVideoMode, bool externalKeyingEnabled, std::string& error);
|
||||
bool PrepareOutputSchedule();
|
||||
bool StartScheduledPlayback();
|
||||
|
||||
@@ -22,7 +22,27 @@ std::string ResolveSenderName(const std::string& configuredName)
|
||||
return configuredName;
|
||||
}
|
||||
|
||||
constexpr std::size_t kBgraBytesPerPixel = 4;
|
||||
bool IsSupportedNdiOutputPixelFormat(VideoIOPixelFormat pixelFormat)
|
||||
{
|
||||
return pixelFormat == VideoIOPixelFormat::Bgra8 || pixelFormat == VideoIOPixelFormat::Uyvy8;
|
||||
}
|
||||
|
||||
bool NdiFourCcForPixelFormat(VideoIOPixelFormat pixelFormat, NDIlib_FourCC_video_type_e& fourCc)
|
||||
{
|
||||
switch (pixelFormat)
|
||||
{
|
||||
case VideoIOPixelFormat::Bgra8:
|
||||
fourCc = NDIlib_FourCC_video_type_BGRA;
|
||||
return true;
|
||||
case VideoIOPixelFormat::Uyvy8:
|
||||
fourCc = NDIlib_FourCC_video_type_UYVY;
|
||||
return true;
|
||||
case VideoIOPixelFormat::V210:
|
||||
case VideoIOPixelFormat::Yuva10:
|
||||
default:
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
void SetCommonVideoFrameFields(
|
||||
NDIlib_video_frame_v2_t& ndiFrame,
|
||||
@@ -30,12 +50,13 @@ void SetCommonVideoFrameFields(
|
||||
unsigned int height,
|
||||
int rowBytes,
|
||||
void* pixels,
|
||||
NDIlib_FourCC_video_type_e fourCc,
|
||||
const VideoOutputEdgeConfig& config)
|
||||
{
|
||||
std::memset(&ndiFrame, 0, sizeof(ndiFrame));
|
||||
ndiFrame.xres = static_cast<int>(width);
|
||||
ndiFrame.yres = static_cast<int>(height);
|
||||
ndiFrame.FourCC = NDIlib_FourCC_video_type_BGRA;
|
||||
ndiFrame.FourCC = fourCc;
|
||||
ndiFrame.frame_rate_N = 60000;
|
||||
ndiFrame.frame_rate_D = 1001;
|
||||
if (config.outputVideoMode.frameRate == "60")
|
||||
@@ -80,15 +101,14 @@ void SetCommonVideoFrameFields(
|
||||
ndiFrame.line_stride_in_bytes = rowBytes;
|
||||
}
|
||||
|
||||
bool CopyBgra8FrameFlippedVertically(const VideoIOOutputFrame& frame, std::vector<unsigned char>& outputPixels, int& outputRowBytes)
|
||||
bool CopyFrameFlippedVertically(const VideoIOOutputFrame& frame, std::vector<unsigned char>& outputPixels, int& outputRowBytes)
|
||||
{
|
||||
if (frame.bytes == nullptr || frame.width == 0 || frame.height == 0 || frame.rowBytes <= 0)
|
||||
if (frame.bytes == nullptr || frame.width == 0 || frame.height == 0 || frame.rowBytes <= 0 || !IsSupportedNdiOutputPixelFormat(frame.pixelFormat))
|
||||
return false;
|
||||
|
||||
const std::size_t width = static_cast<std::size_t>(frame.width);
|
||||
const std::size_t height = static_cast<std::size_t>(frame.height);
|
||||
const std::size_t sourceRowBytes = static_cast<std::size_t>(frame.rowBytes);
|
||||
const std::size_t destinationRowBytes = width * kBgraBytesPerPixel;
|
||||
const std::size_t destinationRowBytes = VideoIORowBytes(frame.pixelFormat, frame.width);
|
||||
if (sourceRowBytes < destinationRowBytes)
|
||||
return false;
|
||||
|
||||
@@ -122,10 +142,15 @@ bool NdiOutput::Initialize(const VideoOutputEdgeConfig& config, CompletionCallba
|
||||
mConfig = config;
|
||||
mCompletionCallback = std::move(completionCallback);
|
||||
PopulateState(mState, config, mSenderName);
|
||||
if (!IsSupportedNdiOutputPixelFormat(config.systemFramePixelFormat))
|
||||
{
|
||||
error = std::string("NDI output does not support system frame format ") + VideoIOPixelFormatName(config.systemFramePixelFormat) + ".";
|
||||
return false;
|
||||
}
|
||||
|
||||
if (config.outputAlphaRequired)
|
||||
{
|
||||
mState.statusMessage = "NDI output sends BGRA frames with alpha when present; output.keying.alphaRequired is a DeckLink-only requirement.";
|
||||
mState.statusMessage = "NDI output can carry BGRA alpha when configured; output.keying.alphaRequired is a DeckLink-only requirement.";
|
||||
}
|
||||
|
||||
if (!AcquireNdiRuntime())
|
||||
@@ -168,7 +193,8 @@ bool NdiOutput::StartScheduledPlayback(std::string& error)
|
||||
|
||||
bool NdiOutput::ScheduleFrame(const VideoIOOutputFrame& frame)
|
||||
{
|
||||
if (frame.bytes == nullptr || frame.pixelFormat != VideoIOPixelFormat::Bgra8)
|
||||
NDIlib_FourCC_video_type_e fourCc = NDIlib_FourCC_video_type_BGRA;
|
||||
if (frame.bytes == nullptr || !NdiFourCcForPixelFormat(frame.pixelFormat, fourCc))
|
||||
{
|
||||
++mScheduleFailures;
|
||||
return false;
|
||||
@@ -190,14 +216,14 @@ bool NdiOutput::ScheduleFrame(const VideoIOOutputFrame& frame)
|
||||
|
||||
std::vector<unsigned char>& stagingBuffer = mStagingBuffers[mNextStagingBuffer];
|
||||
int stagingRowBytes = 0;
|
||||
if (!CopyBgra8FrameFlippedVertically(frame, stagingBuffer, stagingRowBytes))
|
||||
if (!CopyFrameFlippedVertically(frame, stagingBuffer, stagingRowBytes))
|
||||
{
|
||||
++mScheduleFailures;
|
||||
return false;
|
||||
}
|
||||
|
||||
NDIlib_video_frame_v2_t ndiFrame;
|
||||
SetCommonVideoFrameFields(ndiFrame, frame.width, frame.height, stagingRowBytes, stagingBuffer.data(), mConfig);
|
||||
SetCommonVideoFrameFields(ndiFrame, frame.width, frame.height, stagingRowBytes, stagingBuffer.data(), fourCc, mConfig);
|
||||
|
||||
const auto scheduleStart = std::chrono::steady_clock::now();
|
||||
NDIlib_send_send_video_async_v2(static_cast<NDIlib_send_instance_t>(sender), &ndiFrame);
|
||||
@@ -303,14 +329,19 @@ void NdiOutput::CompleteBuffer(void* buffer, VideoIOCompletionResult result)
|
||||
|
||||
void NdiOutput::PopulateState(VideoIOState& state, const VideoOutputEdgeConfig& config, const std::string& senderName)
|
||||
{
|
||||
const VideoIOPixelFormat outputPixelFormat = IsSupportedNdiOutputPixelFormat(config.systemFramePixelFormat)
|
||||
? config.systemFramePixelFormat
|
||||
: VideoIOPixelFormat::Bgra8;
|
||||
state = VideoIOState();
|
||||
state.outputFrameSize = config.outputVideoMode.frameSize;
|
||||
state.inputFrameSize = state.outputFrameSize;
|
||||
state.outputPixelFormat = VideoIOPixelFormat::Bgra8;
|
||||
state.inputPixelFormat = VideoIOPixelFormat::Bgra8;
|
||||
state.outputPixelFormat = outputPixelFormat;
|
||||
state.inputPixelFormat = outputPixelFormat;
|
||||
state.outputFrameRowBytes = VideoIORowBytes(state.outputPixelFormat, state.outputFrameSize.width);
|
||||
state.inputFrameRowBytes = state.outputFrameRowBytes;
|
||||
state.outputPackTextureWidth = state.outputFrameSize.width;
|
||||
state.outputPackTextureWidth = outputPixelFormat == VideoIOPixelFormat::Uyvy8
|
||||
? state.outputFrameSize.width / 2u
|
||||
: state.outputFrameSize.width;
|
||||
state.captureTextureWidth = state.inputFrameSize.width;
|
||||
state.outputDisplayModeName = config.outputVideoMode.displayName;
|
||||
state.inputDisplayModeName = "No input - NDI output session";
|
||||
@@ -322,6 +353,6 @@ void NdiOutput::PopulateState(VideoIOState& state, const VideoOutputEdgeConfig&
|
||||
state.keyerInterfaceAvailable = false;
|
||||
state.externalKeyingActive = false;
|
||||
state.frameBudgetMilliseconds = config.outputVideoMode.frameDurationMilliseconds;
|
||||
state.formatStatusMessage = "NDI output format: BGRA8.";
|
||||
state.formatStatusMessage = std::string("NDI output format: ") + VideoIOPixelFormatName(outputPixelFormat) + ".";
|
||||
}
|
||||
}
|
||||
|
||||
@@ -73,6 +73,7 @@ add_video_shader_test(RenderCadenceCompositorRuntimeStateJsonTests
|
||||
"${SRC_DIR}/app/AppConfig.cpp"
|
||||
"${SRC_DIR}/app/AppConfigProvider.cpp"
|
||||
${VIDEO_MODE_SOURCES}
|
||||
${VIDEO_FORMAT_SOURCES}
|
||||
"${SRC_DIR}/json/JsonWriter.cpp"
|
||||
${RUNTIME_LAYER_SOURCES}
|
||||
${RUNTIME_TEXT_SOURCES}
|
||||
|
||||
@@ -42,6 +42,7 @@ std::filesystem::path WriteConfigFixture()
|
||||
<< " \"device\": \"output-card-1\",\n"
|
||||
<< " \"resolution\": \"2160p\",\n"
|
||||
<< " \"frameRate\": \"60\",\n"
|
||||
<< " \"pixelFormat\": \"uyvy8\",\n"
|
||||
<< " \"keying\": {\n"
|
||||
<< " \"external\": true,\n"
|
||||
<< " \"alphaRequired\": true\n"
|
||||
@@ -78,6 +79,7 @@ void TestLoadsRuntimeHostConfig()
|
||||
Expect(config.input.device == "input-card-1", "input device loads");
|
||||
Expect(config.output.resolution == "2160p", "output resolution loads");
|
||||
Expect(config.output.frameRate == "60", "output frame rate loads");
|
||||
Expect(config.output.pixelFormat == "uyvy8", "output pixel format loads");
|
||||
Expect(config.output.device == "output-card-1", "output device loads");
|
||||
Expect(config.input.backend == "none", "video input backend loads");
|
||||
Expect(config.output.backend == "decklink", "video output backend loads");
|
||||
|
||||
@@ -118,7 +118,7 @@ int main()
|
||||
ExpectContains(json, "\"width\":1920", "state JSON should expose output width");
|
||||
ExpectContains(json, "\"height\":1080", "state JSON should expose output height");
|
||||
ExpectContains(json, "\"input\":{\"backend\":\"decklink\",\"device\":\"default\",\"resolution\":\"1080p\",\"frameRate\":\"59.94\"}", "state JSON should expose nested input config");
|
||||
ExpectContains(json, "\"output\":{\"backend\":\"decklink\",\"device\":\"default\",\"resolution\":\"1080p\",\"frameRate\":\"59.94\",\"keying\"", "state JSON should expose nested output config");
|
||||
ExpectContains(json, "\"output\":{\"backend\":\"decklink\",\"device\":\"default\",\"resolution\":\"1080p\",\"frameRate\":\"59.94\",\"pixelFormat\":\"auto\",\"systemFramePixelFormat\":\"8-bit BGRA\",\"keying\"", "state JSON should expose nested output config");
|
||||
ExpectContains(json, "\"videoOutput\":{\"enabled\":true,\"backend\":\"decklink\"", "state JSON should expose neutral video output status");
|
||||
ExpectContains(json, "\"scheduleFailures\":2", "state JSON should expose neutral video output schedule failures");
|
||||
ExpectContains(json, "\"backendMetrics\":{\"bufferedAvailable\":true,\"buffered\":4", "state JSON should expose backend-specific video output metrics");
|
||||
|
||||
Reference in New Issue
Block a user