Initial audio support

This commit is contained in:
2026-05-04 14:32:29 +10:00
parent 44316b29c2
commit f836c53d10
17 changed files with 977 additions and 10 deletions

View File

@@ -125,6 +125,11 @@ struct ShaderContext
float bypass;
int sourceHistoryLength;
int temporalHistoryLength;
float2 audioRms;
float2 audioPeak;
float audioMonoRms;
float audioMonoPeak;
float4 audioBands;
};
```
@@ -140,6 +145,11 @@ Fields:
- `bypass`: `1.0` when the layer is bypassed, otherwise `0.0`.
- `sourceHistoryLength`: number of usable source-history frames currently available.
- `temporalHistoryLength`: number of usable temporal frames currently available for this layer.
- `audioRms`: left/right RMS level for the audio block synchronized with the rendered output frame.
- `audioPeak`: left/right peak level for the same synchronized audio block.
- `audioMonoRms`: mono RMS level derived from left/right.
- `audioMonoPeak`: mono peak level derived from left/right.
- `audioBands`: four smoothed, normalized low-to-high frequency bands.
## Helper Functions
@@ -149,6 +159,8 @@ The wrapper provides:
float4 sampleVideo(float2 uv);
float4 sampleSourceHistory(int framesAgo, float2 uv);
float4 sampleTemporalHistory(int framesAgo, float2 uv);
float4 sampleAudioWaveform(float x);
float4 sampleAudioSpectrum(float x);
```
`sampleVideo` samples the live decoded source video.
@@ -157,6 +169,10 @@ float4 sampleTemporalHistory(int framesAgo, float2 uv);
`sampleTemporalHistory` samples previous pre-layer input frames for temporal shaders that request `preLayerInput` history. `framesAgo` is clamped into the available range. If no temporal history is available, it falls back to `sampleVideo`.
`sampleAudioWaveform` samples the current synchronized audio waveform texture. `x` is normalized `0..1`; returned waveform channels are encoded from `-1..1` into `0..1`.
`sampleAudioSpectrum` samples the current synchronized audio spectrum texture. Values are normalized `0..1`.
Example:
```slang