PrismCore / API reference / Software playback

Class

SoftwarePlaybackPipeline

final class SoftwarePlaybackPipeline

Declared in Sources/PrismCore/Software/SoftwarePlaybackPipeline.swift · PrismCore 3.5.4

The software decode path (roadmap phase 7): demux with libavformat, decode with libavcodec, render with AVSampleBufferDisplayLayer + AVSampleBufferAudioRenderer under an AVSampleBufferRenderSynchronizer.

This is where sources go that AVPlayer's HLS-fMP4 pipeline cannot carry at all — VP9/VP8, MPEG-2, MPEG-4 ASP, VC-1, interlaced H.264. Unlike the rest of PrismCore, it is a player, not a service: there is no AVPlayer to hand a URL to, so this type owns the transport and the clock.

Swift

let pipeline = SoftwarePlaybackPipeline()
try pipeline.load(url: webmURL)          // synchronous; call off the main thread
// or, after a routing probe: try pipeline.load(probed: probedSource)
hostView.layer.addSublayer(pipeline.displayLayer!)
pipeline.play()

Pacing, and why audio is not gated on video

Both renderers publish isReadyForMoreMediaData and both call back through requestMediaDataWhenReady. The obvious loop — one cursor, paced on the video renderer's readiness — is a classic pipeline trap: as soon as video stops draining (a backgrounded app, a display layer off screen, a decode that briefly outruns the queue) the gate never reopens, and audio starves behind it even though its own renderer is begging for data. That is audible as a dropout, and it is a coupling bug, not a throughput problem.

So the loop reads while either stream still wants data, decodes into two small pending queues, and each renderer drains its own queue on its own callback. Video being full therefore cannot stop audio from being read, decoded and enqueued; the cost is that chasing audio past a full video queue lets the video queue grow by roughly the container's interleave distance, which a hard cap bounds (see Pacing). A pathological interleave stops reads instead of growing memory — the honest failure. An independent audio look-ahead cursor is the refinement this shape leaves room for.

Audio track switching

selectAudioTrack(streamIndex:) swaps the audio decoder mid-playback, leaving the clock and the video renderer untouched (issue #35). The new decoder is built before the old one is torn down — a track whose decoder can't open leaves the current track playing rather than leaving silence — and the demuxer is then rewound to the clock's present so the new track starts where the listener is, not where the read cursor had run ahead to. The rewind re-reads video the renderer already holds; those frames are dropped by timestamp instead of re-enqueued, which is what keeps the video path a bystander. On a source that refuses the rewind (no index), the new track joins at the read position instead — a gap of the queue's look-ahead, which beats refusing the switch.

Not here yet

Subtitle rendering belongs to the host: poll activeSubtitleCues for an overlay. Bitmap subtitles and external subtitle files are not selected here. .ended fires from a synchronizer boundary at the last presentation end — when the speaker has finished the last buffer, not when it was enqueued.

Initializers

init(allowHardwareDecode:audioDelaySeconds:)source

convenience init(allowHardwareDecode: Bool = true, audioDelaySeconds: Double = 0)

The shipping configuration: a real display layer, audio renderer and synchronizer.

Instance properties

onStateChange source

var onStateChange: (@Sendable (State) -> Void)?

Called on the feed queue whenever state changes. Set before load.

state source

var state: State { get }

No documentation comment in the source.

currentTime source

var currentTime: CMTime { get }

Master-clock time, i.e. the presentation time being rendered. On the source's own axis (the pipeline never rebases timestamps to zero).

durationSeconds source

var durationSeconds: Double? { get }

Whole-source duration in seconds — what the pipeline currently knows, not a constant. nil until load, and possibly for a while after: a container can withhold its duration (a Matroska written to a pipe carries none, and libavformat does not estimate one for it), in which case this becomes non-nil the moment the knowledge arrives — at the latest on EOF, when the last timestamp read is the duration. A host that copies this once at load keeps the nil forever; re-read it alongside the position (#58). Stays nil only for sources that genuinely have no end (live ingests).

volume source

var volume: Float { get set }

The pipeline's own output level, 0…1. Scales the audio renderer, not the device — a host's volume drag has no honest system-level lever, so this is the same contract AVPlayer's volume offers.

displayLayer source

var displayLayer: AVSampleBufferDisplayLayer? { get }

The layer to put in the host's view hierarchy. nil when the pipeline was built with injected sinks (tests).

routeDescription source

var routeDescription: String { get }

What each decoder ended up doing, for diagnostics: which route the video took (VideoToolbox zero-copy vs CPU) and the audio's output format.

sourceInfo source

var sourceInfo: SourceInfo? { get }

The probe's description of the loaded source — the same facts SourceProbe reports for routing (tracks with language/title/channels, chapters, subtitle streams), read off this pipeline's own context at load. nil before load.

This is the enumeration half of track selection: a host transport lists sourceInfo.audioTracks (or selectableAudioTracks, already filtered to what this build can decode) and drives selectAudioTrack with the entry's streamIndex.

selectableAudioTracks source

var selectableAudioTracks: [AudioTrackInfo] { get }

The audio tracks selectAudioTrack(streamIndex:) will accept: every audio stream this build has a decoder for, in stream order. A track in sourceInfo.audioTracks but not here exists in the container and cannot be played by this build — worth showing disabled, not hiding.

selectedAudioStreamIndex source

var selectedAudioStreamIndex: Int? { get }

The source stream index of the audio track currently feeding the renderer — the selection a host's audio menu marks. nil when the source has no decodable audio (or before load).

selectableSubtitleTracks source

var selectableSubtitleTracks: [SubtitleTrackInfo] { get }

Embedded text tracks supported by the same converter as HLS. Bitmap tracks remain in sourceInfo but are not selectable on this surface.

selectedSubtitleStreamIndex source

var selectedSubtitleStreamIndex: Int? { get }

nil means Off, including initially. Stream indices belong to the source.

activeSubtitleCues source

var activeSubtitleCues: [TimedTextCue] { get }

Poll alongside currentTime to draw (or clear) a host subtitle overlay. Times use the source axis, like this pipeline's clock. Text is WebVTT payload with optional inline tags, not an attributed string. Reading this never waits for demux I/O. Cues expire even when no more packets arrive. Already shifted by subtitleDelaySeconds.

subtitleDelaySeconds source

var subtitleDelaySeconds: Double { get }

The offset activeSubtitleCues is shifted by, in seconds. Positive values show text later. Range: -10...10 seconds.

audioDelaySeconds source

var audioDelaySeconds: Double { get }

The offset audio is being enqueued with, in seconds. Positive values present audio later. Range: -2...2 seconds. Changeable while playing — see setAudioDelaySeconds(_:completion:).

Under stateLock rather than feed-queue confined: the feed queue writes it, and the host reads it from wherever its lip-sync control lives.

audioDelivery source

var audioDelivery: AudioDelivery { get }

No documentation comment in the source.

failureError source

var failureError: (any Error)? { get }

The terminal error, if any. Polled by a host that saw .failed.

Instance methods

setSubtitleDelaySeconds(_:)source

func setSubtitleDelaySeconds(_ seconds: Double)

Shift subtitle text against the picture. Unlike the audio offset this needs no flush: the store keeps cues on the source axis and the shift is applied when they are read, so the next activeSubtitleCues poll already carries it. A non-finite value becomes zero.

A negative offset shows a cue before the demuxer's look-ahead may have reached it; text further ahead than that appears late until the feed catches up.

selectSubtitleTrack(streamIndex:completion:)source

func selectSubtitleTrack(streamIndex: Int?, completion: (@Sendable (Bool) -> Void)? = nil)

Select embedded text or nil for Off. Conversion runs as packets arrive for every supported text track, so switching needs no A/V flush or seek, including while paused. Completion runs on the feed queue; dispatch to the main queue for UI work and do not call synchronous load/stop from it.

load(url:httpHeaders:startAt:coordinatedHTTP:)source

func load(url: URL, httpHeaders: [String : String] = [:], startAt: CMTime? = nil, coordinatedHTTP: Bool = false) throws

Open the source, pick streams, build decoders, and decode enough to park the master clock on the first frame. Synchronous — call it off the main thread, like HLSRemuxer.run().

The open is bounded (SourceOpenTuning.probeBudget): a server that accepts the connection and then starves the reads would otherwise pin the caller inside avformat_open_input for as long as the socket lives, with no error and no way for the host to give up. On expiry this throws.

Parameter startAt: where playback should begin, on the source's own axis. Seeked BEFORE the first frame is decoded: a resume that loads at the head and then seeks pays for a keyframe's worth of decode it throws straight away, and shows the wrong picture for that long. nil (or zero) starts at the head.

load(probed:startAt:)source

func load(probed: ProbedSource, startAt: CMTime? = nil) throws

Adopt the context a routing probe already opened, instead of opening the source a second time. The same handover PrismCoreSession(probed:) does, for the same reason: over a network the second open is a real round trip and a real download inside the wait the user is watching — and the context has to travel, not merely the probe's conclusions, because avformat_find_stream_info fills state the decoders need (see ProbedSource).

A ProbedSource whose context was already taken (handed to a session that then declined, say) falls back to a fresh open of probed.url, so the call never fails for a reason the host cannot see.

play()source

func play()

No documentation comment in the source.

pause()source

func pause()

No documentation comment in the source.

seek(to:)source

func seek(to time: CMTime)

Seek to time: libavformat lands on the keyframe at or before it, the decoders are flushed, and frames between the keyframe and the target are decoded but discarded (up to Pacing.maxSeekDiscardSeconds of them), so the clock re-anchors on the first frame at or after time. Beyond that window — a very long GOP — playback resumes at the keyframe instead, which is late by less than showing nothing while a whole GOP decodes.

Seeks coalesce: a scrub that queues several before the first has run executes only the newest, because each earlier one would flush and refill a queue the next one flushes again.

setAudioDelaySeconds(_:completion:)source

func setAudioDelaySeconds(_ seconds: Double, completion: (@Sendable (Bool) -> Void)? = nil)

Change the audio offset while the title is playing — lip-sync correction is something a viewer turns with the picture in front of them, not a value chosen before the first frame.

Clamped to +/-2 s; a non-finite value becomes zero. Video, subtitles and the clock are untouched, and the rate never changes.

When it takes effect. On this path, at once: the offset is applied where a decoded buffer is handed to the renderer, so nothing on disk or in a container constrains it. What the renderer already holds was shifted by the OLD value, so the call flushes it and refills from the source at the playhead — the same move a track switch makes. The audible cost is a gap of decode-to-playhead time, not a re-buffer; audioDelaySeconds reports the new value as soon as the call has run, and it is then true of everything the renderer holds.

Works while paused (the new offset is primed at the paused position) and refuses only what a track switch refuses: a pipeline that is not in a playable state.

Parameter completion: called on the feed queue with whether the offset changed. Optional — a slider can fire and forget and read audioDelaySeconds for the settled answer.

selectAudioTrack(streamIndex:completion:)source

func selectAudioTrack(streamIndex: Int, completion: (@Sendable (Bool) -> Void)? = nil)

Switch the audio to another of the source's tracks, mid-playback, without touching the clock or the video renderer (issue #35).

streamIndex is the source stream index, i.e. AudioTrackInfo.streamIndex from selectableAudioTracks. Selecting the already-playing track is a no-op that still reports success. The switch is refused (completion(false)) when the pipeline isn't in a playable state, the index isn't a selectable audio stream, or the new track's decoder fails to open — in every refusal the current track keeps playing, because the old decoder is only torn down after the new one stands.

The audible gap depends on transport and decode-to-playhead time. The clock continues running and the video renderer keeps its queued frames. Works identically while paused — the new track is primed at the paused position and plays on play().

Parameter completion: called on the feed queue with whether the switch happened. Optional — fire-and-forget is fine for a menu tap; read selectedAudioStreamIndex for the settled answer.

stop()source