Multimodal media

injectAudio, injectImages, injectVideo, and injectFiles assemble multimodal blocks from the protocol stream. Each hook returns an array of media handles (one per message containing a matching block in the target namespace).

Runnable example: The multimodal storybook in the streaming cookbook streams story text, generated images, audio narration, and video from scoped graph nodes.

Handle shape

Every media handle exposes the same core fields, plus media-specific extras:

Field	Type	Description
id	string	Stable id for the media block.
messageId	string	Owning message id.
partialBytes	Uint8Array \| undefined	Live-growing byte buffer during streaming.
blob	Blob \| undefined	Settled Blob — populated on message-finish.
objectURL	string \| undefined	Object URL for the settled Blob (auto-revoked).
error	unknown	Fail-loud error surface.
mimeType	string \| undefined	Declared MIME type.

Media-specific extras:

Hook	Extras
injectAudio	transcript?: string, duration?: number, sampleRate?: number
injectImages	width?: number, height?: number
injectVideo	duration?: number, width?: number, height?: number
injectFiles	filename?: string, size?: number

Media hooks

import { injectAudio, injectImages, injectVideo, injectFiles } from "@langchain/angular";

const audios = injectAudio(stream);
const images = injectImages(stream);
const videos = injectVideo(stream);
const files = injectFiles(stream);

Each accepts an optional target argument (subagent / subgraph / { namespace }) to scope the subscription — see Selectors.

injectMediaUrl

injectMediaUrl turns a media handle into a stable blob URL you can pass to <audio/img/video src>:

import { injectAudio, injectMediaUrl } from "@langchain/angular";

function AudioReply({ stream }: { stream: AnyStream }) {
  const audios = injectAudio(stream);
  const latest = audios.at(-1);
  const url = injectMediaUrl(latest);
  return url ? <audio controls src={url} /> : null;
}

injectMediaUrl(undefined) returns undefined — safe to call unconditionally.

injectMediaUrl / injectMediaUrl

injectMediaUrl and injectMediaUrl provide opinionated player handles with play/pause/seek state, built on top of the media hooks:

import { injectAudio, injectMediaUrl } from "@langchain/angular";

function AudioPlayer({ stream }: { stream: AnyStream }) {
  const audio = injectAudio(stream).at(-1);
  const player = injectMediaUrl(audio, { autoPlay: true });

  return (
    <div>
      <button onClick={player.toggle}>{player.isPlaying ? "Pause" : "Play"}</button>
      <progress value={player.currentTime} max={player.duration} />
    </div>
  );
}

injectMediaUrl mirrors the same shape for video handles.

API reference