Multimodal media

useAudio, useImages, useVideo, and useFiles assemble multimodal blocks from the protocol stream. Each hook returns an array of media handles (one per message containing a matching block in the target namespace).

Runnable example: The multimodal storybook in the streaming cookbook streams story text, generated images, audio narration, and video from scoped graph nodes.

Handle shape

Every media handle exposes the same core fields, plus media-specific extras:

Field	Type	Description
id	string	Stable id for the media block.
messageId	string	Owning message id.
partialBytes	Uint8Array \| undefined	Live-growing byte buffer during streaming.
blob	Blob \| undefined	Settled Blob — populated on message-finish.
objectURL	string \| undefined	Object URL for the settled Blob (auto-revoked).
error	unknown	Fail-loud error surface.
mimeType	string \| undefined	Declared MIME type.

Media-specific extras:

Hook	Extras
useAudio	transcript?: string, duration?: number, sampleRate?: number
useImages	width?: number, height?: number
useVideo	duration?: number, width?: number, height?: number
useFiles	filename?: string, size?: number

Media hooks

import { useAudio, useImages, useVideo, useFiles } from "@langchain/vue";

const audios = useAudio(stream);
const images = useImages(stream);
const videos = useVideo(stream);
const files = useFiles(stream);

Each accepts an optional target argument (subagent / subgraph / { namespace }) to scope the subscription — see Selectors.

useMediaURL

useMediaURL turns a media handle into a stable blob URL you can pass to <audio/img/video src>. It owns the object URL lifecycle: URLs are created on demand and revoked when the host Vue scope unmounts or the input media changes.

<script setup lang="ts">
import { computed } from "vue";
import { useAudio, useAudioPlayer, useMediaURL } from "@langchain/vue";

const audio = useAudio(stream);
const latest = computed(() => audio.value.at(-1));
const player = useAudioPlayer(latest);
const downloadUrl = useMediaURL(latest);
</script>

<template>
  <button v-if="latest" @click="player.status === 'playing' ? player.pause() : player.play()">
    {{ player.status === "playing" ? "Pause" : "Play" }}
  </button>
  <a v-if="downloadUrl" :href="downloadUrl" download> Download </a>
</template>

useMediaURL(undefined) returns undefined — safe to call unconditionally.

useAudioPlayer / useVideoPlayer

useAudioPlayer and useVideoPlayer provide opinionated player handles with play/pause/seek state, built on top of the media hooks:

<script setup lang="ts">
import { computed } from "vue";
import { useAudio, useAudioPlayer } from "@langchain/vue";

const audio = useAudio(stream);
const latest = computed(() => audio.value.at(-1));
const player = useAudioPlayer(latest, { autoPlay: true });
</script>

<template>
  <div>
    <button @click="player.toggle">
      {{ player.isPlaying ? "Pause" : "Play" }}
    </button>
    <progress :value="player.currentTime" :max="player.duration" />
  </div>
</template>

useVideoPlayer mirrors the same shape for video handles.

API reference