Skip to main content

Using a video as a texture in Three.js

If you want to embed a video as a texture in Three.js, you have multiple options.

Using @remotion/mediav4.0.387

We recommend that you mount a <Video> tag in headless mode and use the onVideoFrame prop to update the Three.js texture whenever a new frame is being drawn.

VideoTexture.tsx
tsx
import {useThree} from'@react-three/fiber';
import {Video} from'@remotion/media';
import {ThreeCanvas} from'@remotion/three';
import React, {useCallback, useState} from'react';
import {useVideoConfig} from'remotion';
import {CanvasTexture} from'three';
constvideoSrc='https://remotion.media/video.mp4';
constvideoWidth=1920;
constvideoHeight=1080;
constaspectRatio= videoWidth / videoHeight;
constscale=3;
constplaneHeight= scale;
constplaneWidth= aspectRatio * scale;
constInner:React.FC= () => {
const [canvasStuff] =useState(() => {
constcanvas=newOffscreenCanvas(videoWidth, videoHeight);
constcontext= canvas.getContext('2d')!;
consttexture=newCanvasTexture(canvas);
return {canvas, context, texture};
});
const {invalidate} =useThree();
constonVideoFrame=useCallback(
(frame:CanvasImageSource) => {
canvasStuff.context.drawImage(frame, 0, 0, videoWidth, videoHeight);
// This is needed during preview - the frame loop synchronizes with the monitor refresh rate
// By setting this, we signal that the texture has been updated
canvasStuff.texture.needsUpdate =true;
// This is needed during rendering - the frame loop is only triggered on demand.
// We have to call "invalidate()" to trigger a new frame loop.
invalidate();
},
[canvasStuff.context, canvasStuff.texture, invalidate],
);
return (
<>
<Videosrc={videoSrc} onVideoFrame={onVideoFrame} mutedheadless />
<mesh>
<planeGeometryargs={[planeWidth, planeHeight]} />
<meshBasicMaterialcolor={0xffffff} toneMapped={false} map={canvasStuff.texture} />
</mesh>
</>
);
};
exportconstRemotionMediaVideoTexture:React.FC= () => {
const {width, height} =useVideoConfig();
return (
<ThreeCanvasstyle={{backgroundColor: 'white'}} linearwidth={width} height={height}>
<Inner />
</ThreeCanvas>
);
};

Notes

  • By using the headless prop, nothing will be returned by the <Video> tag, so it can be mounted within a <ThreeCanvas> without affecting the rendering.
  • For preview and rendering, separate approaches are needed to invalida the texture when it is being updated. See

Examples

Using <OffthreadVideo>

deprecated in favor of using @remotion/media

You can use the useOffthreadVideoTexture() hook from @remotion/three to get a texture from a video.

Drawbacks:

  • It requires the whole video to be downloaded to disk first before frames can be extracted
  • It does not work in client-side rendering
  • It creates a new texture for each frame, which is less efficient than using @remotion/media

This API is therefore deprecated in favor of using the recommended approach mentioned above.

Using <Html5Video>

deprecated in favor of using @remotion/media

You can use the useVideoTexture() hook from @remotion/three to get a texture from a video.

It has all the drawbacks of the <Html5Video> tag and is therefore deprecated in favor of using the recommended approach mentioned above.

See also

AltStyle によって変換されたページ (->オリジナル) /