How to build a 360° video player with Three.js & Transloadit

Joseph Grabski

Content Lead · Rochester, United Kingdom ·

Capturing your memories in 360° video lets you re-experience them in a whole new dimension. VR is still a niche technology, so instead we will create a video player that lets users upload their own 360° videos and watch them from their laptop, powered by Three.js, Uppy and, of course, Transloadit!

A street-wear inspired design with the text '360° Video' as the main headline.

Projecting the video

Naturally, the core of any video player is displaying a video. This is easy enough with a regular 2D video, but the <video> element is not going to cut it for our purposes. This is where Three.js comes in, letting us project an equirectangular video onto the internal surface of a sphere. Then, by positioning a camera within the sphere, we can look around as if we were really part of the scene. You can see this illustrated in the diagram below.

A vector diagram showcasing how the projection will work, with the camera in the middle of a sphere.

To create this in Three.js, we first need to create Scene, PerspectiveCamera, WebGLRenderer and OrbitControls objects.

Start with an HTML document containing a viewport meta tag. Add this import map in the document's head, before any module scripts. Both Three.js and its addon use the same pinned version.

<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<script type="importmap">
  {
    "imports": {
      "three": "https://cdn.jsdelivr.net/npm/three@0.177.0/build/three.module.js",
      "three/addons/": "https://cdn.jsdelivr.net/npm/three@0.177.0/examples/jsm/"
    }
  }
</script>

Combine the JavaScript blocks below into one <script type="module">, placed after the HTML controls from the upload section. OrbitControls is a separate module, not a property on the THREE object.

import * as THREE from 'three'
import { OrbitControls } from 'three/addons/controls/OrbitControls.js'

const scene = new THREE.Scene()
const camera = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, 1, 100)
const renderer = new THREE.WebGLRenderer()
const controls = new OrbitControls(camera, renderer.domElement)
const radius = 30
const status = document.getElementById('status')

Now, we can initialize the objects in our scene, starting by creating a video with various properties tweaked.

controls.enablePan = false
controls.enableZoom = false

// Points the camera at the horizon by default
// Also moves the camera away for the OrbitControls target slightly so it works
let spherical = new THREE.Spherical(1, Math.PI / 2, 0)
spherical.makeSafe()
camera.position.setFromSpherical(spherical)

const video = document.createElement('video')
video.loop = true
video.muted = true
video.playsInline = true
video.crossOrigin = 'anonymous'
function showVideoError() {
  status.textContent = 'The video could not be loaded. Check its format and cross-origin access headers.'
}
function playVideo() {
  return video.play().then(() => {
    status.textContent = 'Your video is ready.'
  }).catch((error) => {
    if (error.name === 'NotAllowedError') {
      status.textContent = 'Select Play to start the video.'
    } else if (error.name !== 'AbortError') {
      showVideoError()
    }
  })
}
video.addEventListener('error', showVideoError)
video.src = 'https://threejs.org/examples/textures/pano.mp4'
playVideo()
document.getElementById('play-button').addEventListener('click', () => {
  if (!video.paused) {
    video.pause()
    return
  }
  playVideo()
})

renderer.setPixelRatio(Math.min(window.devicePixelRatio, 2))
renderer.setSize(window.innerWidth, window.innerHeight)
document.body.appendChild(renderer.domElement)

const texture = new THREE.VideoTexture(video)
texture.colorSpace = THREE.SRGBColorSpace
const geometry = new THREE.SphereGeometry(radius, 48, 32)
geometry.scale(-1, 1, 1)
const material = new THREE.MeshBasicMaterial({ map: texture })

const sphere = new THREE.Mesh(geometry, material)
scene.add(sphere)

The initial clip is the sample panorama from the Three.js video example. Replace it with your own equirectangular video as needed. Remote video responses must allow cross-origin access so WebGL can use them as textures; setting crossOrigin alone does not grant it. This applies both to the initial clip and to the exported S3 result. Configure your bucket's CORS policy to allow reads from your site's origin.

You'll notice that nothing is visible yet, as we still need to trigger a render of our scene. Create an animate function, which will contain both the requestAnimationFrame and render functions.

function animate() {
  requestAnimationFrame(animate)

  renderer.render(scene, camera)
}

animate()

Et voilà! We can now see the video being played, and we can look around thanks to the OrbitControls addon. You may notice that if we resize the window, we encounter some odd scaling issues. Let's quickly fix that now.

window.addEventListener('resize', onWindowResize)

function onWindowResize() {
  camera.aspect = window.innerWidth / window.innerHeight
  camera.updateProjectionMatrix()

  renderer.setSize(window.innerWidth, window.innerHeight)
}

The above function is pretty simple. We listen for the window being resized. When it is, we'll update both the aspect ratio of our camera, as well as the size of the renderer, to match the new dimensions of the screen.

Adding zoom controls

Adding zoom is pretty simple. While, OrbitControls does offer this, it's not a perfect match for our video player, so let's implement it ourselves. Create an event listener and handler for the scroll wheel, then update the camera's FoV according to the distance scrolled, as well as refreshing the camera's projection matrix.

renderer.domElement.addEventListener('wheel', handleZoom)

function handleZoom(e) {
  camera.fov = THREE.MathUtils.clamp(camera.fov + e.deltaY / 10, 10, 100)
  camera.updateProjectionMatrix()
}

It's really as easy as that!

Uploading videos with Uppy

Let's expand around the player now, by allowing users to upload their own videos. First, we'll sketch out a basic UI.

<div class="floating-box">
  <h2 class="title">Upload a 360° file to view</h2>
  <button id="play-button" type="button">Play / pause</button>
  <p id="status" role="status"></p>
  <!--- from https://loading.io/css/ -->
  <div class="lds-ellipsis" id="loading-dots">
    <div></div>
    <div></div>
    <div></div>
    <div></div>
  </div>
  <div id="container">
    <label for="file-input">Equirectangular video</label>
    <input type="file" id="file-input" accept="video/*" required />
    <button id="submit-button">Submit</button>
  </div>
</div>

body {
  margin: 0;
}

canvas {
  display: block;
}

.floating-box {
  position: absolute;
  z-index: 10;
  top: 16px;
  left: 16px;
  box-sizing: border-box;
  width: min(320px, calc(100vw - 32px));
  max-height: calc(100vh - 32px);
  overflow: auto;
  background: #fff;
  display: flex;
  flex-direction: column;
  padding: 20px;
  align-items: center;
  padding-top: 0px;
  justify-items: center;
  font-family: Helvetica;
}

#container {
  display: flex;
  flex-direction: column;
  gap: 8px;
}

.lds-ellipsis {
  display: inline-block;
  position: relative;
  width: 80px;
  height: 80px;
}
.lds-ellipsis div {
  position: absolute;
  top: 33px;
  width: 13px;
  height: 13px;
  border-radius: 50%;
  background: #000;
  animation-timing-function: cubic-bezier(0, 1, 1, 0);
}
.lds-ellipsis div:nth-child(1) {
  left: 8px;
  animation: lds-ellipsis1 0.6s infinite;
}
.lds-ellipsis div:nth-child(2) {
  left: 8px;
  animation: lds-ellipsis2 0.6s infinite;
}
.lds-ellipsis div:nth-child(3) {
  left: 32px;
  animation: lds-ellipsis2 0.6s infinite;
}
.lds-ellipsis div:nth-child(4) {
  left: 56px;
  animation: lds-ellipsis3 0.6s infinite;
}
@keyframes lds-ellipsis1 {
  0% {
    transform: scale(0);
  }
  100% {
    transform: scale(1);
  }
}
@keyframes lds-ellipsis3 {
  0% {
    transform: scale(1);
  }
  100% {
    transform: scale(0);
  }
}
@keyframes lds-ellipsis2 {
  0% {
    transform: translate(0, 0);
  }
  100% {
    transform: translate(24px, 0);
  }
}

We can now throw Uppy into the mix, by first importing it and then initializing the Transloadit plugin.

Add this import and the following upload code to the same module script as the Three.js code above. Separate module scripts do not share the video and status-element variables.

import { Uppy, Transloadit } from 'https://releases.transloadit.com/uppy/v3.13.0/uppy.min.mjs'

const uppy = new Uppy({
  restrictions: {
    allowedFileTypes: ['video/*'],
    maxNumberOfFiles: 1,
  },
  autoProceed: false,
})
uppy.use(Transloadit, {
  waitForEncoding: true,
  assemblyOptions: {
    params: {
      auth: { key: 'AUTH_KEY' },
      template_id: 'TEMPLATE_ID',
    },
  },
})
uppy.on('transloadit:complete', setSrc)

Be sure to fill in the AUTH_KEY and TEMPLATE_ID fields, with their values from the Transloadit console.

Before deploying publicly, use server-generated signatures and authorize uploads on your server. Never include an Auth Secret in browser code.

Then, we'll need to add an event listener to the submit button, triggering the upload to Transloadit when it's clicked. Also, create variables for the button container and the loading dots, making sure to initially hide the loading dots.

const submit = document.getElementById('submit-button')
submit.addEventListener('click', uploadVideo)

const container = document.getElementById('container')

const loadingDots = document.getElementById('loading-dots')
loadingDots.style.display = 'none'

async function uploadVideo() {
  const input = document.getElementById('file-input')
  const file = input.files[0] // should only ever be one
  if (file == null) return
  uppy.cancelAll()
  container.style.display = 'none'
  loadingDots.style.display = 'block'
  submit.disabled = true
  status.textContent = 'Processing video.'
  try {
    uppy.addFile({ name: file.name, type: file.type, data: file })
    const result = await uppy.upload()
    if (result.failed.length > 0) throw new Error('Video upload failed.')
  } catch {
    status.textContent = 'Video processing failed. Please try again.'
  } finally {
    container.style.display = 'flex'
    loadingDots.style.display = 'none'
    submit.disabled = false
    input.value = ''
  }
}

You may have also noticed earlier that we referenced the setSrc function, as part of the transloadit:complete event. This function will simply take the video URL from our Assembly Status JSON, and update the video element with this new source. Let's create it now.

function setSrc(assembly) {
  const url = assembly.results.exported?.[0]?.ssl_url
  if (typeof url !== 'string') {
    status.textContent = 'No video was returned. Please try again.'
    return
  }
  video.src = url
  video.load()
  status.textContent = 'Loading video.'
  playVideo()
}

Make sure to call video.load() and video.play(), as this causes the element to reload the source, thereby updating the video within our player.

Crafting a Template with Transloadit

After all that, our file upload still won't work, however. The final piece of the puzzle is to create a Template, which will process the uploaded video and return it back to our client.

{
  "steps": {
    ":original": {
      "robot": "/upload/handle"
    },
    "encode_360": {
      "use": ":original",
      "robot": "/video/encode",
      "ffmpeg_stack": "v7",
      "preset": "webm"
    },
    "exported": {
      "use": ["encode_360"],
      "robot": "/s3/store",
      "credentials": "YOUR_AWS_CREDENTIALS",
      "result": true
    }
  }
}

The Template above transcodes the uploaded video into a uniform format, and then exports it to S3 for long-term storage, since Transloadit will only temporarily store files for 24 hours before deleting them. Replace YOUR_AWS_CREDENTIALS with the name of your saved Template Credentials; do not paste S3 secrets into the example.

Drawing to a close

Now that all the pieces of our jigsaw are in place, your 360° video player should be working beautifully. Be sure to show us how you got along on X (Twitter), or tell us about something else you've made using Transloadit!