Chunked browser uploads to Google storage with JavaScript

Tim Koschützki

Co-founder · Berlin, Germany ·

Google Cloud Storage (GCS) supports uploading a single object in multiple requests through its resumable upload protocol. This tutorial builds a local, single-owner upload tool: Node.js authenticates the owner and creates a session, then the browser sends file chunks directly to GCS. It supports files up to 10 GiB, progress after each acknowledged chunk, and pause/resume while the page stays open.

Why chunked uploads?

A resumable upload tracks the bytes GCS has received. After a connection failure, the browser asks for that position and continues from there. The chunks belong to one object; there are no temporary objects to compose or delete, and no 32-component compose limit.

Configure Google Cloud Storage

Use Node.js 24.2 or newer, Yarn, the Google Cloud CLI, and a private bucket. Give the server’s identity the bucket-scoped roles/storage.objectCreator role: this example creates new objects only. For local development, configure Application Default Credentials with an identity that has that permission:

gcloud auth application-default login
mkdir gcs-upload
cd gcs-upload
corepack yarn init
corepack yarn config set nodeLinker node-modules
corepack yarn add --exact google-auth-library@11.0.0

The JSON API handles CORS independently of bucket CORS rules. The server includes the browser’s exact origin when creating the session so subsequent browser requests receive CORS headers. A CORS setting does not authenticate an uploader.

Back-end: create authenticated upload sessions

Save this as server.mjs. The configured token identifies the one owner of this local tool. Every session requires that token, and the server chooses an unpredictable object name inside uploads/owner/. Clients cannot supply bucket names, destination paths, or object names to delete. The ifGenerationMatch=0 precondition also prevents overwriting an existing object.

import { createHash, randomUUID, timingSafeEqual } from 'node:crypto'
import { readFile } from 'node:fs/promises'
import { createServer } from 'node:http'
import { GoogleAuth } from 'google-auth-library'

const ORIGIN = 'http://127.0.0.1:8080'
const MAX_SIZE = 10 * 1024 ** 3
const assets = new Map([
  ['/', ['index.html', 'text/html; charset=utf-8']],
  ['/upload.mjs', ['upload.mjs', 'text/javascript; charset=utf-8']],
  ['/app.mjs', ['app.mjs', 'text/javascript; charset=utf-8']],
])

export function createUploadServer({ auth, bucket, token }) {
  if (!bucket || !token || token.length < 32) throw new Error('Set BUCKET and a 32-character token')
  const digest = (value) => createHash('sha256').update(value).digest()
  const expected = digest(`Bearer ${token}`)

  return createServer((req, res) => {
    res.setHeader('Cache-Control', 'no-store')
    res.setHeader('Referrer-Policy', 'no-referrer')
    const reply = (status, body) => {
      res.writeHead(status, { 'Content-Type': 'application/json' })
      res.end(JSON.stringify(body))
    }
    async function handle() {
      const asset = assets.get(req.url)
      if (req.method === 'GET' && asset) {
        const body = await readFile(new URL(asset[0], import.meta.url))
        res.writeHead(200, { 'Content-Type': asset[1] })
        res.end(body)
        return
      }
      if (req.method !== 'POST' || req.url !== '/sessions') return reply(404, { error: 'Not found' })
      if (!timingSafeEqual(expected, digest(req.headers.authorization ?? ''))) {
        return reply(401, { error: 'Invalid upload token' })
      }
      if (req.headers.origin !== ORIGIN) return reply(403, { error: 'Invalid origin' })
      const chunks = []
      let bytes = 0
      for await (const chunk of req) {
        bytes += chunk.length
        if (bytes > 1024) return reply(413, { error: 'Request too large' })
        chunks.push(chunk)
      }
      let input
      try {
        input = JSON.parse(Buffer.concat(chunks, bytes).toString('utf8'))
      } catch {
        return reply(400, { error: 'Invalid JSON' })
      }
      const size = input?.size
      if (!Number.isSafeInteger(size) || size < 1 || size > MAX_SIZE) {
        return reply(400, { error: 'Choose a nonempty file of at most 10 GiB' })
      }
      const name = `uploads/owner/${randomUUID()}`
      const url = new URL(`https://storage.googleapis.com/upload/storage/v1/b/${encodeURIComponent(bucket)}/o`)
      url.search = new URLSearchParams({ uploadType: 'resumable', name, ifGenerationMatch: '0' })
      const client = await auth.getClient()
      const response = await client.request({
        url: url.href,
        method: 'POST',
        headers: { Origin: ORIGIN, 'X-Upload-Content-Length': String(size) },
        data: { contentType: 'application/octet-stream' },
        timeout: 30_000,
        retry: false,
      })
      const sessionUrl = response.headers.get('location')
      if (!sessionUrl || new URL(sessionUrl).origin !== 'https://storage.googleapis.com') {
        throw new Error('Invalid session response')
      }
      reply(201, { sessionUrl, name })
    }
    handle().catch(() => reply(502, { error: 'Could not create upload session' }))
  })
}

if (import.meta.main) {
  const auth = new GoogleAuth({ scopes: ['https://www.googleapis.com/auth/devstorage.read_write'] })
  createUploadServer({ auth, bucket: process.env.BUCKET, token: process.env.UPLOAD_TOKEN })
    .listen(8080, '127.0.0.1', () => console.log(`Open ${ORIGIN}`))
}

The Google Auth Library keeps Google credentials on the server. The returned session URL is itself a bearer credential: anyone who has it can upload to that one destination. It is not a 15-minute signed URL; GCS resumable sessions expire after one week. Keep session URLs out of logs and shared storage.

Front-end: a minimal chunked uploader

Save this as upload.mjs. The 8 MiB chunk size is a multiple of GCS’s required 256 KiB alignment; the last chunk can be smaller. A 308 response means the upload is incomplete, so the code handles it before the ordinary HTTP error check. It reads the acknowledged byte range instead of assuming GCS accepted the entire request. See the resumable upload protocol.

const CHUNK_SIZE = 8 * 1024 * 1024

export async function uploadFileWithChunks(file, sessionUrl, { signal, onProgress = () => {} } = {}) {
  if (file.size < 1 || file.size > 10 * 1024 ** 3) throw new Error('Invalid file size')
  let offset = 0
  let probe = true
  let failures = 0
  while (true) {
    signal?.throwIfAborted()
    const end = Math.min(offset + CHUNK_SIZE, file.size)
    let response
    try {
      response = await fetch(sessionUrl, {
        method: 'PUT',
        credentials: 'omit',
        signal,
        headers: {
          'Content-Range': probe ? `bytes */${file.size}` : `bytes ${offset}-${end - 1}/${file.size}`,
        },
        body: probe ? new Blob([]) : file.slice(offset, end),
      })
    } catch (error) {
      signal?.throwIfAborted()
      if (++failures > 5) throw error
      await backoff(failures, signal)
      probe = true
      continue
    }
    await response.body?.cancel()
    if (response.status === 200 || response.status === 201) {
      onProgress(100)
      return
    }
    if (response.status === 429 || response.status >= 500) {
      if (++failures > 5) throw new Error('Upload retries exhausted')
      await backoff(failures, signal)
      probe = true
      continue
    }
    if (response.status !== 308) throw new Error(`Upload failed (HTTP ${response.status})`)
    const range = response.headers.get('range')
    const match = range === null ? null : /^bytes=0-(\d+)$/.exec(range)
    if (range !== null && !match) throw new Error('Invalid acknowledged range')
    const next = match ? Number(match[1]) + 1 : 0
    if (!Number.isSafeInteger(next) || next < offset || next >= file.size || (!probe && next > end)) {
      throw new Error('Invalid acknowledged position')
    }
    if (!probe && next === offset) {
      if (++failures > 5) throw new Error('Upload made no progress')
      await backoff(failures, signal)
      probe = true
      continue
    }
    if (next > offset) failures = 0
    offset = next
    onProgress((offset / file.size) * 100)
    probe = false
  }
}

function backoff(attempt, signal) {
  signal?.throwIfAborted()
  return new Promise((resolve, reject) => {
    const abort = () => {
      clearTimeout(timer)
      signal.removeEventListener('abort', abort)
      reject(signal.reason)
    }
    const timer = setTimeout(() => {
      signal?.removeEventListener('abort', abort)
      resolve()
    }, 500 * 2 ** (attempt - 1))
    signal?.addEventListener('abort', abort, { once: true })
  })
}

Retry with exponential back-off

Network errors, rate limits, and server errors trigger up to five retries between progress updates. Each retry first queries the stored position with an empty PUT. A pause aborts both the active request and any retry delay. Permission failures, expired sessions, and malformed ranges stop the upload instead of looping forever. If a final response was lost, the next status probe can confirm completion without uploading the file again.

Save the page as index.html:

<!doctype html>
<html lang="en">
  <meta charset="utf-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1" />
  <title>Resumable GCS upload</title>
  <label>Upload token <input id="token" type="password" autocomplete="off" /></label>
  <label>File <input id="file" type="file" /></label>
  <button id="upload">Upload / resume</button>
  <button id="pause" disabled>Pause</button>
  <progress id="progress" max="100" value="0" aria-label="Upload progress"></progress>
  <p id="status" role="status">Choose a file.</p>
  <script type="module" src="/app.mjs"></script>
</html>

Save the controls as app.mjs:

import { uploadFileWithChunks } from './upload.mjs'

const fileInput = document.getElementById('file')
const tokenInput = document.getElementById('token')
const upload = document.getElementById('upload')
const pause = document.getElementById('pause')
const progress = document.getElementById('progress')
const status = document.getElementById('status')
let session
let controller

fileInput.addEventListener('change', () => {
  session = undefined
  progress.value = 0
})
pause.addEventListener('click', () => controller?.abort())
upload.addEventListener('click', async () => {
  const file = fileInput.files[0]
  if (!file) return
  controller = new AbortController()
  upload.disabled = fileInput.disabled = tokenInput.disabled = true
  pause.disabled = false
  status.textContent = 'Uploading…'
  try {
    if (!session) {
      const response = await fetch('/sessions', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${tokenInput.value}` },
        body: JSON.stringify({ size: file.size }),
        signal: controller.signal,
      })
      if (!response.ok) throw new Error(`Session failed (HTTP ${response.status})`)
      session = await response.json()
    }
    await uploadFileWithChunks(file, session.sessionUrl, {
      signal: controller.signal,
      onProgress: (value) => { progress.value = value },
    })
    status.textContent = `Uploaded to ${session.name}`
    session = undefined
  } catch {
    status.textContent = controller.signal.aborted
      ? 'Paused. Click Upload / resume to continue.'
      : 'Upload failed. Retry, or reselect the file to start a new session.'
  } finally {
    upload.disabled = fileInput.disabled = tokenInput.disabled = false
    pause.disabled = true
  }
})

Start the server in the same directory after setting your existing bucket name and entering a secret token of at least 32 characters. Enter the same token in the page at http://127.0.0.1:8080:

export BUCKET='your-existing-private-bucket'
read -r -s -p 'Upload token: ' UPLOAD_TOKEN
export UPLOAD_TOKEN
node server.mjs

Persist progress across reloads

This example deliberately retains the file and session only in memory. Pausing keeps both; reloading the page or selecting another file starts a new session. Abandoned, incomplete sessions expire without creating temporary chunk objects.

To support reloads in a multi-user application, store sessions on the server against authenticated user IDs and return a session only to its owner. Require the user to reselect the same file and verify its content identity, not just its name and size, before resuming. Always ask GCS for the offset; a locally saved progress percentage is not authoritative.

Security and performance best practices

The token is authentication for a single trusted owner, not a multi-user login system. Keep this demo bound to loopback. Before deploying it, integrate your application’s authentication and quotas, serve it over HTTPS, and keep the object namespace tied to the authenticated user. Treat uploaded bytes as untrusted; a file’s claimed content type does not validate its contents.

The example limits each session to 10 GiB and requests that length when initiating the upload. Keep the bucket private and verify completed object metadata before making a file available downstream. The browser reads only one chunk at a time; progress updates report acknowledged bytes, not bytes still in flight. GCS stores the final object directly, so no cleanup endpoint needs arbitrary delete access.

Transloadit can help, too

For processing files after upload, Transloadit’s 🤖 /google/import Robot can import them using Template Credentials. Uppy provides upload interfaces for applications that need more than this minimal demo.