Resilient file uploads using background sync
Save the file before trying to send it. This walkthrough queues a small file in IndexedDB, keeps it through an offline page reload, and delivers it to a local authenticated receiver. A stable key and a receipt tied to the file’s bytes make retrying safe when the server saved the upload but its response never arrived.
Background Sync gives a service worker another opportunity to retry. It requires a secure context and has limited browser availability. It does not guarantee delivery after you close the browser. Keep the foreground retry button, and remember that browser storage can be evicted.
Understanding chunked uploads
This example retries a whole file: one byte through 10 MiB, at most 10 queued files across accounts, and a 24-hour retry window. Chunking reduces retransmission after an interruption but does not preserve a file held only in page memory. For large files, tus negotiates offsets; you still need to retain the original bytes and authenticate the server.
The resumable client walkthrough covers that separate task. Its server contract differs from the raw-byte receiver below.
Fetch already runs network I/O asynchronously. Here, a Web Lock serializes page and service-worker retries across the origin. Dedicated Web Workers would not add persistence or make server publication atomic.
Set up the local project
Use Node.js 24.15.0 with Corepack available, Yarn 4.12.0, and a browser with IndexedDB, Web Locks, and service workers. The browser path was tested with Chromium 145. Use the exact loopback origin printed by the server; HTTP on loopback is a secure context. Deployment requires HTTPS and your application’s authentication.
Paste this into Bash from the parent directory where you want the demo. It refuses an existing project directory and leaves your shell in its original directory on success or failure. The local lockfile and explicit bundler configuration isolate it from an enclosing project.
(
mkdir background-upload-demo &&
cd background-upload-demo &&
printf '%s\n' '{"name":"background-upload-demo","private":true,"type":"module","packageManager":"yarn@4.12.0"}' > package.json &&
printf '%s\n' 'nodeLinker: node-modules' > .yarnrc.yml &&
touch yarn.lock &&
corepack yarn add --exact idb@8.0.3 esbuild@0.27.3
)
Save the following files inside background-upload-demo. Create its public directory when saving
public/index.html. No existing application, cloud account, or upload endpoint is required.
Run a receiver that commits bytes and receipts together
Save server.mts. It listens only on loopback and prints fresh passwords for the two demonstration
accounts at startup. Login creates an HttpOnly cookie session. Mutation requests must also match
the origin, session owner, and CSRF token. The browser stores only an account ID with its queue;
it fetches the current session’s token for each retry.
The receiver stores the BLOB, size, SHA-256 digest, key, and owner in one SQLite row. A transaction serializes quota checking and insertion. It returns 201 after the commit, or 200 with the same receipt on an identical replay. Changed bytes or timestamps receive 409. Expired keys receive 410, including when a replay tries to supply a newer timestamp. This demo retains rows permanently, so receipt cleanup cannot accidentally make a previously accepted key new again.
SQLite’s atomic commit documentation explains the storage boundary. This uses Node’s built-in SQLite API, which is a release candidate in the tested Node version. Persistence assumes a working local disk; it is not a replicated storage service.
import { createHash, randomUUID } from 'node:crypto'
import { readFile } from 'node:fs/promises'
import { createServer, type IncomingMessage, type ServerResponse } from 'node:http'
import { DatabaseSync } from 'node:sqlite'
import { parseArgs } from 'node:util'
const { values } = parseArgs({ options: { port: { type: 'string', default: '8090' } } })
const port = Number(values.port)
if (!Number.isInteger(port) || port < 0 || port > 65535) throw new Error('Invalid port')
const maxSize = 10 * 1024 * 1024
const lifetime = 24 * 60 * 60 * 1000
const passwords = { alice: randomUUID(), bob: randomUUID() }
const uuid = /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/
let origin = ''
class HttpError extends Error {
status: number
constructor(status: number, message: string) { super(message); this.status = status }
}
async function body(request: IncomingMessage, limit: number): Promise<Buffer> {
const chunks: Buffer[] = []
let size = 0
for await (const chunk of request) {
if (!Buffer.isBuffer(chunk)) throw new HttpError(400, 'Invalid body')
size += chunk.length
if (size > limit) throw new HttpError(413, 'File too large')
chunks.push(chunk)
}
return Buffer.concat(chunks)
}
async function main(): Promise<void> {
const db = new DatabaseSync('uploads.sqlite')
db.exec(`
PRAGMA journal_mode = WAL;
PRAGMA synchronous = FULL;
PRAGMA busy_timeout = 5000;
CREATE TABLE IF NOT EXISTS sessions (
id TEXT PRIMARY KEY, owner TEXT NOT NULL, csrf TEXT NOT NULL, expires INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS uploads (
owner TEXT NOT NULL, id TEXT NOT NULL, created INTEGER NOT NULL,
sha256 TEXT NOT NULL, size INTEGER NOT NULL, bytes BLOB NOT NULL,
PRIMARY KEY (owner, id)
);
`)
const reply = (response: ServerResponse, status: number, value: unknown) => {
response.writeHead(status, { 'Content-Type': 'application/json' })
response.end(JSON.stringify(value))
}
async function handle(request: IncomingMessage, response: ServerResponse): Promise<void> {
response.setHeader('Cache-Control', 'no-store')
response.setHeader('X-Content-Type-Options', 'nosniff')
if (request.headers.host !== new URL(origin).host) throw new HttpError(403, 'Wrong host')
const path = new URL(request.url ?? '/', origin).pathname
if (request.method === 'POST' && request.headers.origin !== origin) {
throw new HttpError(403, 'Wrong origin')
}
if (path === '/login' && request.method === 'GET') {
response.writeHead(200, { 'Content-Type': 'text/html' })
response.end(`<!doctype html><html lang="en"><meta name="viewport" content="width=device-width">
<meta name="color-scheme" content="light dark"><title>Local upload login</title><form method="post" action="/login">
<label>Account <select name="owner"><option>alice</option><option>bob</option></select></label>
<label>Password <input name="password" type="password" autocomplete="current-password" required></label>
<button>Sign in</button></form></html>`)
return
}
if (path === '/login' && request.method === 'POST') {
const form = new URLSearchParams((await body(request, 2048)).toString('utf8'))
const owner = form.get('owner')
if ((owner !== 'alice' && owner !== 'bob') || form.get('password') !== passwords[owner]) {
throw new HttpError(401, 'Invalid local login')
}
const id = randomUUID()
db.prepare('INSERT INTO sessions VALUES (?, ?, ?, ?)').run(id, owner, randomUUID(), Date.now() + lifetime)
response.setHeader('Set-Cookie', `session=${id}; HttpOnly; SameSite=Strict; Path=/; Max-Age=86400`)
response.writeHead(303, { Location: '/' }).end()
return
}
const cookie = request.headers.cookie?.split(';').map(part => part.trim()).find(part => part.startsWith('session='))
const session = db.prepare('SELECT * FROM sessions WHERE id = ? AND expires > ?').get(cookie?.slice(8) ?? '', Date.now())
if (path.startsWith('/api/') || path === '/received') {
if (!session) throw new HttpError(401, 'Sign in again')
}
if (path === '/api/upload-session' && request.method === 'GET') {
reply(response, 200, { owner: session?.owner, csrf: session?.csrf })
return
}
if (path === '/api/queued-uploads' && request.method === 'POST') {
if (!session || request.headers['x-upload-owner'] !== session.owner || request.headers['x-csrf-token'] !== session.csrf) {
throw new HttpError(403, 'Owner or CSRF mismatch')
}
const id = request.headers['idempotency-key']
const created = Number(request.headers['x-upload-created'])
if (typeof id !== 'string' || !uuid.test(id) || !Number.isSafeInteger(created)) {
throw new HttpError(400, 'Invalid upload identity')
}
if (request.headers['content-type'] !== 'application/octet-stream') throw new HttpError(415, 'Send raw bytes')
const bytes = await body(request, maxSize)
if (bytes.length === 0) throw new HttpError(400, 'Choose a nonempty file')
const sha256 = createHash('sha256').update(bytes).digest('hex')
let status = 200
db.exec('BEGIN IMMEDIATE')
try {
const previous = db.prepare('SELECT * FROM uploads WHERE owner = ? AND id = ?').get(session.owner, id)
const originalCreated = previous?.created ?? created
if (typeof originalCreated !== 'number' || Date.now() - originalCreated >= lifetime) {
throw new HttpError(410, 'Upload expired')
}
if (created > Date.now() + 60_000) throw new HttpError(400, 'Clock is ahead')
if (previous) {
if (previous.sha256 !== sha256 || previous.size !== bytes.length || previous.created !== created) {
throw new HttpError(409, 'Key already belongs to different content')
}
} else {
const usage = db.prepare('SELECT COALESCE(SUM(size), 0) AS total FROM uploads WHERE owner = ?').get(session.owner)
if (typeof usage?.total !== 'number' || usage.total + bytes.length > 100 * 1024 * 1024) {
throw new HttpError(429, 'Local account quota reached')
}
db.prepare('INSERT INTO uploads VALUES (?, ?, ?, ?, ?, ?)').run(session.owner, id, created, sha256, bytes.length, bytes)
status = 201
}
db.exec('COMMIT')
} catch (error) {
db.exec('ROLLBACK')
throw error
}
reply(response, status, { id, owner: session.owner, size: bytes.length, sha256 })
return
}
if (path === '/received' && request.method === 'GET') {
const rows = db.prepare('SELECT id, size, sha256 FROM uploads WHERE owner = ? ORDER BY id').all(session?.owner ?? '')
response.writeHead(200, { 'Content-Type': 'text/html' })
response.end(`<!doctype html><html lang="en"><meta name="color-scheme" content="light dark"><title>Received uploads</title>
<h1>Received uploads</h1><p>${rows.length} stored file(s)</p><ul>${rows.map(row =>
`<li><a href="/api/files/${row.id}">${row.id}</a>: ${row.size} bytes, SHA-256 ${row.sha256}</li>`
).join('')}</ul><a href="/">Queue</a></html>`)
return
}
if (path.startsWith('/api/files/') && request.method === 'GET') {
const row = db.prepare('SELECT bytes FROM uploads WHERE owner = ? AND id = ?').get(session?.owner ?? '', path.slice(11))
if (!row || !(row.bytes instanceof Uint8Array)) throw new HttpError(404, 'File not found')
response.writeHead(200, { 'Content-Type': 'application/octet-stream', 'Content-Disposition': 'attachment; filename="upload.bin"' })
response.end(row.bytes)
return
}
const files: Record<string, [string, string]> = {
'/': ['index.html', 'text/html'],
'/upload-page.js': ['upload-page.js', 'text/javascript'],
'/upload-sw.js': ['upload-sw.js', 'text/javascript'],
}
const file = files[path]
if (request.method !== 'GET' || !file) throw new HttpError(404, 'Not found')
const content = await readFile(new URL(`./public/${file[0]}`, import.meta.url))
response.writeHead(200, { 'Content-Type': file[1] })
response.end(content)
}
const server = createServer((request, response) => {
void handle(request, response).catch(error => {
if (!(error instanceof HttpError)) console.error('Receiver request failed')
response.setHeader('Connection', 'close')
reply(response, error instanceof HttpError ? error.status : 500, {
error: error instanceof HttpError ? error.message : 'Upload not confirmed',
})
})
})
server.requestTimeout = 30_000
await new Promise<void>((resolve, reject) => {
server.once('error', reject)
server.listen(port, '127.0.0.1', resolve)
})
const address = server.address()
if (!address || typeof address === 'string') throw new Error('No listening address')
origin = `http://127.0.0.1:${address.port}`
console.log(JSON.stringify({ ready: origin, passwords }))
const stop = () => {
server.close(() => db.close())
server.closeAllConnections()
}
process.once('SIGINT', stop)
process.once('SIGTERM', stop)
}
main().catch(error => { console.error('Receiver startup failed:', error.message); process.exitCode = 1 })
Integrating background sync
Save queue.ts. Hashing happens before the insertion transaction, and fetching finishes before a
new deletion transaction starts. IndexedDB transactions can close while unrelated asynchronous work
is pending; see idb’s transaction lifetime documentation.
Awaiting tx.done confirms the local write committed. Deletion requires a 200/201 receipt matching
the key, owner, size, and digest, rather than merely a successful HTTP response.
import { openDB } from 'idb'
const maxSize = 10 * 1024 * 1024
const lifetime = 24 * 60 * 60 * 1000
const lockName = 'durable-file-uploads'
async function digest(file) {
const bytes = await crypto.subtle.digest('SHA-256', await file.arrayBuffer())
return Array.from(new Uint8Array(bytes), byte => byte.toString(16).padStart(2, '0')).join('')
}
async function database() {
return openDB('durable-file-uploads', 1, {
upgrade(db) {
db.createObjectStore('uploads', { keyPath: 'id' })
},
})
}
export async function enqueue(file, owner) {
if (!owner || !(file instanceof File) || file.size === 0 || file.size > maxSize) {
throw new Error('Choose a file between 1 byte and 10 MiB while signed in.')
}
const sha256 = await digest(file)
const db = await database()
try {
const tx = db.transaction('uploads', 'readwrite')
const id = crypto.randomUUID()
// Observe request and transaction failures together, including quota-induced aborts.
await Promise.all([
(async () => {
// Counting and inserting in one transaction also bounds concurrent tabs.
if ((await tx.store.count()) >= 10) {
throw new Error('The queue is full. Retry or discard queued files first.')
}
await tx.store.add({ id, owner, file, name: file.name, sha256, created: Date.now() })
})(),
tx.done,
])
return id
} finally {
db.close()
}
}
export async function drain(signal = new AbortController().signal) {
return navigator.locks.request(lockName, { signal }, async () => {
const db = await database()
try {
// The count is bounded at ten. No readwrite transaction spans a fetch.
const entries = await db.getAll('uploads')
if (entries.length === 0) return { sent: 0, remaining: 0 }
let sent = 0
const sessionResponse = await fetch('/api/upload-session', {
credentials: 'same-origin',
cache: 'no-store',
redirect: 'error',
signal: AbortSignal.any([signal, AbortSignal.timeout(30_000)]),
})
if (!sessionResponse.ok) throw new Error('Sign in again before retrying.')
const session = await sessionResponse.json()
if (
typeof session.owner !== 'string' ||
!session.owner ||
typeof session.csrf !== 'string' ||
!session.csrf
) {
throw new Error('Invalid upload session.')
}
for (const entry of entries) {
signal.throwIfAborted()
if (entry.owner !== session.owner) continue
if (Date.now() - entry.created >= lifetime) {
throw new Error('A queued file expired. Discard it before retrying.')
}
const response = await fetch('/api/queued-uploads', {
method: 'POST',
credentials: 'same-origin',
redirect: 'error',
body: entry.file,
headers: {
'Content-Type': 'application/octet-stream',
'X-Upload-Created': String(entry.created),
'X-CSRF-TOKEN': session.csrf,
'X-Upload-Owner': entry.owner,
'Idempotency-Key': entry.id,
},
signal: AbortSignal.any([signal, AbortSignal.timeout(30_000)]),
})
if (!response.ok || ![200, 201].includes(response.status)) {
throw new Error(`Upload not confirmed (HTTP ${response.status}).`)
}
const receipt = await response.json()
if (receipt.id !== entry.id || receipt.owner !== entry.owner ||
receipt.size !== entry.file.size || receipt.sha256 !== entry.sha256) {
throw new Error('Invalid upload receipt.')
}
const tx = db.transaction('uploads', 'readwrite')
await Promise.all([tx.store.delete(entry.id), tx.done])
sent++
}
return { sent, remaining: entries.length - sent }
} finally {
db.close()
}
})
}
export async function discard(owner) {
// Serialize with uploads so a deletion cannot race a foreground or background replay.
await navigator.locks.request(lockName, async () => {
const db = await database()
try {
const tx = db.transaction('uploads', 'readwrite')
await Promise.all([
(async () => {
for (const entry of await tx.store.getAll()) {
if (entry.owner === owner) await tx.store.delete(entry.id)
}
})(),
tx.done,
])
} finally {
db.close()
}
})
}
Save upload-sw.ts. The worker caches only the static page and its bundle, so reopening works
offline after the first successful online visit. It never caches login, sessions, receipts, or file
responses. A failed sync rejects waitUntil and leaves unacknowledged entries available for retry.
import { drain } from './queue.ts'
const cacheName = 'upload-shell-v1'
const shell = ['/', '/upload-page.js']
self.addEventListener('install', event => {
event.waitUntil((async () => {
const cache = await caches.open(cacheName)
await cache.addAll(shell)
await self.skipWaiting()
})())
})
self.addEventListener('activate', event => event.waitUntil(self.clients.claim()))
self.addEventListener('fetch', event => {
const url = new URL(event.request.url)
if (event.request.method === 'GET' && url.origin === self.location.origin && shell.includes(url.pathname)) {
event.respondWith(fetch(event.request).catch(async () => {
const response = await caches.match(url.pathname)
return response ?? Response.error()
}))
}
})
self.addEventListener('sync', event => {
if (event.tag === 'file-upload-sync') event.waitUntil(drain())
})
Save public/index.html:
<!doctype html>
<html lang="en">
<head>
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta charset="utf-8">
<meta name="color-scheme" content="light dark">
<title>Offline file queue</title>
</head>
<body>
<h1>Offline file queue</h1>
<p><a href="/login">Sign in or switch account</a> · <a href="/received">Received uploads</a></p>
<p id="account"></p>
<label>File <input id="file" type="file"></label>
<button id="queue" type="button">Queue file</button>
<button id="retry" type="button">Retry queued files</button>
<button id="stop" type="button">Stop foreground retry</button>
<button id="discard" type="button">Discard my queued files</button>
<p id="status" role="status"></p>
<script type="module" src="/upload-page.js"></script>
</body>
</html>
Save upload-page.ts. The controls disable during a foreground operation, preventing repeated
clicks from enqueueing the same selection twice. Saving locally succeeds even when Background Sync
is unavailable. After an offline reopen, the last observed account ID labels the queue; the server
still checks the current cookie session before accepting any bytes.
import { discard, drain, enqueue } from './queue.ts'
const input = document.getElementById('file')
const status = document.getElementById('status')
const account = document.getElementById('account')
const controls = ['queue', 'retry', 'discard'].map(id => document.getElementById(id))
let owner = localStorage.getItem('upload-owner')
let busy = false
let controller = null
let registration = null
function available() {
return owner && 'indexedDB' in globalThis && navigator.locks
}
function render() {
account.textContent = owner ? `Local queue owner: ${owner}` : 'Sign in online before queuing files.'
input.disabled = busy || !available()
for (const button of controls) button.disabled = busy || !available()
}
async function schedule() {
if (!registration?.sync) return false
await registration.sync.register('file-upload-sync')
return true
}
async function action(operation) {
if (busy) return
busy = true
render()
try { await operation() }
catch { status.textContent = 'Retry stopped or failed. Some files may already be received; others remain queued.' }
finally { busy = false; controller = null; render() }
}
document.getElementById('queue').onclick = () => action(async () => {
const file = input.files?.[0]
if (!file) return
try {
await enqueue(file, owner)
input.value = ''
status.textContent = 'Saved locally. Use Retry queued files to upload now.'
} catch {
status.textContent = 'Could not save. Check the 10 MiB file limit, ten-file queue limit, and available storage.'
return
}
try {
if (await schedule()) status.textContent = 'Saved locally. Background retry requested.'
} catch { status.textContent = 'Saved locally. Background retry unavailable; use Retry queued files.' }
})
document.getElementById('retry').onclick = () => action(async () => {
controller = new AbortController()
const result = await drain(controller.signal)
status.textContent = `Delivered ${result.sent} file(s). ${result.remaining} remain queued.`
})
document.getElementById('stop').onclick = () => controller?.abort()
document.getElementById('discard').onclick = () => action(async () => {
await discard(owner)
status.textContent = 'Local queue discarded. Already received uploads remain on the server.'
})
render()
try {
const response = await fetch('/api/upload-session', { cache: 'no-store', redirect: 'error' })
if (response.ok) {
const session = await response.json()
if (!['alice', 'bob'].includes(session.owner)) throw new Error('Invalid session')
owner = session.owner
localStorage.setItem('upload-owner', owner)
} else if (response.status === 401) {
owner = null
localStorage.removeItem('upload-owner')
}
} catch { /* Offline: retain only the previously observed account ID, never credentials. */ }
render()
if (!available()) status.textContent = 'Durable uploads require sign-in, IndexedDB, and Web Locks.'
try {
if ('serviceWorker' in navigator) {
registration = await navigator.serviceWorker.register('/upload-sw.js')
await navigator.serviceWorker.ready
status.textContent = 'Ready for offline use. Background Sync is optional.'
}
} catch { status.textContent = 'Offline page setup failed. Keep this page open to queue and retry.' }
From the original parent directory, build both bundles and start the receiver. The && chain stops
before startup if either build fails, even when older bundles remain. Ctrl+C stops the receiver and
returns you to that same shell. If port 8090 is occupied, choose another port and keep using that
origin; browser storage is scoped to the origin, including its port.
(
cd background-upload-demo &&
corepack yarn esbuild upload-page.ts --bundle --format=esm --target=es2022 --tsconfig-raw='{}' --outfile=public/upload-page.js &&
corepack yarn esbuild upload-sw.ts --bundle --format=iife --target=es2022 --tsconfig-raw='{}' --outfile=public/upload-sw.js &&
node server.mts --port=8090
)
The terminal prints a JSON object with ready and temporary passwords. Open
http://127.0.0.1:8090/login, choose alice, enter its printed password, and press
Sign in. Wait for
Ready for offline use. Background Sync is optional. before disconnecting.
Passwords change on restart; existing unexpired sessions and received uploads survive in
background-upload-demo/uploads.sqlite.
- Take the browser offline or stop the receiver, choose a nonempty file up to 10 MiB, and press Queue file. A message beginning with Saved locally. confirms the IndexedDB commit, not delivery.
- Reload or close and reopen the page at the same origin while offline. The cached page loads; you do not need to select the file again.
- Reconnect. Supporting browsers may deliver the queue through Background Sync. To request a foreground attempt, press Retry queued files.
- Open Received uploads. Each stored file has one key, a byte
count, a SHA-256 digest, and a download link. Download
upload.binand compare its bytes with your original. A retry of the same queue entry must leave one stored row and the same receipt.
Disconnecting only a page’s requests in DevTools can leave its worker online. For a manual offline check, stop the receiver after preparing the page, then queue and reload. Restart it on the same port before retrying. Browser scheduling is best effort, so a background attempt that does not run is a reason to use the button, not proof that the queue was lost.
Recover without claiming an unconfirmed upload succeeded
A 401 means you need to sign in again. When another account is active, retries skip the original owner’s entries. Sign back in as that owner to deliver them, or use Discard my queued files to remove their local copies. Discarding never deletes already received server files. Do not retain sensitive files on a shared device without the user’s consent.
An expired file, HTTP failure, wrong receipt, or interrupted request stays queued. Some earlier entries may already have arrived. Check the received list before discarding; retrying an unchanged entry is safe within its retry window. The server’s 100 MiB quota per account includes all retained files, so a full server needs deliberate data management, not repeated clicking. A failed local write must show a save failure, never a delivery message.
Stop foreground retry aborts only the current page’s attempt. It cannot undo a committed upload or cancel a separately scheduled background attempt. The Web Lock also serializes local discard with delivery, so one cannot delete an entry midway through the other.
For deployment, replace demonstration passwords and sessions with your existing login system, keep receipts or durable expiration records when deleting objects, and enforce your media policy before publication. This loopback example accepts arbitrary nonempty bytes as private downloads; it does not scan files, rate-limit login, replicate storage, or promise recovery after browser eviction. For larger files, retain the same ownership and confirmation checks around a resumable protocol.