Real-time file processing with Deno and WebSockets
With the rise of real-time applications, developers are constantly seeking efficient ways to handle live data streams and process files on the fly. Deno, a modern runtime for JavaScript and TypeScript, offers robust WebSocket support and enhanced security features that make it an excellent choice for building secure, real-time file processing systems.
In this DevTip, we'll explore how to leverage Deno's WebSocket capabilities and its secure permissions model to implement a real-time WebSocket server. This pattern can serve as a foundation for real-time file processing tasks—such as monitoring and processing file data—as part of a broader application ecosystem.
Why Deno?
Deno provides a secure runtime that includes TypeScript support, an integrated permissions system, and a sandboxed environment. These features help ensure that operations, including file processing tasks, are executed safely without exposing unnecessary system resources. Unlike Node.js, Deno focuses on security and simplicity, giving you more control over network and file system access.
Setting up a WebSocket server
Create a new file called server.ts and add the following code:
Deno.serve((req) => {
if (req.headers.get('upgrade') !== 'websocket') {
return new Response(null, { status: 501 })
}
const { socket, response } = Deno.upgradeWebSocket(req)
socket.addEventListener('open', () => {
console.log('A client connected!')
})
socket.addEventListener('message', (event) => {
if (event.data === 'ping') {
// In a real application, you might trigger file processing here
socket.send('pong')
}
})
socket.addEventListener('error', (event) => {
console.error('WebSocket error:', event)
})
socket.addEventListener('close', () => {
console.log('A client disconnected')
})
return response
})
Running the server
To run the server, execute:
deno run --allow-net server.ts
Understanding the WebSocket lifecycle
WebSocket connections transition through several states:
- CONNECTING (0): The initial state while establishing the connection.
- OPEN (1): The connection is active and ready for data transfer.
- CLOSING (2): The connection is in the process of closing.
- CLOSED (3): The connection has been terminated.
For example, you can check if a socket is ready before sending a message:
function isSocketReady(socket: WebSocket): boolean {
return socket.readyState === WebSocket.OPEN
}
socket.onmessage = (event) => {
if (!isSocketReady(socket)) {
return
}
// Handle message
}
Client implementation
Below is a streamlined client example to test the server:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<title>WebSocket Test</title>
</head>
<body>
<script>
const ws = new WebSocket('ws://localhost:8000')
ws.onopen = () => {
console.log('Connected to the server')
ws.send('ping')
}
ws.onmessage = (event) => {
console.log('Received:', event.data)
}
ws.onerror = (error) => {
console.error('WebSocket error:', error)
}
ws.onclose = () => {
console.log('Disconnected from the server')
}
</script>
</body>
</html>
Security best practices
When building real-time applications, especially those involving file processing, consider these security measures:
1. Input validation
Ensure incoming data is correctly formatted to avoid processing unexpected inputs. These security
snippets illustrate checks to integrate into your server’s message handler; the ping/pong server
above does not apply them automatically. If a message refers to a file, accept an authorized file
ID mapped to a server-controlled path. Never pass an untrusted filename or path directly to Deno’s
file APIs, and scope any --allow-read or --allow-write permission to the required directory.
function validateMessage(data: unknown): boolean {
if (typeof data !== 'string') {
return false
}
try {
const message = JSON.parse(data)
return typeof message === 'object' && message !== null && !Array.isArray(message)
&& typeof message.type === 'string'
} catch {
return false
}
}
2. Rate limiting
Integrate a rate limiter to restrict accepted application messages. The illustrative in-memory
limiter below is per process; expire inactive client entries in a long-running service, limit total
connections, and use an authenticated identity or server-assigned connection ID for clientId.
class RateLimiter {
private requests = new Map<string, number[]>()
private readonly limit: number
private readonly window: number
constructor(limit = 100, window = 60000) {
this.limit = limit
this.window = window
}
isAllowed(clientId: string): boolean {
const now = Date.now()
const timestamps = this.requests.get(clientId) || []
const recent = timestamps.filter((time) => now - time < this.window)
if (recent.length >= this.limit) {
return false
}
recent.push(now)
this.requests.set(clientId, recent)
return true
}
}
3. Message size limits
Check the UTF-8 byte length of accepted text messages. JavaScript string length counts UTF-16 code units, so it is not a byte count. This check runs after WebSocket has buffered and delivered the message, and encoding the string allocates another buffer. It limits subsequent application work, not network buffering or peak memory. The WebSocket API lacks backpressure; use a server or gateway with explicit WebSocket message limits, connection limits, and rate limits to bound untrusted traffic before application processing.
Place this handler inside Deno.serve, where the upgraded socket is in scope:
const MAX_MESSAGE_SIZE = 1024 * 1024 // 1 MiB of UTF-8 text
const encoder = new TextEncoder()
socket.addEventListener('message', (event) => {
if (typeof event.data !== 'string') {
socket.close(1003, 'Only text messages are accepted')
return
}
if (encoder.encode(event.data).byteLength > MAX_MESSAGE_SIZE) {
socket.close(1009, 'Message size exceeds limit')
return
}
// Process message
})
Conclusion
Deno's built-in WebSocket support and strong security model make it a powerful tool for developing real-time applications, including those that process files dynamically. By applying the best practices outlined in this guide, you can build secure and efficient systems capable of handling real-time data and file operations.
At Transloadit, we recognize the value of secure, real-time communication. Explore our File Filtering service to see how advanced file handling can complement your real-time applications.