Draft image alt text with AWS Rekognition and Node.js

Kevin van Zonneveld

Co-founder · Amsterdam, The Netherlands ·

Providing descriptive alt text for images is crucial for accessibility and SEO. However, manually writing alt text for numerous images can be time-consuming. AWS Rekognition and Node.js can suggest image labels to help an editor write a description. Rekognition detects objects and scenes; it does not generate a contextual caption or determine the image’s purpose on a page.

Why automate alt text?

Alt text helps visually impaired users understand image content through screen readers and provides context to search engines. Label suggestions can speed up review, but must not be inserted directly into an image’s alt attribute. Follow the W3C alt decision tree to decide whether an image needs a description, its link or button purpose described, or an empty alternative for a decorative image. A list of detected objects cannot make those decisions.

AWS Rekognition and Node.js integration

AWS Rekognition is a powerful image analysis service capable of detecting objects, scenes, and activities within images. Integrating it with Node.js allows developers to quickly build scalable applications that provide suggestions for a human-reviewed alt text workflow.

Setting up your environment

First, ensure you have:

An AWS account and an IAM identity allowed to call rekognition:DetectLabels
Node.js 22.12 or later installed
AWS SDK v3 for JavaScript

Install the AWS SDK:

npm install @aws-sdk/client-rekognition@3 dotenv@17

Configure a local AWS profile or an IAM role through the SDK’s default credential provider chain. Set the region in a local .env file, and exclude that file from version control:

AWS_REGION=your-region

If you use credential environment variables, temporary credentials also need AWS_SESSION_TOKEN alongside AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. Do not embed credentials in source code.

Image requirements

Before implementing the solution, be aware of AWS Rekognition's image requirements:

Supported formats: JPEG and PNG
Maximum image size: 5MB for direct bytes upload
Maximum image size: 15MB when referenced as an S3 object
For DetectLabels, each dimension must be between 80 and 10,000 pixels

See the Rekognition quotas for operation-specific requirements.

Implementing label suggestions

Here’s a Node.js script that returns label suggestions for editorial review:

import { RekognitionClient, DetectLabelsCommand } from '@aws-sdk/client-rekognition'
import fs from 'fs'
import dotenv from 'dotenv'

dotenv.config({ quiet: true })

const client = new RekognitionClient({ region: process.env.AWS_REGION })

async function detectImageLabels(image, minConfidence = 75) {
  if (!Number.isFinite(minConfidence) || minConfidence < 0 || minConfidence > 100) {
    throw new Error('Confidence must be a number between 0 and 100.')
  }
  const command = new DetectLabelsCommand({
    Image: image,
    MaxLabels: 5,
    MinConfidence: minConfidence,
  })
  const response = await client.send(command)
  return (response.Labels ?? [])
    .map((label) => label.Name)
    .filter((name) => typeof name === 'string' && name.trim().length > 0)
}

async function suggestImageLabels(imagePath, minConfidence = 75) {
  const imageBytes = fs.readFileSync(imagePath)
  return detectImageLabels({ Bytes: imageBytes }, minConfidence)
}

// Example usage
suggestImageLabels('./example.jpg').then(console.log).catch(() => {
  console.error('Unable to retrieve image label suggestions.')
  process.exitCode = 1
})

Save this code with a .mjs extension to use ES modules, or set "type": "module" in package.json. An empty array means no usable labels were returned. Request and file-read failures reject the promise so the application can distinguish them from a successful analysis.

Processing images from S3

In production environments, you'll often need to process images stored in Amazon S3. Here's how to extend the script above to work with S3. The bucket must be in the client’s region, and the calling identity needs s3:GetObject permission for the image:

async function suggestImageLabelsFromS3(bucket, key, minConfidence = 75) {
  return detectImageLabels({ S3Object: { Bucket: bucket, Name: key } }, minConfidence)
}

// Example usage
suggestImageLabelsFromS3('my-bucket', 'images/example.jpg').then(console.log).catch(() => {
  console.error('Unable to retrieve image label suggestions from S3.')
  process.exitCode = 1
})

Handling edge cases and errors

Handle failures where your application calls the helper. Keep errors separate from label data; an error message must never become alt text or a cached successful response:

try {
  const labels = await suggestImageLabels('./example.jpg')
  console.log('Labels for review:', labels)
} catch (error) {
  if (error.name === 'InvalidImageFormatException') {
    console.error('Invalid image format. Only JPEG and PNG are supported.')
  } else if (error.name === 'InvalidParameterException') {
    console.error('Invalid parameters provided to AWS Rekognition')
  } else if (error.name === 'ImageTooLargeException') {
    console.error('Image exceeds the input limit: 5MB for bytes, 15MB for S3 objects.')
  } else if (error.name === 'AccessDeniedException') {
    console.error('AWS credentials lack permission to use Rekognition')
  } else if (error.name === 'ThrottlingException') {
    console.error('AWS request rate exceeded limits')
  } else {
    console.error('Unable to retrieve image label suggestions.')
  }
  process.exitCode = 1
}

Enhancing functionality

To further enhance your label suggestion workflow:

Implement caching

Store successful label suggestions to avoid redundant API calls. This optional example needs a running Redis server and npm install redis@5. Append it to the script above. Hash the actual image bytes and analyze that same buffer so changing a file at the same path invalidates its cache:

import { createHash } from 'node:crypto'
import { createClient } from 'redis'

const redisClient = createClient()
redisClient.on('error', () => console.error('Redis connection error.'))
await redisClient.connect()

async function getCachedOrSuggestImageLabels(imagePath) {
  const imageBytes = fs.readFileSync(imagePath)
  const imageHash = createHash('sha256').update(imageBytes).digest('hex')
  // Change the version if label settings or the interpretation of the result changes.
  const cacheKey = `image_labels:v1:${process.env.AWS_REGION}:${imageHash}`

  const cachedLabels = await redisClient.get(cacheKey)
  if (cachedLabels !== null) return JSON.parse(cachedLabels)

  const labels = await detectImageLabels({ Bytes: imageBytes })

  await redisClient.set(cacheKey, JSON.stringify(labels), { EX: 60 * 60 * 24 * 30 })

  return labels
}

try {
  console.log(await getCachedOrSuggestImageLabels('./example.jpg'))
} catch {
  console.error('Unable to retrieve cached image label suggestions.')
  process.exitCode = 1
} finally {
  await redisClient.close()
}

Batch processing

For a small batch, process multiple images simultaneously. Use a bounded queue for larger lists and keep concurrency within your Rekognition quota:

async function batchProcessImages(imagePaths) {
  return Promise.all(imagePaths.map((path) => suggestImageLabels(path)))
}

// Example usage
try {
  const results = await batchProcessImages(['image1.jpg', 'image2.jpg', 'image3.jpg'])
  console.log(results)
} catch {
  console.error('Unable to retrieve suggestions for the complete batch.')
  process.exitCode = 1
}

Confidence threshold tuning

Adjust the confidence threshold based on your needs:

suggestImageLabels('./example.jpg', 90)
  .then(console.log)
  .catch(() => {
    console.error('Unable to retrieve image label suggestions.')
    process.exitCode = 1
  })

Closing thoughts

AWS Rekognition and Node.js can support an editorial workflow by suggesting labels and reusing successful analyses. Review those suggestions against the actual image and its purpose on the page before writing alt text. Confidence scores describe model predictions, not accessibility quality.

If you're looking for a managed solution, consider exploring Transloadit's 🤖 Artificial Intelligence Service, which simplifies image analysis for this workflow.