Last updated: February 5, 2025
# Export files to Amazon S3 in Node.js: a complete guide
#### Kevin van Zonneveld
Co-founder · Amsterdam, The Netherlands · Show bio
[](https://x.com/kvz)[](https://github.com/kvz)
Exporting files to Amazon S3 is a common requirement for modern web applications. Whether you're building a file storage system, implementing backups, or managing user uploads, understanding how to efficiently export files to Amazon S3 using Node.js is essential.
## Prerequisites
Before we begin, ensure you have:
* **Node.js 20 or later installed**
* **An AWS account with access credentials**
* **Basic understanding of JavaScript and Node.js**
* **A project set up with "type": "module" in your `package.json`**
## Setting up AWS credentials
To interact with Amazon S3 from your Node.js application, you'll need to configure your AWS credentials. There are several ways to do this:
1. **Environment Variables**: Set the following environment variables in your system or `.env` file:
```bash
AWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY=YOUR_SECRET_ACCESS_KEY
AWS_REGION=YOUR_AWS_REGION
```
2. **Shared Credentials File**: Use the AWS credentials file located at `~/.aws/credentials`:
```ini
[default]
aws_access_key_id = YOUR_ACCESS_KEY_ID
aws_secret_access_key = YOUR_SECRET_ACCESS_KEY
```
3. **IAM Roles**: If you're running your application on AWS services like EC2, you can assign an IAM role to your instance and the AWS SDK will automatically use those credentials.
## Installing and using the AWS SDK
We'll start with the official AWS SDK for JavaScript (v3). Install the necessary packages with:
```bash
npm install @aws-sdk/client-s3 @aws-sdk/lib-storage @aws-sdk/node-http-handler
```
Here's a basic example of uploading a file to Amazon S3:
```typescript
import {
S3Client,
PutObjectCommand,
S3ServiceException,
type PutObjectCommandInput,
} from '@aws-sdk/client-s3'
import { NodeHttpHandler } from '@aws-sdk/node-http-handler'
import { createReadStream } from 'fs'
const s3Client = new S3Client({
region: 'us-east-1',
requestHandler: new NodeHttpHandler({
socketTimeout: 3000,
timeoutError: true,
}),
})
async function uploadFile(filePath: string, bucketName: string, key: string) {
try {
const uploadParams: PutObjectCommandInput = {
Bucket: bucketName,
Key: key,
Body: createReadStream(filePath),
}
const command = new PutObjectCommand(uploadParams)
await s3Client.send(command)
console.log('Upload completed successfully')
} catch (err) {
if (err instanceof S3ServiceException) {
console.error(`S3 Error: ${err.name} - ${err.message}`)
} else {
console.error('Upload failed:', err)
}
}
}
// Usage
await uploadFile('path/to/file.txt', 'my-bucket', 'file.txt')
```
This example uses the `PutObjectCommand` to upload a file to S3. The `createReadStream` function streams the file for efficient uploading.
## Handling large files with multipart upload
For large files, it's recommended to use multipart uploads to improve reliability and performance. The AWS SDK provides utilities to simplify this process. Using the `Upload` class from`@aws-sdk/lib-storage` handles multipart uploads automatically:
```typescript
import { S3Client, S3ServiceException } from '@aws-sdk/client-s3'
import { Upload } from '@aws-sdk/lib-storage'
import { createReadStream } from 'fs'
const s3Client = new S3Client({ region: 'us-east-1' })
async function uploadLargeFile(filePath: string, bucketName: string, key: string) {
try {
const upload = new Upload({
client: s3Client,
params: {
Bucket: bucketName,
Key: key,
Body: createReadStream(filePath),
},
})
upload.on('httpUploadProgress', (progress) => {
console.log(`Progress: ${progress.loaded}/${progress.total}`)
})
await upload.done()
console.log('Large file upload completed successfully')
} catch (err) {
if (err instanceof S3ServiceException) {
console.error(`S3 Error: ${err.name} - ${err.message}`)
} else {
console.error('Upload failed:', err)
}
}
}
// Usage
await uploadLargeFile('path/to/large-file.zip', 'my-bucket', 'large-file.zip')
```
## Security best practices
When exporting files to Amazon S3, it's crucial to follow security best practices:
1. **Use IAM Roles**: Instead of hardcoding access keys, use IAM roles when deploying to AWS services like EC2 or ECS. The AWS SDK automatically uses HTTPS and loads credentials from environment variables, shared credentials files, or IAM roles if available.
```typescript
import { S3Client } from '@aws-sdk/client-s3'
import { NodeHttpHandler } from '@aws-sdk/node-http-handler'
const s3Client = new S3Client({
region: 'us-east-1',
maxAttempts: 3,
requestHandler: new NodeHttpHandler({
socketTimeout: 3000,
timeoutError: true,
}),
})
```
2. **Implement Least Privilege Access**: Restrict your IAM user or role permissions to only those necessary for the task.
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:PutObject"],
"Resource": "arn:aws:s3:::your-bucket/*"
}
]
}
```
3. **Enable Server-Side Encryption**: Protect your data at rest by enabling server-side encryption.
```typescript
const uploadParams: PutObjectCommandInput = {
Bucket: bucketName,
Key: key,
Body: createReadStream(filePath),
ServerSideEncryption: 'AES256',
}
```
4. **Use HTTPS**: The AWS SDK v3 uses HTTPS by default for all API calls, ensuring data in transit is encrypted.
## Error handling and retries
Implement robust error handling and retry logic to make your application resilient. The AWS SDK includes built-in retry logic, but you can customize it as needed:
```typescript
import {
S3Client,
PutObjectCommand,
S3ServiceException,
type PutObjectCommandInput,
} from '@aws-sdk/client-s3'
const s3Client = new S3Client({
region: 'us-east-1',
maxAttempts: 3,
})
async function uploadFileWithRetry(filePath: string, bucketName: string, key: string) {
let attempt = 0
while (attempt < 3) {
try {
await uploadFile(filePath, bucketName, key)
return
} catch (err) {
attempt++
if (attempt === 3) {
console.error(`Failed to upload after 3 attempts.`)
throw err
}
console.log(`Retrying upload (${attempt}/3)...`)
}
}
}
```
For additional control, you can implement custom retry strategies tailored to your needs.
## TypeScript support
The AWS SDK for JavaScript v3 offers robust TypeScript support to help catch errors during development and provide improved autocomplete in your IDE. Below is a basic example demonstrating how to define types when uploading a file:
```typescript
import { S3Client, PutObjectCommand, type PutObjectCommandInput } from '@aws-sdk/client-s3'
import { createReadStream } from 'fs'
const s3Client = new S3Client({ region: 'us-east-1' })
const uploadParams: PutObjectCommandInput = {
Bucket: 'my-bucket',
Key: 'my-key',
Body: createReadStream('file.txt'),
}
async function uploadFile() {
try {
const command = new PutObjectCommand(uploadParams)
await s3Client.send(command)
console.log('File uploaded successfully.')
} catch (error) {
console.error('Error uploading file:', error)
}
}
uploadFile()
```
This example demonstrates how to leverage TypeScript to define and enforce the structure of your parameters, enhancing maintainability and catching potential issues during development.
## Conclusion
Efficiently exporting files to Amazon S3 using Node.js involves understanding how to interact with the AWS SDK and following best practices for security and error handling. By leveraging the AWS SDK and implementing the strategies discussed, you can build robust file export functionality in your Node.js applications.
At Transloadit, we focus on making file processing and infrastructure tasks easier. If you're looking for a solution to handle file uploads and processing without the hassle, feel free to[check out our services](/index.md).
\#nodejs#aws#s3#file-exporting-service
### 👩💻 Join 20k+ developers
Sign up for our [monthly newsletter](/newsletters.md) to receive direct links to 3 exclusive tech — and 2 product updates. No less, no more.
Your email:
Get access
## File uploading and encoding. Made simple.
Transloadit streamlines file handling for developers, trusted by brands like Coursera and The New York Times. We’re known for a reliable API, top-notch support, and a strong commitment to open source, with projects like [Uppy](https://uppy.io) and [Tus](https://tus.io) setting standards in file processing.
[Sign up](/c/)[Book a Demo](https://survey.typeform.com/to/kRg47Xi5)
No credit card needed · 5 GB included in the free plan
Cancel anytime