Export files to Amazon S3 in Node.js: a complete guide
Use PutObjectCommand for a single upload and Upload from @aws-sdk/lib-storage for multipart
uploads. The CLI below exports a local file to Amazon S3, downloads the resulting object, and checks
its SHA-256, size, and metadata before reporting success. Both upload paths return a failing exit
status when an upload or verification fails.
Prerequisites
You need Bash on Linux, Node.js, separately installed Corepack, and an existing private, general-purpose S3 bucket. This example was tested with Node.js 24.21.0 and 26.8.1, Corepack 0.36.0, Yarn 4.12.0, and AWS SDK packages 3.1149.0. Use a maintained Node.js 24 LTS or 26 release; these are tested versions, rather than an intrinsic minimum for the SDK. Node.js 26 does not bundle Corepack.
Use a regular local file and keep its contents unchanged until verification finishes. Quote its
path when invoking the CLI. Spaces, literal percent signs, and supplementary Unicode characters are
supported. Paths containing the Unicode replacement character U+FFFD are refused, including a
legitimate filename containing that character. Node.js can replace invalid native argument bytes
with it, making two different filenames indistinguishable. Rename such files before using this CLI.
Choose an unused object key under a prefix you own. This example refuses to replace an existing
object. The credential needs s3:PutObject for uploads, s3:GetObject for verification, and
s3:AbortMultipartUpload for cleanup after multipart failures.
Setting up AWS credentials
The SDK uses its default credential provider chain.
Use an existing authenticated AWS profile locally, or an IAM role on AWS. Temporary environment
credentials also work; they require AWS_SESSION_TOKEN as well as the access key and secret key.
Do not put credentials in the program or commit them with the project.
The commands below read the bucket’s region from AWS_REGION. For a named shared profile, set
AWS_PROFILE in the shell that runs the commands. If credentials expire, renew them through your
usual authentication process before retrying. The example does not create a bucket or configure IAM.
Installing and using the AWS SDK
Paste this block in the directory where you want a new s3-export project. It checks each executable
before creating files and runs in a subshell, so it leaves your caller’s directory and options
unchanged. An existing s3-export directory stops setup before any files are overwritten.
(
set -eu
command -v node >/dev/null || { printf 'Install Node.js first.\n' >&2; exit 1; }
command -v corepack >/dev/null || { printf 'Install Corepack first.\n' >&2; exit 1; }
mkdir s3-export
cd s3-export
cat > package.json <<'JSON'
{
"name": "s3-export",
"private": true,
"type": "module",
"packageManager": "yarn@4.12.0",
"dependencies": {
"@aws-sdk/client-s3": "3.1149.0",
"@aws-sdk/lib-storage": "3.1149.0"
}
}
JSON
touch yarn.lock
corepack yarn install
)
The local manifest and lockfile give this child project its own package and module boundary, even
inside an npm or Yarn project. Yarn still reads applicable home and ancestor .yarnrc.yml settings.
Check the effective version with corepack yarn --version from s3-export; it should be 4.12.0.
If an inherited yarnPath, registry, or network policy prevents installation, resolve that policy
with the project owner or use a directory outside that configuration. Do not silently bypass it.
If installation fails after project creation, keep the files and retry installation from the caller:
(cd s3-export && corepack yarn install)
Save this complete program as s3-export/export.ts:
import { createHash } from 'node:crypto'
import { createReadStream } from 'node:fs'
import { stat } from 'node:fs/promises'
import { Readable } from 'node:stream'
import {
GetObjectCommand,
PutObjectCommand,
S3Client,
S3ServiceException,
type PutObjectCommandInput,
} from '@aws-sdk/client-s3'
import { Upload } from '@aws-sdk/lib-storage'
class ReaderError extends Error {}
async function digest(source: AsyncIterable<Uint8Array>): Promise<{ bytes: number; sha256: string }> {
const hash = createHash('sha256')
let bytes = 0
for await (const chunk of source) {
bytes += chunk.byteLength
hash.update(chunk)
}
return { bytes, sha256: hash.digest('hex') }
}
async function main(): Promise<{ bucket: string; key: string; bytes: number; sha256: string }> {
const [mode, file, bucket, key, ...extra] = process.argv.slice(2)
if (!['put', 'multipart', 'verify'].includes(mode ?? '') || !file || !bucket || !key || extra.length) {
throw new ReaderError('Usage: export.ts put|multipart|verify FILE BUCKET KEY')
}
if (file.includes('\uFFFD')) {
throw new ReaderError('FILE must not contain U+FFFD; rename the file before exporting it.')
}
const region = process.env.AWS_REGION
if (!region) throw new ReaderError('Set AWS_REGION to the bucket’s region.')
const info = await stat(file)
if (!info.isFile()) throw new ReaderError('FILE must be a regular file.')
const expected = await digest(createReadStream(file))
if (expected.bytes !== info.size) throw new ReaderError('The input changed while being read.')
const client = new S3Client({ region, maxAttempts: 1 })
try {
if (mode !== 'verify') {
const body = createReadStream(file)
try {
const params: PutObjectCommandInput = {
Bucket: bucket,
Key: key,
Body: body,
ContentLength: expected.bytes,
ContentType: 'application/octet-stream',
Metadata: { sha256: expected.sha256 },
ServerSideEncryption: 'AES256',
IfNoneMatch: '*',
}
if (mode === 'put') {
await client.send(new PutObjectCommand(params))
} else {
const upload = new Upload({
client,
params,
queueSize: 2,
leavePartsOnError: false,
})
upload.on('httpUploadProgress', ({ loaded, total }) => {
console.error(`Transferred ${loaded ?? 0}/${total ?? expected.bytes} bytes`)
})
await upload.done()
}
} finally {
body.destroy()
}
}
const stored = await client.send(new GetObjectCommand({ Bucket: bucket, Key: key }))
if (!(stored.Body instanceof Readable)) throw new ReaderError('No readable S3 response body.')
const actual = await digest(stored.Body)
if (
actual.bytes !== expected.bytes ||
actual.sha256 !== expected.sha256 ||
stored.ContentLength !== expected.bytes ||
stored.Metadata?.sha256 !== expected.sha256 ||
stored.ContentType !== 'application/octet-stream' ||
stored.ServerSideEncryption !== 'AES256'
) {
throw new ReaderError('Stored bytes or metadata do not match the input.')
}
return { bucket, key, ...actual }
} finally {
client.destroy()
}
}
main()
.then((outcome) => console.log(JSON.stringify(outcome)))
.catch((error: unknown) => {
if (error instanceof ReaderError) {
console.error(error.message)
} else {
const reason = error instanceof S3ServiceException ? error.name : 'LocalError'
console.error(`Export failed (${reason}). Check the file, region, credentials, and permissions.`)
}
process.exitCode = 1
})
Run the following from the directory containing s3-export. Replace the region, bucket, and key
with your own destination. The sample file is created only if absent, so a rerun preserves its contents:
(
cd s3-export &&
(set -C; printf 'S3 export example\n' > sample.txt) &&
AWS_REGION=us-east-1 corepack yarn node export.ts put sample.txt YOUR_BUCKET exports/sample.txt
)
A successful run prints JSON containing the exact bucket, key, byte count, and SHA-256. Verification reads the object back from that key; it does not treat an ETag or the upload response as a local-file checksum. The download costs an additional request and reads all the bytes again.
Handling large files with multipart upload
The multipart helper
splits a stream into buffered parts. For known-length input it chooses a part size
of at least 5 MiB, adjusted for the part-count limit. queueSize: 2 permits two part uploads at a time; it does
not load the whole file into memory. Small inputs, including empty files, use a single PUT instead.
Use the same program and substitute multipart for put:
(
cd s3-export &&
AWS_REGION=us-east-1 corepack yarn node export.ts multipart /path/to/large-file.zip YOUR_BUCKET exports/large-file.zip
)
Progress is printed to stderr. Even a progress event showing all bytes transferred is not completion:
the program awaits upload.done() and then verifies the stored object. leavePartsOnError: false
asks the helper to abort after a part-upload failure. In this pinned version, an error from
CompleteMultipartUpload,
including a key collision, leaves the unfinished upload behind.
Have the bucket owner inspect and abort that upload; terminating the process can also leave parts.
S3 allows up to 5 GB in one PUT and up to 50 TB through multipart upload, but the JavaScript helper and its maximum part count do not establish support for every service-size maximum. For objects above 5 TB, AWS’s upload guide directs you to the S3 Transfer Manager in Java, Python, or the AWS CLI. This walkthrough verifies single and multipart behavior with small fixtures; it is not a 50 TB transfer recipe.
Security best practices
Keep the bucket private and scope access to the intended prefix. For this program’s upload, verification, and the helper’s multipart-abort operations, an administrator can grant this object-level policy, replacing the bucket and prefix:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:PutObject", "s3:GetObject", "s3:AbortMultipartUpload"],
"Resource": "arn:aws:s3:::YOUR_BUCKET/exports/*"
}
]
}
These permissions follow the multipart operation requirements
and GetObject requirements.
Bucket policies can still deny a request. The program uses HTTPS and explicitly requests S3-managed
encryption with AES256; it does not request a public ACL. A bucket requiring SSE-KMS needs a
different encryption configuration and the applicable KMS permissions, so use that bucket’s policy
rather than adding broad permissions to this example.
Error handling and retries
The IfNoneMatch: '*' condition
refuses an existing key at the upload’s final write. A collision returns failure and preserves the
existing object; use a new key instead of removing someone else’s data. Both upload modes also
fail if credentials are rejected or the requested file is missing.
maxAttempts: 1 sets the SDK to one attempt per request. The setting counts the initial request,
so a higher value allows retries. A consumed single-PUT stream cannot simply be replayed; the
multipart helper buffers parts, which lets the SDK retry those part requests. If you retry the whole
CLI, it opens a fresh stream. An upload failure can have an uncertain stored result, and a verification
failure can occur after the upload succeeded. Inspect that exact key before retrying. To verify
without uploading again, run:
(
cd s3-export &&
AWS_REGION=us-east-1 corepack yarn node export.ts verify sample.txt YOUR_BUCKET exports/sample.txt
)
This checks the bytes and metadata against the current local file and exits nonzero on a mismatch. It does not delete a mismatching object or roll back a completed upload.
TypeScript support
Node.js runs this file using native type stripping.
The setup declares "type": "module", and corepack yarn node supplies Yarn’s dependency loader,
including in Plug’n’Play projects. No tsx, ts-node, or compile step is needed. Native execution
does not typecheck: PutObjectCommandInput helps your editor check request fields, while the runtime
checks still validate arguments and stored results.
Conclusion
Use the printed JSON as the result of a completed upload and verification, and keep stderr available when diagnosing a failure. If exporting is one step in a broader file-processing workflow, check out our services.