Upload a file to Amazon S3 with Boto3

Kevin van Zonneveld

Co-founder · Amsterdam, The Netherlands ·

Use Boto3’s upload_file() to send a local file to an existing private S3 bucket. This walkthrough gives you a command that takes an explicit destination, waits for the upload to finish, and exits with a failure status when the file or transfer fails. Credentials and bucket permissions stay in your AWS configuration.

Boto3’s managed upload handles multipart transfers for large files. You supply a filename, bucket name, and object key; you do not need to split the file yourself.

Prepare your environment

The commands below use Bash and Python 3.12 with venv and pip on Linux. The example uses Boto3 1.43.100. Before running it, you need:

An existing private, general purpose S3 bucket, its AWS Region, and a prefix you may write to, such as incoming/. Keep S3 Block Public Access enabled.
An authenticated AWS profile. For a workstation, use your organization’s temporary credentials, such as an IAM Identity Center profile. Complete that setup first; for an existing profile named uploads, renew its session with aws sso login --profile uploads using AWS CLI v2. Do not put access keys in the script.
Permission to perform s3:PutObject on the destination objects, for example arn:aws:s3:::your-bucket-name/incoming/*. Allow s3:AbortMultipartUpload on that scope for failed multipart cleanup. See AWS’s multipart permissions. The script does not list or download objects, so it does not need s3:ListBucket or s3:GetObject.

Create a fresh directory and an isolated Python environment. The && chain stops if a step fails; if boto3-upload already exists, choose another directory name before continuing.

mkdir boto3-upload &&
cd boto3-upload &&
python3 -m venv .venv &&
.venv/bin/python -m pip install 'boto3==1.43.100'

Continue from inside boto3-upload after installation succeeds. The command below assumes the uploads profile and us-east-1; replace them with your profile and the bucket’s Region. Boto3’s credential chain can also use an attached IAM role on AWS. In that environment, omit AWS_PROFILE instead of copying workstation credentials to the server. Environment credentials can take precedence over a profile, so clear stale credential overrides from your shell if Boto3 selects the wrong identity.

Save the upload command

Save this complete program as upload.py in the new directory. It accepts one regular file, including an empty file, and rejects a missing path or a directory before creating an S3 client.

import argparse
import sys
from pathlib import Path

import boto3
from boto3.exceptions import S3UploadFailedError
from botocore.exceptions import BotoCoreError, ClientError


def main():
    parser = argparse.ArgumentParser(description="Upload one file to S3.")
    parser.add_argument("file", type=Path, help="Local file to upload")
    parser.add_argument("bucket", help="Existing S3 bucket name")
    parser.add_argument("key", help="Full destination object key")
    args = parser.parse_args()
    if not args.bucket or not args.key:
        parser.error("bucket and key must not be empty")

    try:
        if not args.file.is_file():
            parser.error("file must be an existing regular file")
        s3 = boto3.client("s3")
        s3.upload_file(str(args.file), args.bucket, args.key)
    except (S3UploadFailedError, BotoCoreError, ClientError, OSError) as error:
        print(f"Upload failed ({type(error).__name__}).", file=sys.stderr)
        return 1

    print(f"Uploaded to s3://{args.bucket}/{args.key}")
    return 0


if __name__ == "__main__":
    sys.exit(main())

The upload implementation returns None on success and raises on failure. Do not test its return value for truthiness. The program prints success only after the call completes, and catches both managed transfer failures and lower-level SDK or filesystem errors. It reports the exception type without dumping the full service response.

The client uses HTTPS with certificate verification by default. Keep those defaults for AWS, and remove any custom endpoint overrides left over from local testing. Encryption at rest and access permissions are separate settings, covered below.

Run it with an explicit object key

Choose a local file and its destination before running the command. Here, ./report.pdf is an existing file you place in boto3-upload; you can substitute another path. Replace your-bucket-name with your bucket name, without an s3:// prefix.

AWS_PROFILE=uploads AWS_DEFAULT_REGION=us-east-1 \
  .venv/bin/python upload.py './report.pdf' 'your-bucket-name' 'incoming/report.pdf'

The key is the complete name inside the bucket. S3 does not infer it from the local path or append the filename to incoming/. Quote paths and keys containing spaces; for a local filename that starts with a hyphen, include its ./ prefix.

For the destination shown above, successful output is:

Uploaded to s3://your-bucket-name/incoming/report.pdf

Rerunning this command writes the same key again without a prompt. In an unversioned bucket, that replaces the existing object. With versioning enabled, S3 keeps a new version. Use a different key if you need to retain separate uploads. See AWS’s overwrite and versioning behavior.

For a scheduler or another script, use the process status: 0 means the upload completed, 1 means a caught upload failure, and 2 means invalid arguments or a missing/non-file input. The local file remains in place. A connection failure can leave the remote outcome uncertain if S3 accepted a request before the response was lost; check the destination before retrying when a duplicate version would matter. Keep the source file unchanged while uploading it.

Use the bucket’s encryption settings

Amazon S3 encrypts all new objects at rest. SSE-S3, using AES256, is the initial bucket default and has no additional encryption charge. An administrator can select a different default, such as SSE-KMS. Because this program sends no encryption override, S3 uses the bucket’s configured default.

For SSE-KMS, the uploading identity needs kms:GenerateDataKey on the key; multipart uploads also need kms:Decrypt. The key must be in the bucket’s Region, and its key policy must permit the intended use. AWS documents these KMS permissions and requirements. A bucket policy that requires explicit encryption headers can reject this script even when bucket default encryption is configured. Ask the bucket owner whether default-based uploads are allowed before using this example; do not weaken that policy to make an upload pass.

Keep access control with the bucket owner

Omitting an ACL does not make an arbitrary bucket private. Use the private bucket and scoped identity prepared earlier. New buckets default to the bucket owner enforced setting, which disables ACLs. Adding ACL='private' to such an upload can cause AccessControlListNotSupported; manage access through policies and keep Block Public Access enabled, as described in AWS’s S3 security guidance.

Have the bucket owner maintain bucket-wide rules, including any requirement to deny non-HTTPS requests. The uploader should not install a new bucket policy each time it sends a file.

Diagnose a failed upload

Invalid input, status 2: check the path relative to your current directory. A directory is not an uploadable file; this command does not recurse through it.
NoCredentialsError or an expired session: select the intended profile and renew its login. Confirm that the scheduler or service has its own configured identity if the command works only in your interactive shell.
S3UploadFailedError: check the bucket name, exact key prefix, upload permission, and any bucket-policy denial with the owner. For SSE-KMS, check the key permissions too. This exception can wrap multiple service errors; its type alone does not establish that access was denied.
EndpointConnectionError or another connection error: check the network, Region, proxy, and endpoint configuration. Do not disable certificate verification to bypass a TLS error.
PermissionError or another OSError: check that the process can read the file and that it has not been moved or removed during the upload.

Transfer Acceleration, lifecycle rules, and event notifications are separate bucket-management decisions. In particular, putting an S3 notification configuration replaces the existing configuration; adding a Lambda trigger belongs in a reviewed infrastructure change with the existing destinations preserved. Get the single-file command working with the intended identity and prefix before wiring it into a scheduled job.