Last updated: May 12, 2025
# Export files to Cloudflare R2 using Scala and the AWS SDK
#### Kevin van Zonneveld
Co-founder · Amsterdam, The Netherlands · Show bio
[](https://x.com/kvz)[](https://github.com/kvz)
Cloudflare R2 combines friendly pricing—no egress fees—with an S3-compatible API, making it easy to integrate with existing tooling. In this DevTip, you’ll learn how to push files to R2 from Scala using the AWS SDK v2, Cats Effect for asynchronous operations, and FS2 for streaming.
## Why choose Cloudflare R2?
R2 is object storage that speaks the same language as Amazon S3 while charging **zero** for data leaving its buckets (egress). That’s perfect for edge-heavy workloads or public downloads where bandwidth prices quickly add up. Because the API matches S3, you can leverage the robust AWS SDKs.
## Set up your Scala project
Add the necessary dependencies to your `build.sbt` file. We'll use the AWS SDK v2 for S3 interactions, Cats Effect for managing asynchronous operations, and FS2 for file streaming.
```scala
// build.sbt
libraryDependencies ++= Seq(
// AWS SDK v2 for S3 (compatible with R2)
"software.amazon.awssdk" % "s3" % "2.25.12",
// Cats Effect for asynchronous programming
"org.typelevel" %% "cats-effect" % "3.6.1",
// FS2 for streaming
"co.fs2" %% "fs2-core" % "3.10.2",
"co.fs2" %% "fs2-io" % "3.10.2",
// Bridge FS2 streams to Reactive Streams (for AWS SDK AsyncRequestBody)
"co.fs2" %% "fs2-reactive-streams" % "3.10.2"
)
```
## Configure the AWS SDK client
You need to configure the S3 client with your R2 credentials and the specific R2 endpoint. R2 requires the region to be set, typically `auto`, but the SDK might require a specific AWS region like `us-east-1` for signing purposes, even though the endpoint points to Cloudflare.
```scala
import software.amazon.awssdk.auth.credentials.{AwsBasicCredentials, StaticCredentialsProvider}
import software.amazon.awssdk.regions.Region
import software.amazon.awssdk.services.s3.S3Client
import software.amazon.awssdk.services.s3.S3AsyncClient
import java.net.URI
val accountId = sys.env("R2_ACCOUNT_ID") // Your Cloudflare Account ID
val accessKey = sys.env("R2_ACCESS_KEY_ID") // Your R2 Access Key ID
val secretKey = sys.env("R2_SECRET_ACCESS_KEY") // Your R2 Secret Access Key
val r2Endpoint = s"https://$accountId.r2.cloudflarestorage.com"
val credentials = AwsBasicCredentials.create(accessKey, secretKey)
val credProvider = StaticCredentialsProvider.create(credentials)
// Synchronous Client
val s3SyncClient: S3Client = S3Client.builder()
.credentialsProvider(credProvider)
.region(Region.US_EAST_1) // Use a valid AWS region for signing
.endpointOverride(URI.create(r2Endpoint))
.build()
// Asynchronous Client (for Cats Effect / FS2 integration)
val s3AsyncClient: S3AsyncClient = S3AsyncClient.builder()
.credentialsProvider(credProvider)
.region(Region.US_EAST_1) // Use a valid AWS region for signing
.endpointOverride(URI.create(r2Endpoint))
.build()
```
###### Note
Always fetch credentials securely, for example, from environment variables or a secrets manager, not hardcoded in your source code.
## Basic synchronous upload
For simple cases or scripts, a synchronous upload is straightforward using the `S3Client`.
```scala
import software.amazon.awssdk.services.s3.model.PutObjectRequest
import software.amazon.awssdk.core.sync.RequestBody
import java.nio.file.Paths
val bucketName = "your-r2-bucket-name"
val fileKey = "uploads/hello.txt" // The object key (path) in the bucket
val filePath = Paths.get("./hello.txt") // Local file to upload
val putReq = PutObjectRequest.builder()
.bucket(bucketName)
.key(fileKey)
.contentType("text/plain") // Set appropriate content type
.build()
try {
val response = s3SyncClient.putObject(putReq, RequestBody.fromFile(filePath))
println(s"Upload successful! ETag: ${response.eTag()}")
} catch {
case e: Exception => println(s"Upload failed: ${e.getMessage}")
}
```
The AWS SDK automatically handles signing the request (AWS Signature V4) using the provided credentials.
## Asynchronous upload with cats effect
For better performance and integration into non-blocking applications, use the `S3AsyncClient` with Cats Effect. We need a small helper to bridge `CompletableFuture` (used by the SDK) to `IO`.
```scala
import cats.effect.{IO, IOApp, Outcome}
import software.amazon.awssdk.services.s3.model.PutObjectResponse
import java.util.concurrent.CompletableFuture
// Helper to convert CompletableFuture to IO
def fromCompletableFuture[A](cfa: => CompletableFuture[A]): IO[A] = {
IO.async_ { cb =>
IO {
val cf = cfa
cf.handle[Unit] { (res, err) =>
err match {
case null => cb(Right(res))
case ex => cb(Left(ex))
}
}
Some(IO(cf.cancel(true))) // Cancellation logic
}
}
}
object AsyncUploader extends IOApp.Simple {
val bucketName = "your-r2-bucket-name"
val fileKey = "uploads/async-hello.txt"
val filePath = Paths.get("./hello.txt")
val putReq = PutObjectRequest.builder()
.bucket(bucketName)
.key(fileKey)
.contentType("text/plain")
.build()
def run: IO[Unit] = {
val uploadIO: IO[PutObjectResponse] = fromCompletableFuture {
s3AsyncClient.putObject(putReq, RequestBody.fromFile(filePath))
}
uploadIO.flatMap(resp => IO.println(s"Async upload successful! ETag: ${resp.eTag()}"))
.handleErrorWith(err => IO.println(s"Async upload failed: ${err.getMessage}"))
.guaranteeCase { // Ensure client cleanup
case Outcome.Succeeded(_) | Outcome.Errored(_) | Outcome.Canceled() =>
IO(s3AsyncClient.close()) *> IO(s3SyncClient.close()) // Close both if used
}
}
}
```
## Handling responses and errors
The AWS SDK throws specific exceptions for different error conditions (e.g., `S3Exception`,`NoSuchBucketException`, `SdkClientException`). You can catch these for fine-grained error handling.
```scala
import software.amazon.awssdk.services.s3.model.S3Exception
import software.amazon.awssdk.core.exception.SdkClientException
// Inside your IO error handler or try-catch block
// ...
.handleErrorWith {
case e: S3Exception if e.statusCode() == 403 =>
IO.println("Access Denied (403): Check credentials or bucket policy.")
case e: S3Exception if e.statusCode() == 404 =>
IO.println("Not Found (404): Check bucket name or object key.")
case e: S3Exception =>
IO.println(s"S3 Service Error (${e.statusCode()}): ${e.awsErrorDetails().errorMessage()}")
case e: SdkClientException =>
IO.println(s"SDK Client Error (e.g., network issue): ${e.getMessage}")
case e: Exception =>
IO.println(s"Generic Error: ${e.getMessage}")
}
```
## Retry logic with exponential back-off
Network glitches or temporary service issues (like rate limiting) happen. Implement a retry mechanism using Cats Effect.
```scala
import scala.concurrent.duration.*
import cats.effect.Temporal
def retryIO[A](ioa: IO[A], maxRetries: Int, delay: FiniteDuration)(implicit T: Temporal[IO]): IO[A] = {
ioa.handleErrorWith { error =>
if (maxRetries > 0) {
// Add specific checks here, e.g., only retry on 429 or network errors
T.sleep(delay) *> retryIO(ioa, maxRetries - 1, delay * 2) // Exponential backoff
} else {
IO.raiseError(error) // Max retries reached
}
}
}
// Example usage:
// val uploadIO = fromCompletableFuture(...)
// retryIO(uploadIO, maxRetries = 3, delay = 1.second).flatMap(...)
```
## Upload many files in parallel
Use `parTraverseN` from Cats Effect to upload multiple files concurrently, limiting the concurrency to avoid overwhelming resources.
```scala
import cats.syntax.all.*
import cats.effect.implicits.*
case class FileTask(localPath: String, targetKey: String)
def uploadFileAsync(task: FileTask): IO[PutObjectResponse] = {
val putReq = PutObjectRequest.builder()
.bucket(bucketName)
.key(task.targetKey)
.build() // ContentType could be inferred or set dynamically
fromCompletableFuture {
s3AsyncClient.putObject(putReq, RequestBody.fromFile(Paths.get(task.localPath)))
}
}
val filesToUpload = List(
FileTask("./a.txt", "uploads/a.txt"),
FileTask("./b.jpg", "uploads/b.jpg"),
FileTask("./c.pdf", "uploads/c.pdf")
)
val parallelUploadsIO: IO[List[PutObjectResponse]] =
filesToUpload.parTraverseN(4)(uploadFileAsync) // Limit to 4 concurrent uploads
// Execute parallelUploadsIO within your IOApp or effect context
// Remember error handling for individual uploads within the list
```
## Stream large files using fs2
For large files, avoid loading the entire content into memory. Stream the file using FS2 and the AWS SDK's `AsyncRequestBody`.
```scala
import fs2.io.file.{Files, Path => Fs2Path}
import fs2.interop.reactivestreams.*
import software.amazon.awssdk.core.async.AsyncRequestBody
import cats.effect.kernel.Resource
def uploadStream(localPath: String, targetKey: String, contentType: String): IO[PutObjectResponse] = {
val fs2Path = Fs2Path(localPath)
// Resource to ensure the file stream is closed
Files[IO].size(fs2Path).flatMap { fileSize =>
val fileStream: fs2.Stream[IO, Byte] = Files[IO].readAll(fs2Path, 64 * 1024) // 64KB chunks
// Convert FS2 stream to Reactive Streams Publisher
val publisher = fileStream.toUnicastPublisher()
Resource.eval(publisher).use { pub =>
val requestBody = AsyncRequestBody.fromPublisher(pub)
val putReq = PutObjectRequest.builder()
.bucket(bucketName)
.key(targetKey)
.contentType(contentType)
// .contentLength(fileSize) // Recommended for progress tracking
.build()
fromCompletableFuture(s3AsyncClient.putObject(putReq, requestBody))
}
}
}
// Example usage:
// uploadStream("./large-video.mp4", "uploads/large-video.mp4", "video/mp4").flatMap(...)
```
## Security best practices
1. **Secrets Management:** Never hardcode credentials. Use environment variables, configuration files outside source control, or dedicated secrets management services.
2. **Least Privilege:** Create R2 API tokens with the minimum permissions required (e.g., only`PutObject` for a specific bucket).
3. **Input Validation:** Validate file types, sizes, and potentially scan user-uploaded content\_before\_ uploading to R2 to prevent abuse and storage of malicious files.
4. **HTTPS:** Always use HTTPS. The R2 endpoint enforces this, but ensure your client configuration doesn't disable TLS verification.
5. **Bucket Policies:** Configure R2 bucket policies to restrict access further, complementing API token permissions.
## Common errors and their fixes
|Error|Likely Cause|Quick Fix|
|-|-|-|
|403 Forbidden|Invalid credentials or insufficient permissions|Verify Access Key/Secret Key, check R2 token permissions|
|404 Not Found|Bucket name typo or non-existent bucket|Double-check bucket name spelling|
|InvalidSignatureException|Incorrect signing (clock skew, wrong region)|Sync server time (NTP), ensure SDK uses a valid region for signing (us-east-1)|
|429 Too Many Requests|Exceeded R2 API request rate limits|Implement exponential back-off and retry logic|
|SdkClientException|Network issue, DNS resolution failure, timeout|Check network connectivity, increase client timeouts if needed|
|Timeout Error|Slow network or very large file upload|Increase SDK client timeouts, use streaming/multipart upload|
## Wrap-up
Using the AWS SDK v2 for Java provides a robust and idiomatic way to interact with Cloudflare R2 from Scala. By leveraging the SDK's features combined with libraries like Cats Effect and FS2, you can build efficient, asynchronous, and resilient file export solutions. Remember to handle errors gracefully, implement retries, and follow security best practices.
If you prefer a managed solution for file processing pipelines that include exporting to various storage providers like R2, the/our [🤖 /cloudflare/store](/docs/robots/cloudflare-store.md) Robot automates R2 exports within a Transloadit workflow—no extra SDK integration required on your end.
\#scala#cloudflare-r2#file-export#aws-sdk#s3#asynchronous-programming#cats-effect#fs2#file-exporting-service
### 👩💻 Join 20k+ developers
Sign up for our [monthly newsletter](/newsletters.md) to receive direct links to 3 exclusive tech — and 2 product updates. No less, no more.
Your email:
Get access
## File uploading and encoding. Made simple.
Transloadit streamlines file handling for developers, trusted by brands like Coursera and The New York Times. We’re known for a reliable API, top-notch support, and a strong commitment to open source, with projects like [Uppy](https://uppy.io) and [Tus](https://tus.io) setting standards in file processing.
[Sign up](/c/)[Book a Demo](https://survey.typeform.com/to/kRg47Xi5)
No credit card needed · 5 GB included in the free plan
Cancel anytime