Last updated: February 5, 2025
# Efficiently export files to Microsoft Azure using cURL
#### Tim Koschützki
Co-founder · Berlin, Germany · Show bio
[](https://x.com/tim%5Fkos)[](https://github.com/tim-kos)
Exporting files to Microsoft Azure Storage using cURL provides a flexible and efficient way to interact with cloud storage directly from the command line. This guide demonstrates how to export files to Azure Blob Storage using cURL, highlighting the recommended use of Microsoft Entra ID authentication, secure token management, and robust error handling practices.
## Setting up Azure storage account
Before using cURL with Azure Storage, create an Azure Storage account and container through the Azure Portal or Azure CLI. We recommend enabling Microsoft Entra ID (formerly Azure AD) authentication for superior security and streamlined token management.
## Authentication methods
Azure Blob Storage supports two primary authentication methods:
### Microsoft Entra ID (recommended)
Obtain an access token using the Azure CLI. Ensure you have logged in using `az login`.
```bash
token=$(az account get-access-token --resource https://storage.azure.com/ --query accessToken -o tsv)
```
Use the token in cURL requests:
```bash
curl -fsSL -X PUT \
-H "Authorization: Bearer $token" \
-H "x-ms-version: 2023-11-03" \
-H "x-ms-blob-type: BlockBlob" \
-H "Content-Type: application/octet-stream" \
--data-binary "@localfile.txt" \
"https://youraccount.blob.core.windows.net/container/remotefile.txt"
```
### Shared access Signature (alternative)
For scenarios where Microsoft Entra ID is not feasible, use SAS tokens with strict security controls. This method also supports additional integrity checks using the `Content-MD5` header:
```bash
curl -fsSL -X PUT \
-H "x-ms-blob-type: BlockBlob" \
-H "x-ms-version: 2023-11-03" \
-H "Content-Type: application/octet-stream" \
-H "Content-MD5: $(openssl dgst -md5 -binary localfile.txt | base64)" \
--data-binary "@localfile.txt" \
"https://youraccount.blob.core.windows.net/container/remotefile.txt?your_sas_token"
```
## Handling large file uploads
For files that exceed the limit of 5,000 MiB per write operation, use the Block Blob API to upload large files in chunks. With API versions 2019-12-12 and later, the maximum block size is 4,000 MiB, allowing a maximum blob size of nearly 190.7 TiB when using up to 50,000 blocks.
```bash
#!/bin/bash
file="largefile.txt"
block_size=$((4000*1024*1024)) # 4,000 MiB blocks
base_url="https://youraccount.blob.core.windows.net/container/$file"
# Split file into blocks
split -b $block_size "$file" block_
# Upload blocks
for block in block_*; do
block_id=$(echo -n "$block" | base64)
curl -fsSL -X PUT \
-H "Authorization: Bearer $token" \
-H "x-ms-version: 2023-11-03" \
-H "x-ms-blob-type: BlockBlob" \
--data-binary "@$block" \
"$base_url?comp=block&blockid=$block_id"
done
# Create block list xml
echo '' > blocklist.xml
for block in block_*; do
block_id=$(echo -n "$block" | base64)
echo "$block_id" >> blocklist.xml
done
echo '' >> blocklist.xml
# Commit blocks
curl -fsSL -X PUT \
-H "Authorization: Bearer $token" \
-H "x-ms-version: 2023-11-03" \
-H "Content-Type: application/xml" \
--data-binary @blocklist.xml \
"$base_url?comp=blocklist"
```
## Error handling and retries
Implement robust error handling with exponential backoff for transient errors. The function below retries on HTTP 408, 500, 502, 503, and 504, while immediately aborting on authentication failures (HTTP 403):
```bash
upload_with_retry() {
local url="$1"
local file="$2"
local max_attempts=5
local attempt=1
local wait_time=2
while [ $attempt -le $max_attempts ]; do
response=$(curl -fsSL -w "%{http_code}" -X PUT \
-H "Authorization: Bearer $token" \
-H "x-ms-version: 2023-11-03" \
-H "x-ms-blob-type: BlockBlob" \
--data-binary "@$file" \
"$url")
status=${response: -3}
if [ "$status" -eq 201 ]; then
return 0
elif [ "$status" -eq 403 ]; then
echo "Authentication failed. Check credentials."
return 1
elif [ "$status" -eq 408 ] || [ "$status" -eq 500 ] || [ "$status" -eq 502 ] || [ "$status" -eq 503 ] || [ "$status" -eq 504 ]; then
echo "Transient error $status encountered. Retrying in $wait_time seconds..."
else
echo "Upload failed with status $status."
return 1
fi
attempt=$((attempt + 1))
sleep $wait_time
wait_time=$((wait_time * 2))
done
echo "File upload failed after $max_attempts attempts."
return 1
}
```
## Monitoring and validation
Check the upload status and blob metadata using the following command:
```bash
curl -fsSL -I \
-H "Authorization: Bearer $token" \
-H "x-ms-version: 2023-11-03" \
"https://youraccount.blob.core.windows.net/container/file.txt"
```
Monitor transfer progress with the progress bar option:
```bash
curl -fsSL --progress-bar \
-H "Authorization: Bearer $token" \
-H "x-ms-version: 2023-11-03" \
-H "x-ms-blob-type: BlockBlob" \
--data-binary "@largefile.txt" \
"https://youraccount.blob.core.windows.net/container/largefile.txt"
```
## Security best practices
* Use Microsoft Entra ID authentication whenever possible to leverage improved security and token management.
* Always use HTTPS for secure data transfers with Azure Storage.
* Restrict permissions granted by SAS tokens and use short expiration times.
* Enforce IP address restrictions to limit access to your storage account.
* Set the minimum required permissions for each operation.
* Monitor and log all operations for audit and troubleshooting purposes.
* Regularly rotate credentials to mitigate potential security breaches.
* Validate file integrity using the `Content-MD5` header.
## Performance optimization
To maximize transfer performance:
* Choose appropriate block sizes (up to 4,000 MiB per block).
* Enable concurrent uploads where possible for multiple files.
* Select the closest Azure region to reduce latency.
* Monitor network bandwidth and latency to optimize transfers.
* Use compression when appropriate, considering that some files are already compressed.
* Implement comprehensive error handling and retries for robust uploads.
## Conclusion
Integrating Azure Blob Storage with cURL offers a powerful strategy for managing cloud storage operations. By following these updated authentication methods, security best practices, and performance optimization tips, you can build a reliable and secure file transfer solution. For additional advanced file handling capabilities, consider exploring Transloadit's services, which support features like Uppy and Tus for enhanced file processing.
\#dotnet#tus#file-uploading#resumable-uploads#handling-uploads-service
### 👩💻 Join 20k+ developers
Sign up for our [monthly newsletter](/newsletters.md) to receive direct links to 3 exclusive tech — and 2 product updates. No less, no more.
Your email:
Get access
## File uploading and encoding. Made simple.
Transloadit streamlines file handling for developers, trusted by brands like Coursera and The New York Times. We’re known for a reliable API, top-notch support, and a strong commitment to open source, with projects like [Uppy](https://uppy.io) and [Tus](https://tus.io) setting standards in file processing.
[Sign up](/c/)[Book a Demo](https://survey.typeform.com/to/kRg47Xi5)
No credit card needed · 5 GB included in the free plan
Cancel anytime