Using rclone to manage files in DigitalOcean Spaces via CLI
Use rclone’s S3 backend to upload a directory to a private DigitalOcean Space, list its objects,
and download the files again. This walkthrough checks the downloaded bytes and demonstrates what
sync deletes inside a dedicated demo prefix, without making the bucket public.
Prepare a private Space
You need an existing Standard Storage Space and its regional origin endpoint, such as
fra1.digitaloceanspaces.com. Use a Spaces access key with read, write, and delete access to that
bucket. A DigitalOcean API token is a different credential. DigitalOcean’s
access-key guide explains
bucket-scoped keys; do not copy an AWS IAM policy into Spaces and assume it grants the same access.
The commands below target Linux on Intel/AMD 64-bit processors, with Bash, cURL, unzip, and GNU coreutils installed. The tested rclone release is 1.75.2. File names in this example use UTF-8, including a nested binary file; the empty file is intentional. Keep this small sequential exercise separate from a working backup or a directory that another process is changing.
Install rclone in a new working directory
Paste this block from a writable directory. It downloads the
versioned rclone release, checks its published SHA-256
checksum, and installs the executable locally. It refuses an existing rclone-spaces directory.
The parentheses keep navigation and shell options inside a subshell.
(
set -eu
for tool in curl sha256sum unzip install mv mkdir; do
command -v "$tool" >/dev/null || { echo "Missing prerequisite: $tool" >&2; exit 1; }
done
mkdir rclone-spaces &&
cd rclone-spaces &&
curl -fsSLo rclone.zip https://downloads.rclone.org/v1.75.2/rclone-v1.75.2-linux-amd64.zip &&
curl -fsSLo SHA256SUMS https://downloads.rclone.org/v1.75.2/SHA256SUMS &&
mv rclone.zip rclone-v1.75.2-linux-amd64.zip &&
sha256sum --check --ignore-missing SHA256SUMS &&
unzip -q rclone-v1.75.2-linux-amd64.zip &&
install -m 755 rclone-v1.75.2-linux-amd64/rclone rclone &&
./rclone version
)
The version output should start with rclone v1.75.2. If downloading or checking the archive fails,
stop there. Inspect the partial rclone-spaces directory before removing it yourself and repeating
setup; the installer does not overwrite an existing project. Run the remaining commands from inside
this new directory.
Configure the Spaces remote
From inside rclone-spaces, create a private, empty configuration file. This refuses an existing
file and leaves your usual rclone configuration alone:
(umask 077; set -C; : > spaces.conf)
Edit spaces.conf to contain the following settings, replacing the credential placeholders and
endpoint. Keep this file outside version control.
[spaces]
type = s3
provider = DigitalOcean
env_auth = false
access_key_id = YOUR_SPACES_ACCESS_KEY
secret_access_key = YOUR_SPACES_SECRET_KEY
endpoint = fra1.digitaloceanspaces.com
region =
location_constraint =
acl = private
no_check_bucket = true
list_version = 1
The DigitalOcean backend instructions leave region
and location_constraint blank: the endpoint identifies the region. Use the origin endpoint,
without a bucket name or CDN hostname. Setting
no_check_bucket skips bucket existence/creation checks;
it assumes the bucket already exists and does not grant object permissions. This example explicitly
uses ListObjects version 1 for listing.
Upload, list, and download a file tree
Save the following as spaces-demo.sh in the same working directory. Pass your bucket name and a
fresh UUID when running it. The script uses that UUID to isolate its remote destination and refuses
a nonempty prefix or existing local upload/download directories.
#!/usr/bin/env bash
set -euo pipefail
space=${1:?Supply a bucket name}
run_id=${2:?Supply a fresh UUID}
[[ "$space" =~ ^[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$ ]] || { echo 'Invalid bucket name' >&2; exit 1; }
[[ "$run_id" =~ ^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$ ]] || { echo 'Invalid UUID' >&2; exit 1; }
remote="spaces:$space/rclone-demo-$run_id"
[[ ! -e upload && ! -e download ]] || { echo 'Use a fresh local working directory' >&2; exit 1; }
existing=$(./rclone --config ./spaces.conf lsf "$remote" --recursive)
[[ -z "$existing" ]] || { echo 'Choose a new, empty demo prefix' >&2; exit 1; }
mkdir upload
mkdir download
mkdir upload/nested
printf 'Hello from rclone\n' > upload/hello.txt
printf '\000\001\177\200\377' > 'upload/nested/café 💾.bin'
: > upload/empty.txt
printf 'Remove this demo object during sync\n' > upload/obsolete.txt
./rclone --config ./spaces.conf copy ./upload "$remote"
./rclone --config ./spaces.conf lsjson "$remote" --recursive --files-only --s3-list-chunk 2
./rclone --config ./spaces.conf copy "$remote" ./download
./rclone --config ./spaces.conf check ./upload ./download --download
printf 'Verified the downloaded file tree. Remote prefix: %s\n' "$remote"
Run it with your Space’s bucket name. Linux supplies a new UUID through the file used below:
bash spaces-demo.sh your-space "$(cat /proc/sys/kernel/random/uuid)"
The JSON listing should contain hello.txt, empty.txt, obsolete.txt, and
nested/café 💾.bin. The two-object listing page size deliberately exercises pagination on this
small tree; it is not a throughput recommendation. copy transfers a directory’s contents, so the
remote keys do not include an extra upload/ directory. It
preserves destination-only files, but it can replace
changed files with matching names. It is not a no-overwrite operation.
The final check --download reads both local trees
and compares their contents. A successful transfer status or a matching file size alone is not this
byte check. If any command fails, the script stops; completed uploads can remain in the demo prefix,
and the local download can be incomplete. Inspect those results before retrying. For a new run,
use a fresh working directory and UUID rather than deleting an unfamiliar destination to make the
script proceed.
Preview a scoped sync before deleting anything
sync makes the destination match the source, including
deleting destination-only files. It is one-way, not a merge of independent edits. Reversing the
arguments would make the local directory the destination and could delete local-only files.
After the transfer succeeds, copy the printed remote prefix into the command below. Save the block
as spaces-sync.sh. It removes only the local demo’s obsolete.txt, then previews a sync into the
same UUID prefix. Review the preview before confirming the actual sync. Do not substitute the bucket
root or a prefix shared with another writer.
#!/usr/bin/env bash
set -euo pipefail
remote=${1:?Supply the printed demo prefix}
[[ "$remote" =~ ^spaces:[a-z0-9][a-z0-9-]{1,61}[a-z0-9]/rclone-demo-[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$ ]] || { echo 'Expected a demo prefix' >&2; exit 1; }
rm -f ./upload/obsolete.txt
./rclone --config ./spaces.conf sync ./upload "$remote" --dry-run
read -r -p 'Type sync to apply these destination changes: ' answer
[[ "$answer" == sync ]] || { echo 'Sync canceled'; exit 1; }
./rclone --config ./spaces.conf sync ./upload "$remote"
./rclone --config ./spaces.conf check ./upload "$remote" --download
bash spaces-sync.sh spaces:your-space/rclone-demo-REPLACE_WITH_THE_PRINTED_UUID
The preview should name obsolete.txt for deletion. After confirmation, the remote prefix should
contain the other three files, and the last check compares their bytes against upload. Your
original download still contains all four files. A dry run describes the state it observed;
it does not lock the destination against later changes. Keep other writers out of this prefix.
The remaining demo objects stay in your Space until you remove them.
Diagnose a failed command
Check the command’s exit status and diagnostic before moving on. An access refusal can come from an
invalid key or insufficient bucket permissions; it does not justify making the files public. Check
the selected spaces.conf, bucket name, and origin endpoint. Listing a known bucket prefix is a
better test for this tutorial than listing every bucket in the account.
A failed copy can leave some successful files behind. Use the listing to identify them and repeat
the byte check after completing the transfer. Do not run sync as a repair step for an incomplete
upload. A mismatch from check means you should inspect the named file before using its download.
For scheduling, filtering, or client-side encryption beyond this exercise, consult the
rclone documentation. These require their own configuration. The s3 settings above do not apply rclone’s client-side
encryption or compression.