Efficiently import files from Supabase in Python

Kevin van Zonneveld

Co-founder · Amsterdam, The Netherlands ·

Supabase stores objects in object storage and their metadata in PostgreSQL, making it ideal for managing large files, assets, and user uploads. In this DevTip, we demonstrate how to import files from Supabase using Python with practical examples and best practices for handling both single files and directories.

Introduction to Supabase storage

Supabase Storage provides a simple yet powerful way to store and serve large files. By organizing files into buckets—much like AWS S3—it becomes easy to manage application assets, user uploads, and other file-based content. This approach ensures efficient storage and quick retrieval when needed.

Overview of the Python ecosystem for file imports

Python boasts a rich ecosystem for file handling and API interactions. The official open-source Python SDK for Supabase simplifies the process of connecting to your Supabase projects and managing storage buckets. This guide leverages that SDK to streamline file imports into your Python applications.

Step-by-step guide for setting up Supabase and Python integration

1. Install required Python libraries

Before starting, ensure you are running Python 3.9 or higher. Install the official Supabase Python client using pip:

pip install supabase==2.13.0

2. Configure access credentials and permissions in Supabase

Access your Supabase project settings to locate your API URL and API key under the "API" section. Use the project's legacy anon JWT key and authenticate a user whose Storage policies allow the operations below. The pinned SDK 2.13.0 validates project keys as JWTs, so it does not accept the newer sb_publishable_… key format. Initialize the client before using any bucket example:

import os
import tempfile
from pathlib import Path
from storage3.exceptions import StorageApiError
from supabase import create_client, Client

supabase: Client = create_client(
    os.environ['SUPABASE_URL'], os.environ['SUPABASE_PROJECT_KEY']
)
supabase.auth.sign_in_with_password({
    'email': os.environ['SUPABASE_USER_EMAIL'],
    'password': os.environ['SUPABASE_USER_PASSWORD'],
})

Create a private bucket in the dashboard, or use the following optional provisioning example only with an identity permitted to create buckets. Ordinary download users do not need that permission:

supabase.storage.create_bucket('my-bucket',
    options={
        "public": False,
        "allowed_mime_types": ["image/png", "image/jpeg"],
        "file_size_limit": 1024000,  # Limit file size to ~1 MB
    }
)

These options restrict declared MIME types and upload sizes; they do not scan content. Storage access policies determine which authenticated users can list and download objects.

3. Establish a connection to your Supabase bucket

Reuse the authenticated supabase client initialized above for the following downloads.

How to import files from Supabase using Python

Importing a single file

The following function demonstrates how to download a single file from a specified bucket. It retrieves the file from Supabase and writes it to a local destination:

def download_file(bucket_name: str, file_path: str, destination: str) -> bool:
    try:
        response = supabase.storage.from_(bucket_name).download(file_path)
        target = Path(destination)
        target.parent.mkdir(parents=True, exist_ok=True)
        # Replace the destination only after a complete download and local write.
        with tempfile.TemporaryDirectory(dir=target.parent) as staging:
            temporary = Path(staging) / 'download'
            temporary.write_bytes(response)
            os.replace(temporary, target)
        print(f"File downloaded successfully to {destination}")
        return True
    except StorageApiError as e:
        print(f"Storage error: {e}")
    except Exception as e:
        print(f"Error downloading file: {str(e)}")
    return False

# Example usage
download_file('my-bucket', 'folder/image.jpg', 'local/image.jpg')

This synchronous SDK call holds the downloaded bytes in memory. Use it for files that fit in available memory; asynchronous execution alone does not make it a streaming download. A failed download or write leaves an existing destination unchanged.

Importing multiple files from a directory

The following function pages through the immediate files in one remote folder, skipping subfolders. It saves those files directly into a trusted local destination directory. It does not mirror a remote directory tree, and remote keys are never interpreted as local directory paths:

def import_directory(bucket_name: str, prefix: str = "", destination: str = "downloads") -> None:
    try:
        root = Path(destination).resolve()
        prefix = prefix.rstrip('/')
        offset = 0
        page_size = 100
        while True:
            files = supabase.storage.from_(bucket_name).list(path=prefix, options={
                'limit': page_size, 'offset': offset,
                'sortBy': {'column': 'name', 'order': 'asc'},
            })
            for file in files:
                if file.get('id') is None:
                    continue  # Folder placeholders have no object ID.
                name = file['name']
                if not name or name in {'.', '..'} or any(char in name for char in '/\\:\0'):
                    raise ValueError('Unsafe object name in listing')
                file_path = f'{prefix}/{name}' if prefix else name
                if not download_file(bucket_name, file_path, str(root / name)):
                    raise RuntimeError('Import stopped after a failed download')
            if len(files) < page_size:
                break
            offset += page_size

    except StorageApiError as e:
        print(f"Storage error: {e}")
    except Exception as e:
        print(f"Error importing directory: {str(e)}")

# Example usage
import_directory('my-bucket', 'images/')

Storage limitations and considerations

File size limits and storage quotas depend on the plan and project configuration. Consult Supabase's file limit documentation.
Keep downloads within this example's memory budget. Use bounded concurrency and handle throttling responses when building a larger importer.
Offset pagination assumes the remote folder is stable while listing. Concurrent changes may cause entries to be missed or repeated.

Authentication and security

Before performing storage operations, always authenticate your users. For example, sign in with a user's email and password to ensure that subsequent operations are performed within a secure context:

# Authenticate the user prior to executing storage operations
user = supabase.auth.sign_in_with_password({
    "email": os.environ['SUPABASE_USER_EMAIL'],
    "password": os.environ['SUPABASE_USER_PASSWORD']
})

# Proceed with storage operations using the authenticated context
response = supabase.storage.from_("private-bucket").download("file.txt")

Remember to avoid hard-coding sensitive credentials; instead, use secure environment variables or configuration files.

Best practices for importing files from Supabase

Use proper authentication: Ensure that the user is authenticated before initiating storage operations.
Handle exceptions specifically: Use StorageApiError to catch and manage file storage errors precisely.
Configure bucket policies: Set strict access controls and allowable MIME types to safeguard your files.
Validate file types and sizes: Rely on bucket configuration to enforce server-side restrictions.
Secure your credentials: Store API keys and secrets in environment variables rather than embedding them in code.

Additional resources to optimize Supabase and Python workflows

Conclusion

Importing files from Supabase in Python is straightforward with the proper setup and security measures. The official Supabase Python SDK offers a robust interface for handling both single file downloads and bulk imports while maintaining secure access protocols.

For advanced file processing or further automation, consider exploring Transloadit's Python SDK for powerful file handling solutions.