Efficient Flask file uploads: a step-by-step guide

Kevin van Zonneveld

Co-founder · Amsterdam, The Netherlands ·

File uploads are a crucial feature in modern web applications, enabling users to share and store data efficiently. Flask, a lightweight yet powerful Python web framework, offers robust capabilities to handle file uploads securely and efficiently. In this step-by-step guide, we'll explore how to implement file uploads in Flask applications, covering best practices, security measures, and advanced techniques.

Setting up your Flask environment

To get started with file uploads in Flask, we'll set up a basic Flask application with the necessary dependencies. First, install the system dependencies for python-magic:

# For Debian/Ubuntu
sudo apt-get install libmagic1

# For macOS
brew install libmagic

# For Windows
# Install libmagic DLLs separately as described in python-magic's Windows instructions.

Now install Flask and its dependencies using pip:

pip install flask==3.1.0 flask-wtf==1.2.2 python-magic==0.4.27
export FLASK_SECRET_KEY="$(python -c 'import secrets; print(secrets.token_hex(32))')"

On Windows, follow the python-magic installation instructions for compatible libmagic binaries; they are not installed automatically. Set FLASK_SECRET_KEY in your shell or deployment's secret configuration and keep it stable across application workers.

Let's create a basic Flask application structure in a file named app.py:

from flask import Flask
import os

app = Flask(__name__)
app.config['SECRET_KEY'] = os.environ['FLASK_SECRET_KEY']
app.config['UPLOAD_FOLDER'] = os.path.join(app.instance_path, 'uploads')
app.config['MAX_CONTENT_LENGTH'] = 16 * 1024 * 1024  # 16 MiB total request limit

# Ensure the upload folder exists
os.makedirs(app.config['UPLOAD_FOLDER'], mode=0o700, exist_ok=True)

Configuration Explained:

SECRET_KEY: A secret key used by Flask-WTF for securely signing the session cookie and for other security-related needs.
UPLOAD_FOLDER: Private storage outside Flask's static directory. Do not expose the instance directory through your web server.
MAX_CONTENT_LENGTH: The maximum total request size (16 MiB), including multipart fields and overhead. The allowed file size is slightly smaller.

Building the file upload form

To facilitate file uploads, we'll create a Flask-WTF form that includes validation for the uploaded files.

First, create a form class in forms.py:

from flask_wtf import FlaskForm
from flask_wtf.file import FileField, FileRequired, FileAllowed

class UploadForm(FlaskForm):
    file = FileField('File', validators=[
        FileRequired(),
        FileAllowed(['jpg', 'jpeg', 'png', 'pdf'], 'Allowed file types are jpg, jpeg, png, pdf')
    ])

This form uses FileField for file input and includes validators to ensure that a file is provided and that it has an allowed file extension.

Next, create a template templates/upload.html for the upload form:

<!doctype html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <title>Upload File</title>
  </head>
  <body>
    <h1>Upload File</h1>
    <form method="POST" enctype="multipart/form-data">
      {{ form.hidden_tag() }} {{ form.file.label }} {{ form.file }}
      {% for errors in form.errors.values() %}
      {% for error in errors %}<p role="alert">{{ error }}</p>{% endfor %}
      {% endfor %}
      <input type="submit" value="Upload" />
    </form>
    {% with messages = get_flashed_messages(with_categories=true) %} {% if messages %}
    <ul>
      {% for category, message in messages %}
      <li class="{{ category }}">{{ message }}</li>
      {% endfor %}
    </ul>
    {% endif %} {% endwith %}
  </body>
</html>

This template renders the form and includes a section to display flashed messages for user feedback.

Implementing file upload handling

Now, let's handle file uploads in our Flask application with robust MIME type validation, file extension checks, and error handling. Update your app.py to include the necessary imports and routes:

from flask import render_template, redirect, url_for, flash, request, jsonify
from werkzeug.exceptions import HTTPException, RequestEntityTooLarge
from forms import UploadForm
import magic
import tempfile

ALLOWED_TYPES = {
    'image/jpeg': ('jpg', 'jpeg'),
    'image/png': ('png',),
    'application/pdf': ('pdf',),
}

def save_upload(file):
    if file is None or not file.filename:
        raise ValueError('No file selected')
    extension = file.filename.rsplit('.', 1)[-1].lower()
    file_type = magic.from_buffer(file.read(2048), mime=True)
    file.seek(0)
    extensions = ALLOWED_TYPES.get(file_type)
    if extensions is None or extension not in extensions:
        raise ValueError('File type or extension not allowed')

    # Exclusive temporary-file creation gives unique names and mode 0600.
    with tempfile.NamedTemporaryFile(
        dir=app.config['UPLOAD_FOLDER'], prefix='upload-',
        suffix='.' + extensions[0], delete=False,
    ) as target:
        try:
            file.save(target)
        except Exception:
            os.unlink(target.name)
            raise
    return os.path.basename(target.name)

# Error handler for file size limit
@app.errorhandler(RequestEntityTooLarge)
def handle_file_too_large(e):
    return jsonify({'error': 'The upload request exceeds 16 MiB.'}), 413

@app.route('/', methods=['GET', 'POST'])
def upload():
    form = UploadForm()
    if form.validate_on_submit():
        try:
            save_upload(form.file.data)
            flash('File uploaded successfully', 'success')
        except ValueError as e:
            flash(str(e), 'danger')
        except Exception as e:
            app.logger.error('Upload failed (%s)', type(e).__name__)
            flash('The file could not be saved. Please try again.', 'danger')
        return redirect(url_for('upload'))
    return render_template('upload.html', form=form)

Security Features:

MIME sniffing checks a file signature, but does not prove the file is harmless. Scan or process untrusted content appropriately before making it available to other users.
The helper checks that the extension agrees with the detected type.
Server-generated names and extensions prevent user filenames from controlling local paths.
Enforcing a file size limit protects against resource exhaustion.
Client errors are sanitized; server diagnostics record only the exception type.

Creating a REST API endpoint

For programmatic file uploads, let's implement a REST API endpoint:

@app.route('/api/upload', methods=['POST'])
def api_upload():
    try:
        filename = save_upload(request.files.get('file'))
        return jsonify({
            'message': 'File uploaded successfully',
            'filename': filename
        }), 200
    except ValueError as e:
        return jsonify({'error': str(e)}), 400

Testing the API Endpoint:

After adding the routes and error handler to app.py, start Flask, then use cURL from another terminal to test the API endpoint:

flask --app app run

curl -F 'file=@/path/to/your/file.jpg' http://localhost:5000/api/upload

Handling large file uploads

For efficient handling of large file uploads, consider these strategies:

Configure nginx for large uploads

If using Nginx as a reverse proxy, add these settings to your configuration:

http {
    client_max_body_size 16M;
    proxy_read_timeout 600;
    proxy_connect_timeout 600;
    proxy_send_timeout 600;
}

Implement chunked uploads

For large file uploads, consider using a chunked upload approach. The tus protocol is an excellent choice for this purpose, providing resumable upload capabilities.

Use asynchronous processing

Celery moves processing to a worker after the HTTP upload is saved; it does not remove the request size limit. This optional worker example targets Linux/macOS. Install Celery with its Redis transport, install Redis, and run a local Redis broker:

pip install 'celery[redis]==5.6.3'
redis-server

Append this code to app.py. The example computes a SHA-256 digest in a worker:

from celery import Celery
import hashlib

celery = Celery('tasks', broker='redis://localhost:6379/0')

@celery.task
def process_uploaded_file(file_path):
    digest = hashlib.sha256()
    with open(file_path, 'rb') as source:
        for chunk in iter(lambda: source.read(1024 * 1024), b''):
            digest.update(chunk)
    return digest.hexdigest()

@app.route('/upload-large', methods=['POST'])
def upload_large_file():
    try:
        filename = save_upload(request.files.get('file'))
    except ValueError as e:
        return jsonify({'error': str(e)}), 400
    file_path = os.path.join(app.config['UPLOAD_FOLDER'], filename)

    # Queue the file for processing
    task = process_uploaded_file.delay(file_path)
    return jsonify({'message': 'File uploaded and queued for processing', 'task_id': task.id}), 202

Run these in separate terminals from the project directory, with FLASK_SECRET_KEY set in both:

celery -A app:celery worker --loglevel=INFO
flask --app app run

The web process and worker must share the same private upload directory. This example has no result backend or status endpoint. A broker failure returns a sanitized error; the saved file remains for operator cleanup. Keep Redis private. The API examples demonstrate validation, so add your application's authentication, authorization, and rate limits before exposing them publicly.

Error handling and validation

Implement comprehensive error handling to improve reliability:

@app.errorhandler(Exception)
def handle_unexpected_error(error):
    if isinstance(error, HTTPException):
        return error
    app.logger.error('Request failed (%s)', type(error).__name__)
    return jsonify({'error': 'An unexpected error occurred'}), 500

Conclusion

Implementing secure and efficient file uploads in Flask requires careful attention to security, performance, and user experience. By following best practices and incorporating robust validation and error handling, you can build a reliable file upload system for your Flask applications.

For more advanced file upload capabilities, consider using open-source tools like tus or Uppy, which can be integrated with Flask to support features such as resumable uploads and progress tracking. Additionally, Transloadit offers comprehensive file uploading and processing services.