OpenStack Swift provides scalable and reliable object storage, making it an excellent choice for .NET developers looking to integrate cloud storage into their applications. In this guide, we explore how to export files to OpenStack Swift using .NET C# with practical examples and up-to-date code snippets to help you implement this functionality in your projects.

Introduction to OpenStack Swift

OpenStack Swift is an open-source object storage system that is highly available, distributed, and scalable. It allows you to store and retrieve large amounts of unstructured data through its RESTful API interface.

For .NET developers, integrating OpenStack Swift enhances your applications with robust cloud storage capabilities for various file types, including images, videos, and documents.

Setting up your environment

Prerequisites

Before starting, ensure you have:

  • Visual Studio or Visual Studio Code
  • .NET 6.0 or later (LTS) or .NET 7.0 (Current)
  • An OpenStack Swift account with valid credentials
  • NuGet package manager

Install the SwiftClient package via NuGet:

dotnet add package SwiftClient

Implementing Swift storage operations

First, create a Swift storage service with proper configuration:

using SwiftClient;
using Microsoft.Extensions.Configuration;
using System;
using System.Collections.Generic;
using System.IO;
using System.Threading.Tasks;

public class SwiftStorageService
{
    private readonly ISwiftClient _swiftClient;
    private readonly IConfiguration _configuration;

    public SwiftStorageService(IConfiguration configuration)
    {
        _configuration = configuration;
        _swiftClient = new SwiftClient.Client()
            .WithCredentials(new SwiftCredentials
            {
                Username = _configuration["Swift:Username"],
                Password = _configuration["Swift:Password"],
                Endpoints = new List<string> { _configuration["Swift:Endpoint"] }
            })
            .SetRetryCount(6)
            .SetRetryPerEndpointCount(2);
    }

    public async Task<bool> UploadFileAsync(string containerName, string objectName, Stream fileStream)
    {
        try
        {
            var response = await _swiftClient.PutObject(containerName, objectName, fileStream);
            return response.IsSuccess;
        }
        catch (SwiftException ex)
        {
            throw new ApplicationException($"Swift storage error: {ex.Message}", ex);
        }
    }
}

Secure configuration management

Store your Swift credentials securely using configuration files, environment variables, or trusted secret managers. Create an appsettings.json:

{
  "Swift": {
    "Username": "your_username",
    "Password": "your_password",
    "Endpoint": "https://your-swift-endpoint:8080"
  }
}

For production, employ secure configuration providers:

public class Program
{
    public static void Main(string[] args)
    {
        var builder = WebApplication.CreateBuilder(args);

        builder.Configuration
            .SetBasePath(Directory.GetCurrentDirectory())
            .AddJsonFile("appsettings.json", optional: false)
            .AddEnvironmentVariables()
            .AddUserSecrets<Program>();

        var app = builder.Build();
        app.Run();
    }
}

Handling large files

For large file uploads, the SwiftClient library handles automatic chunking. Use the following method to upload large files:

public async Task<bool> UploadLargeFileAsync(string containerName, string objectName, Stream fileStream)
{
    try
    {
        var response = await _swiftClient.PutObject(containerName, objectName, fileStream);
        return response.IsSuccess;
    }
    catch (SwiftException ex)
    {
        if (ex.StatusCode == 507)
        {
            throw new ApplicationException("Storage quota exceeded", ex);
        }
        throw new ApplicationException($"Swift storage error: {ex.Message}", ex);
    }
}

Implementing robust error handling

Develop a comprehensive error handling and retry strategy for the Swift operations:

using System;
using System.Threading.Tasks;

public class SwiftStorageException : Exception
{
    public int StatusCode { get; }

    public SwiftStorageException(string message, int statusCode, Exception innerException = null)
        : base(message, innerException)
    {
        StatusCode = statusCode;
    }
}

public async Task<SwiftResponse> ExecuteWithRetryAsync(Func<Task<SwiftResponse>> operation)
{
    int retryCount = 0;
    const int maxRetries = 6;
    const int baseDelay = 1000; // delay in milliseconds

    while (true)
    {
        try
        {
            var response = await operation();
            if (!response.IsSuccess)
            {
                switch (response.StatusCode)
                {
                    case 401:
                        throw new SwiftStorageException("Authentication failed", 401);
                    case 403:
                        throw new SwiftStorageException("Access denied", 403);
                    case 404:
                        throw new SwiftStorageException("Resource not found", 404);
                    case 507:
                        throw new SwiftStorageException("Storage quota exceeded", 507);
                    default:
                        throw new SwiftStorageException("Unexpected error", response.StatusCode);
                }
            }

            return response;
        }
        catch (SwiftException ex) when (retryCount < maxRetries &&
            (ex.StatusCode == 429 || ex.StatusCode >= 500))
        {
            retryCount++;
            int delay = baseDelay * (int)Math.Pow(2, retryCount - 1);
            await Task.Delay(delay);
        }
        catch (Exception ex)
        {
            throw new ApplicationException("An unexpected error occurred during the operation.", ex);
        }
    }
}

Performance optimization

Enhance your workflow by optimizing performance with parallel uploads and connection pooling.

Parallel uploads

Implement parallel uploads to expedite file transfers:

using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

public async Task UploadFilesInParallelAsync(string containerName, IEnumerable<(string objectName, Stream fileStream)> files)
{
    var uploadTasks = files.Select(file =>
        UploadFileAsync(containerName, file.objectName, file.fileStream));

    await Task.WhenAll(uploadTasks);
}

Configure connection pooling

Reuse a single HttpClient instance to improve efficiency:

using System;
using System.Net.Http;

public class SwiftClientFactory
{
    private static readonly HttpClient HttpClient = new()
    {
        MaxConnectionsPerServer = 20,
        Timeout = TimeSpan.FromMinutes(5)
    };

    public static ISwiftClient CreateClient(SwiftCredentials credentials)
    {
        return new SwiftClient.Client()
            .WithCredentials(credentials)
            .WithHttpClient(HttpClient)
            .SetRetryCount(6);
    }
}

Practical example: complete implementation

Below is a complete example that demonstrates how to integrate the Swift storage service in a file upload workflow:

using System;
using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;

public class FileUploadService
{
    private readonly SwiftStorageService _storageService;
    private readonly ILogger<FileUploadService> _logger;

    public FileUploadService(SwiftStorageService storageService, ILogger<FileUploadService> logger)
    {
        _storageService = storageService;
        _logger = logger;
    }

    public async Task<bool> ProcessFileUploadAsync(IFormFile file, string containerName)
    {
        try
        {
            using var stream = file.OpenReadStream();
            var objectName = $"{DateTime.UtcNow:yyyy/MM/dd}/{Guid.NewGuid()}/{file.FileName}";
            var success = await _storageService.UploadFileAsync(containerName, objectName, stream);

            if (success)
            {
                _logger.LogInformation("File {FileName} uploaded successfully", file.FileName);
                return true;
            }

            _logger.LogError("Failed to upload file {FileName}", file.FileName);
            return false;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Error uploading file {FileName}", file.FileName);
            return false;
        }
    }
}

Conclusion

Integrating OpenStack Swift into your .NET applications brings robust cloud storage capabilities, automatic chunking for large files, secure authentication management, and sophisticated error handling—all of which lead to improved reliability and performance. By leveraging parallel uploads and connection pooling, you can further optimize your workflow.

For enhanced file upload capabilities, consider exploring solutions like Tus and Uppy for resumable uploads that work seamlessly with modern Swift storage systems.