Scan files for viruses in Go with ClamAV

Scanning files for viruses is crucial for maintaining the security and integrity of your applications, especially those handling user uploads. In this DevTip, we'll explore how you can integrate the open-source antivirus tool ClamAV with your Go applications to effectively detect and handle malicious files.

Introduction to virus scanning in Go

Virus scanning is essential for applications that handle user-uploaded files or process external data. Integrating antivirus scanning directly into your Go application helps prevent malware from spreading through your system, protecting both your users and your infrastructure.

Overview of ClamAV and its capabilities

ClamAV is a widely-used, open-source antivirus engine designed for detecting trojans, viruses, malware, and other malicious threats. Key features include:

• A versatile command-line scanner (clamscan).
• A multi-threaded daemon (clamd) for high-performance scanning.
• Regularly updated virus definition databases via freshclam.
• Libraries and APIs for integration into various applications.

Setting up ClamAV in your Go environment

First, you need to install ClamAV on your system. The process varies depending on your operating system. For Debian/Ubuntu systems:

# Update package lists
sudo apt-get update

# Install ClamAV and the daemon
sudo apt-get install clamav clamav-daemon

# Ensure the virus definitions are up-to-date
# Stop freshclam if it's running to avoid conflicts
sudo systemctl stop clamav-freshclam
# Manually update the definitions
sudo freshclam
# Restart freshclam to enable automatic updates
sudo systemctl start clamav-freshclam

# Start the ClamAV daemon
sudo systemctl start clamav-daemon

Verify that the clamd daemon is running:

sudo systemctl status clamav-daemon

You should see output indicating the service is active (running).

Implementing go-clamd for virus scanning

To interact with the ClamAV daemon from Go, we can use the go-clamd library. Install it using the standard go get command:

go get github.com/dutchcoders/go-clamd

Here's a basic example demonstrating how to connect to the clamd daemon (via its Unix socket, which is common) and scan a specific file:

package main

import (
	"fmt"
	"log"

	"github.com/dutchcoders/go-clamd"
)

func main() {
	// Connect to the ClamAV daemon using its Unix socket
	// Ensure your Go application has permissions to access this socket
	c := clamd.NewClamd("unix:/var/run/clamav/clamd.ctl")

	// Alternatively, connect via TCP if clamd is configured for it
	// c := clamd.NewClamd("tcp://127.0.0.1:3310")

	// Path to the file you want to scan
	filePath := "/path/to/your/file/to/scan.txt"

	// Initiate the scan
	response, err := c.ScanFile(filePath)
	if err != nil {
		log.Fatalf("Failed to scan file %s: %v", filePath, err)
	}

	// The ScanFile function returns a channel. Iterate over it.
	// For a single file scan, there will typically be one result.
	for result := range response {
		fmt.Printf("File: %s, Status: %s\n", result.Filename, result.Status)
		if result.Status == clamd.RES_FOUND {
			fmt.Printf("Virus description: %s\n", result.Description)
		} else if result.Status == clamd.RES_ERROR {
			fmt.Printf("Error scanning file: %s\n", result.Description)
		}
	}
}

Practical examples and code snippets

Let's create a reusable function to scan a file and return an error if a virus is found or if scanning fails.

package main

import (
	"context"
	"fmt"
	"log"
	"os"
	"time"

	"github.com/dutchcoders/go-clamd"
)

// scanAndHandle checks a file and returns an error if a virus is detected.
func scanAndHandle(filePath string) error {
	c := clamd.NewClamd("unix:/var/run/clamav/clamd.ctl")
	response, err := c.ScanFile(filePath)
	if err != nil {
		return fmt.Errorf("clamd scan initiation error for %s: %w", filePath, err)
	}

	for result := range response {
		switch result.Status {
		case clamd.RES_FOUND:
			// Virus detected!
			return fmt.Errorf("virus detected in %s: %s", result.Filename, result.Description)
		case clamd.RES_ERROR:
			// An error occurred during scanning
			return fmt.Errorf("clamd scan error for %s: %s", result.Filename, result.Description)
		case clamd.RES_OK:
			// File is clean
			log.Printf("File %s is clean.", result.Filename)
		default:
			// Unexpected status
			return fmt.Errorf("unexpected scan status for %s: %s", result.Filename, result.Status)
		}
	}

	// If the loop finishes without returning, the file is clean.
	return nil
}

// scanWithTimeout wraps the scan operation with a timeout.
func scanWithTimeout(filePath string, timeout time.Duration) error {
	c := clamd.NewClamd("unix:/var/run/clamav/clamd.ctl")

	ctx, cancel := context.WithTimeout(context.Background(), timeout)
	defer cancel()

	resultCh := make(chan *clamd.ScanResult, 1)
	errorCh := make(chan error, 1)

	go func() {
		response, err := c.ScanFile(filePath)
		if err != nil {
			errorCh <- fmt.Errorf("clamd scan initiation error: %w", err)
			return
		}
		// Assuming one result for ScanFile
		result := <-response
		if result == nil {
			errorCh <- fmt.Errorf("received nil result from scan channel")
			return
		}
		resultCh <- result
	}()

	select {
	case result := <-resultCh:
		switch result.Status {
		case clamd.RES_FOUND:
			return fmt.Errorf("virus detected in %s: %s", result.Filename, result.Description)
		case clamd.RES_ERROR:
			return fmt.Errorf("clamd scan error for %s: %s", result.Filename, result.Description)
		case clamd.RES_OK:
			log.Printf("File %s is clean (within timeout).", result.Filename)
			return nil // Clean
		default:
			return fmt.Errorf("unexpected scan status for %s: %s", result.Filename, result.Status)
		}
	case err := <-errorCh:
		return err // Error during scan initiation or processing
	case <-ctx.Done():
		return fmt.Errorf("scan timeout for %s after %v", filePath, timeout)
	}
}

// Example usage (replace with actual file paths and error handling)
func main() {
	testFilePath := "/tmp/testfile.txt" // Create a dummy file for testing
	_ = os.WriteFile(testFilePath, []byte("This is a test file."), 0644)
	defer os.Remove(testFilePath)

	err := scanAndHandle(testFilePath)
	if err != nil {
		log.Printf("Scan result: %v", err)
	}

	err = scanWithTimeout(testFilePath, 5*time.Second)
	if err != nil {
		log.Printf("Scan with timeout result: %v", err)
	}
}

Configuring ClamAV for production use

For production environments, tuning ClamAV's configuration is important for performance and reliability. Key configuration files are typically /etc/clamav/clamd.conf and /etc/clamav/freshclam.conf.

1. Edit clamd.conf:

   • MaxFileSize: Set a reasonable limit based on expected uploads to prevent excessive memory usage (e.g., MaxFileSize 100M).
   • MaxScanSize: Similar to MaxFileSize, limits the amount of data scanned per file.
   • MaxThreads: Adjust the number of scanning threads based on your server's CPU cores (e.g., MaxThreads 10).
   • StreamMaxLength: Limits the size for streamed data scans.
   • TCPSocket 3310 / TCPAddr 127.0.0.1: Uncomment and configure if you prefer TCP connections over Unix sockets.
   • LocalSocket /var/run/clamav/clamd.ctl: Ensure this matches the path used in your Go code if using Unix sockets. Check permissions.
   • User clamav: Ensure the specified user exists and has necessary permissions.

2. Edit freshclam.conf:

   • Checks 24: Configure how many times per day freshclam checks for updates (e.g., Checks 12 for every 2 hours).

3. Set up Logging in clamd.conf:

   # Path to the log file
   LogFile /var/log/clamav/clamd.log
   # Enable time stamps in log entries
   LogTime yes
   # Enable verbose logging (useful for debugging)
   # LogVerbose yes
   # Log clean files (can be noisy)
   # LogClean yes
   

Remember to restart the clamav-daemon service after changing configuration files: sudo systemctl restart clamav-daemon.

Testing with eicar

The EICAR test file is a non-malicious string standardized by the European Institute for Computer Antivirus Research to test antivirus configurations safely.

package main

import (
	"fmt"
	"log"
	"os"

	"github.com/dutchcoders/go-clamd" // Assuming scanAndHandle is in the same package
)

// TestClamAVWithEICAR verifies the ClamAV setup using the EICAR test string.
func TestClamAVWithEICAR() {
	// The standard EICAR test string
	eicar := []byte(`X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*`)

	// Create a temporary file
	tmpFile, err := os.CreateTemp("", "eicar-test-*.txt")
	if err != nil {
		log.Fatalf("Failed to create temp file: %v", err)
	}
	// Ensure the temporary file is removed afterwards
	defer os.Remove(tmpFile.Name())
	filePath := tmpFile.Name()

	// Write the EICAR string to the temporary file
	if _, err := tmpFile.Write(eicar); err != nil {
		tmpFile.Close() // Close file before logging fatal error
		log.Fatalf("Failed to write EICAR content to %s: %v", filePath, err)
	}
	// Close the file so clamd can read it
	if err := tmpFile.Close(); err != nil {
		log.Fatalf("Failed to close temp file %s: %v", filePath, err)
	}

	log.Printf("Scanning EICAR test file: %s", filePath)

	// Scan the file using our previously defined function
	err = scanAndHandle(filePath) // Assumes scanAndHandle is accessible

	// Check the result
	if err != nil {
		// We expect an error indicating the virus was found
		fmt.Println("ClamAV successfully detected the EICAR test virus:", err)
	} else {
		// If err is nil, the test file was NOT detected
		log.Fatal("ClamAV setup FAILED: EICAR test file was not detected.")
	}
}

// You would call TestClamAVWithEICAR() from your main or test function
// func main() {
//     TestClamAVWithEICAR()
// }

Running this should produce output indicating that the EICAR test virus was found. If not, review your ClamAV installation, daemon status, and configuration.

Best practices for virus scanning in Go applications

• Scan Immediately: Scan files as soon as they are received (e.g., after upload, before storing permanently).
• Use Unix Sockets: Prefer Unix sockets for local clamd communication for better performance and security compared to TCP loopback, but ensure correct permissions.
• Configure Permissions: Ensure the user running your Go application has read/write permissions on the ClamAV daemon's socket (/var/run/clamav/clamd.ctl by default).
• Implement Timeouts: Use context timeouts (as shown in scanWithTimeout) to prevent scans from blocking indefinitely, especially with large files or a busy daemon.
• Use Streaming API: For very large files, consider using clamd.ScanStream if memory is a concern, although ScanFile is often sufficient as clamd handles file reading efficiently.
• Monitor Daemon Status: Before attempting scans, check if the clamd service is running. You could implement a health check endpoint in your app.
• Handle Errors Gracefully: Distinguish between "file is clean," "virus found," and "scan error" results. Log errors appropriately. Decide on application behavior for scan errors (e.g., reject file, quarantine, retry).
• Resource Limits: Configure clamd resource limits (MaxFileSize, MaxScanSize, MaxThreads) appropriately for your server capacity and expected workload.
• Asynchronous Scanning: For web applications, perform scans asynchronously in background workers to avoid blocking user requests.
• Keep Definitions Updated: Ensure freshclam runs regularly to keep virus definitions current. Monitor its logs (/var/log/clamav/freshclam.log).
• Monitor ClamAV Resources: Keep an eye on CPU and memory usage of the clamd process, especially under load.

Troubleshooting common issues

• Connection Errors (Socket/TCP):
  • Verify clamd is running (systemctl status clamav-daemon).
  • Check if the socket path (/var/run/clamav/clamd.ctl) or TCP address/port (127.0.0.1:3310) in your Go code matches the clamd.conf configuration.
  • Ensure your Go application has the necessary permissions to access the Unix socket (check group membership or ACLs).
• Scan Timeouts:
  • Increase the timeout value in your Go code (scanWithTimeout example).
  • Check clamd logs (/var/log/clamav/clamd.log) for errors or performance issues.
  • Increase clamd resources (MaxThreads) if the server is under heavy load.
  • Consider if the file size exceeds MaxFileSize or MaxScanSize configured in clamd.conf.
• Permission Denied Errors:
  • Check file permissions of the file being scanned. The clamav user (or the user clamd runs as) needs read access.
  • Check permissions on the ClamAV socket file if using Unix sockets.
• High Resource Usage:
  • clamd can be CPU and memory intensive during scans. Monitor usage with tools like top or htop.
  • Adjust MaxThreads in clamd.conf based on available CPU cores.
  • Ensure sufficient RAM is available.
• EICAR Not Detected:
  • Verify clamd is running and using up-to-date virus definitions (sudo freshclam).
  • Check clamd logs for errors during the scan attempt.
  • Ensure the EICAR string was written correctly to the test file.

Conclusion and additional resources

Integrating ClamAV with Go using the go-clamd library provides a robust way to add virus scanning capabilities to your applications, significantly enhancing security. Remember to configure ClamAV properly for your environment and follow best practices for reliable scanning.

For further details, consult the official documentation:

• ClamAV Official Documentation
• go-clamd GitHub Repository
• go-clamd Go Package Documentation

At Transloadit, we use ClamAV in our 🤖 /file/virusscan robot, which is part of our file filtering service. the robot is built on top of ClamAV with daily Signature updates, and can be configured to either exclude malicious files silently or stop processing with an error when malicious content is detected.