Uploading and processing CSV files via a REST API is a common requirement in modern web applications. In this guide, we will show you how to build a secure REST API using Node.js and Express that handles CSV file uploads, parses file content using csv-parser, and stores the data for later retrieval. We use Multer to manage file uploads with robust security checks.

Prerequisites

Before you begin, ensure you have the following installed:

  • Node.js (version 22.x LTS or higher)
  • npm (Node Package Manager)

Setting up a Node.js and Express Server

  1. Create a new directory for your project and initialize it:

    mkdir csv-upload-api
cd csv-upload-api
npm init -y

  2. Install the necessary dependencies with explicit version numbers to ensure compatibility:

    npm install express@4.18.2 multer@1.4.5-lts.1 csv-parser@3.0.0

  3. Create a file named server.js and add the following code to set up a basic Express server:

    const express = require('express')
const app = express()
const port = process.env.PORT || 3000

app.use(express.json())

app.listen(port, () => {
  console.log(`Server is running on port ${port}`)
})

Implementing the file upload endpoint

To securely handle CSV file uploads, configure Multer with file size limits and file type verification:

const multer = require('multer')
const path = require('path')
const fs = require('fs')
const csv = require('csv-parser')

const upload = multer({
  dest: 'uploads/',
  limits: { fileSize: 1024 * 1024 * 5 }, // 5MB limit
  fileFilter: (req, file, cb) => {
    if (file.mimetype !== 'text/csv' || path.extname(file.originalname).toLowerCase() !== '.csv') {
      return cb(new Error('Only CSV files are allowed'), false)
    }
    cb(null, true)
  },
})

// Multer error handling middleware
app.use((error, req, res, next) => {
  if (error instanceof multer.MulterError) {
    return res.status(400).send(error.message)
  }
  next(error)
})

Parsing and storing CSV data

In this section, we build the /upload endpoint to parse the CSV file and store its contents. A global variable holds the parsed data, and a GET endpoint is provided to retrieve it.

let storedData = []

app.post('/upload', upload.single('file'), (req, res) => {
  if (!req.file) {
    return res.status(400).send('No file uploaded.')
  }

  const results = []
  const stream = fs
    .createReadStream(req.file.path)
    .pipe(csv())
    .on('data', (data) => {
      results.push(data)
    })
    .on('end', () => {
      storedData = results
      fs.unlink(req.file.path, (err) => {
        if (err) {
          console.error('Error deleting file:', err)
          return res.status(500).send('Error cleaning up uploaded file.')
        }
        res.send(`File uploaded and processed. ${results.length} records stored.`)
      })
    })
    .on('error', (err) => {
      console.error('Error processing file:', err)
      fs.unlink(req.file.path, () => {
        res.status(500).send('Error processing file.')
      })
    })

  // Handle any additional stream errors
  stream.on('error', (err) => {
    console.error('Stream error:', err)
    fs.unlink(req.file.path, () => {
      res.status(500).send('Error reading file stream.')
    })
  })
})

app.get('/data', (req, res) => {
  res.json(storedData)
})

Testing the API

You can test the API using the following cURL commands or Postman.

Test a successful upload:

curl -X POST -F "file=@data.csv" http://localhost:3000/upload

Test with an invalid file type:

curl -X POST -F "file=@image.jpg" http://localhost:3000/upload

Test exceeding the file size limit:

curl -X POST -F "file=@large.csv" http://localhost:3000/upload

Retrieve the stored data:

curl http://localhost:3000/data

Alternatively, use Postman by setting the request method to POST, the URL to http://localhost:3000/upload, and selecting the "form-data" option in the Body tab. Add a key named file with type File, choose your CSV file, and send your request.

Production considerations

Before deploying, consider these important best practices:

  • Use environment variables for configuration (such as port numbers and upload limits).
  • Implement proper logging using a production-grade logger.
  • Add rate limiting to upload endpoints to prevent abuse.
  • Consider using cloud storage services (e.g., AWS S3 or Google Cloud Storage) instead of the local filesystem.
  • Implement robust authentication and authorization for enhanced security.
  • Use HTTPS to secure all communications in production.
  • Monitor server resources and plan for appropriate scaling strategies.

Conclusion

This guide demonstrated how to build a secure REST API for uploading and processing CSV files with Node.js and Express. By combining Multer for secure file handling with csv-parser for data extraction, the API efficiently processes CSV uploads while incorporating robust error management. For additional scalability solutions, consider exploring the file handling services available from Transloadit.