Uploading and processing CSV files via a REST API

Uploading and processing CSV files via a REST API is a common requirement in modern web applications. In this guide, we will show you how to build a secure REST API using Node.js and Express that handles CSV file uploads, parses file content using csv-parser, and stores the data for later retrieval. We use Multer to manage file uploads with robust security checks.
Prerequisites
Before you begin, ensure you have the following installed:
- Node.js (version 22.x LTS or higher)
- npm (Node Package Manager)
Setting up a Node.js and Express Server
-
Create a new directory for your project and initialize it:
mkdir csv-upload-api cd csv-upload-api npm init -y
-
Install the necessary dependencies with explicit version numbers to ensure compatibility:
npm install express@4.18.2 multer@1.4.5-lts.1 csv-parser@3.0.0
-
Create a file named
server.js
and add the following code to set up a basic Express server:const express = require('express') const app = express() const port = process.env.PORT || 3000 app.use(express.json()) app.listen(port, () => { console.log(`Server is running on port ${port}`) })
Implementing the file upload endpoint
To securely handle CSV file uploads, configure Multer with file size limits and file type verification:
const multer = require('multer')
const path = require('path')
const fs = require('fs')
const csv = require('csv-parser')
const upload = multer({
dest: 'uploads/',
limits: { fileSize: 1024 * 1024 * 5 }, // 5MB limit
fileFilter: (req, file, cb) => {
if (file.mimetype !== 'text/csv' || path.extname(file.originalname).toLowerCase() !== '.csv') {
return cb(new Error('Only CSV files are allowed'), false)
}
cb(null, true)
},
})
// Multer error handling middleware
app.use((error, req, res, next) => {
if (error instanceof multer.MulterError) {
return res.status(400).send(error.message)
}
next(error)
})
Parsing and storing CSV data
In this section, we build the /upload
endpoint to parse the CSV file and store its contents. A
global variable holds the parsed data, and a GET endpoint is provided to retrieve it.
let storedData = []
app.post('/upload', upload.single('file'), (req, res) => {
if (!req.file) {
return res.status(400).send('No file uploaded.')
}
const results = []
const stream = fs
.createReadStream(req.file.path)
.pipe(csv())
.on('data', (data) => {
results.push(data)
})
.on('end', () => {
storedData = results
fs.unlink(req.file.path, (err) => {
if (err) {
console.error('Error deleting file:', err)
return res.status(500).send('Error cleaning up uploaded file.')
}
res.send(`File uploaded and processed. ${results.length} records stored.`)
})
})
.on('error', (err) => {
console.error('Error processing file:', err)
fs.unlink(req.file.path, () => {
res.status(500).send('Error processing file.')
})
})
// Handle any additional stream errors
stream.on('error', (err) => {
console.error('Stream error:', err)
fs.unlink(req.file.path, () => {
res.status(500).send('Error reading file stream.')
})
})
})
app.get('/data', (req, res) => {
res.json(storedData)
})
Testing the API
You can test the API using the following cURL commands or Postman.
Test a successful upload:
curl -X POST -F "file=@data.csv" http://localhost:3000/upload
Test with an invalid file type:
curl -X POST -F "file=@image.jpg" http://localhost:3000/upload
Test exceeding the file size limit:
curl -X POST -F "file=@large.csv" http://localhost:3000/upload
Retrieve the stored data:
curl http://localhost:3000/data
Alternatively, use Postman by setting the request method to POST, the URL to
http://localhost:3000/upload
, and selecting the "form-data" option in the Body tab. Add a key
named file
with type File, choose your CSV file, and send your request.
Production considerations
Before deploying, consider these important best practices:
- Use environment variables for configuration (such as port numbers and upload limits).
- Implement proper logging using a production-grade logger.
- Add rate limiting to upload endpoints to prevent abuse.
- Consider using cloud storage services (e.g., AWS S3 or Google Cloud Storage) instead of the local filesystem.
- Implement robust authentication and authorization for enhanced security.
- Use HTTPS to secure all communications in production.
- Monitor server resources and plan for appropriate scaling strategies.
Conclusion
This guide demonstrated how to build a secure REST API for uploading and processing CSV files with Node.js and Express. By combining Multer for secure file handling with csv-parser for data extraction, the API efficiently processes CSV uploads while incorporating robust error management. For additional scalability solutions, consider exploring the file handling services available from Transloadit.