Processing file uploads in node.js


One of the important elements of a web application is form processing and file uploads. PHP has excellent support for these kind of things which makes it so popular with developers.
Programmers moving to node.js find that it does not have an easy way to process uploaded files. However some excellent node modules can make it extremely easy to handle file uploads in node.js.

In this post we will look into one such library – Formidable.

Installing required node modules

Formidable is the main module that will help us in building a file upload service easily. Install it using npm.

npm install formidable@latest

We will also use another module for file copying purposes – fs-extra. This module adds a few extra file system methods that aren’t included in the native node fs module.

npm install fs-extra

File upload processing

First we will build a simple node.js server that displays a file upload form, sans any css. Couldn’t be any simpler than this. We will add additional features to this code gradually to make it easier to understand how everything works.

var formidable = require('formidable'),
    http = require('http'),
    util = require('util'),
    fs   = require('fs-extra');
 
http.createServer(function(req, res) {
 
  /* Display the file upload form. */
  res.writeHead(200, {'content-type': 'text/html'});
  res.end(
    '<form action="/upload" enctype="multipart/form-data" method="post">'+
    '<input type="text" name="title"><br>'+
    '<input type="file" name="upload" multiple="multiple"><br>'+
    '<input type="submit" value="Upload">'+
    '</form>'
  );
 
}).listen(8080);

This just displays the HTML form without any processing after form submission. This includes a single text field and a file input field.

Next we will add form processing code using ‘Formidable’ to handle the uploads.

var formidable = require('formidable'),
    http = require('http'),
    util = require('util'),
    fs   = require('fs-extra');
 
http.createServer(function(req, res) {
  /* Process the form uploads */
  if (req.url == '/upload' && req.method.toLowerCase() == 'post') {
    var form = new formidable.IncomingForm();
    form.parse(req, function(err, fields, files) {
      res.writeHead(200, {'content-type': 'text/plain'});
      res.write('received upload:\n\n');
      res.end(util.inspect({fields: fields, files: files}));
    });
 
    return;
  }
 
  /* Display the file upload form. */
  res.writeHead(200, {'content-type': 'text/html'});
  res.end(
    '<form action="/upload" enctype="multipart/form-data" method="post">'+
    '<input type="text" name="title"><br>'+
    '<input type="file" name="upload" multiple="multiple"><br>'+
    '<input type="submit" value="Upload">'+
    '</form>'
  );
 
}).listen(8080);

Here we check if the requesting url is ‘/upload’ and the method is ‘post’ and then use the parse method to process the incoming node request. Here we just output the processed form details using the util.inspect method to the browser. Running the above on my machine displays the following in the browser.

received upload:
 
{ fields: { title: 'test' },
  files: 
   { upload: 
   { size: 15505,
    path:'C:\\DOCUME~1\\A\\LOCALS~1\\Temp\\56ddb418815b783103fb7efe703d3eba',
    name:'1157743_10151652039852585_141602505_n.jpg',
    type:'image/jpeg',
    hash:null,
    lastModifiedDate:Fri Aug 23 2013 14:57:12 GMT+0530 (India Standard Time),
    _writeStream: [Object] } } }

As you can see the file has been uploaded to a temporary directory in my ‘document’ folder with the name ’56ddb418815b783103fb7efe703d3eba’. Now all I’ve to do is copy the same to the desired location. For this we will use the node ‘fs-extra’ module, which makes it quite easy to copy files in node.js.

var formidable = require('formidable'),
    http = require('http'),
    util = require('util'),
    fs   = require('fs-extra');
 
http.createServer(function(req, res) {
  /* Process the form uploads */
  if (req.url == '/upload' && req.method.toLowerCase() == 'post') {
    var form = new formidable.IncomingForm();
    form.parse(req, function(err, fields, files) {
      res.writeHead(200, {'content-type': 'text/plain'});
      res.write('received upload:\n\n');
      res.end(util.inspect({fields: fields, files: files}));
    });
 
    form.on('end', function(fields, files) {
        /* Temporary location of our uploaded file */
        var temp_path = this.openedFiles[0].path;
        /* The file name of the uploaded file */
        var file_name = this.openedFiles[0].name;
        /* Location where we want to copy the uploaded file */
        var new_location = 'c:/localhost/nodejs/';
 
        fs.copy(temp_path, new_location + file_name, function(err) {  
            if (err) {
                console.error(err);
            } else {
                console.log("success!")
            }
        });
    });
 
    return;
  }
 
  /* Display the file upload form. */
  res.writeHead(200, {'content-type': 'text/html'});
  res.end(
    '<form action="/upload" enctype="multipart/form-data" method="post">'+
    '<input type="text" name="title"><br>'+
    '<input type="file" name="upload" multiple="multiple"><br>'+
    '<input type="submit" value="Upload">'+
    '</form>'
  );
 
}).listen(8080);

Here we use the end event of the Formidable module to copy the file to a new location. The end event is emitted when the entire request has been received, and all contained files have finished flushing to disk.

You can check for upload errors using the error event. The event is emitted when there is an error processing the incoming form. A request that experiences an error is automatically paused, you will have to manually call request.resume() if you want the request to continue firing ‘data’ events.

form.on('error', function(err) {
        console.error(err);
    });

Checking upload progress

For large file uploads it would be better if we provide some progress indicator for the client side. Formidable makes it easier to gather that information. This can be accomplished using the progress event, emitted after each incoming chunk of data that has been parsed. Here we will just log the progress to the console.

form.on('progress', function(bytesReceived, bytesExpected) {
        var percent_complete = (bytesReceived / bytesExpected) * 100;
        console.log(percent_complete.toFixed(2));
    });

This is all there is to uploading files in node.js using Formidable. The complete code is given below.

var formidable = require('formidable'),
    http = require('http'),
    util = require('util'),
    fs   = require('fs-extra');
 
http.createServer(function(req, res) {
  /* Process the form uploads */
  if (req.url == '/upload' && req.method.toLowerCase() == 'post') {
    var form = new formidable.IncomingForm();
    form.parse(req, function(err, fields, files) {
      res.writeHead(200, {'content-type': 'text/plain'});
      res.write('received upload:\n\n');
      res.end(util.inspect({fields: fields, files: files}));
    });
 
    form.on('progress', function(bytesReceived, bytesExpected) {
        var percent_complete = (bytesReceived / bytesExpected) * 100;
        console.log(percent_complete.toFixed(2));
    });
 
    form.on('error', function(err) {
        console.error(err);
    });
 
    form.on('end', function(fields, files) {
        /* Temporary location of our uploaded file */
        var temp_path = this.openedFiles[0].path;
        /* The file name of the uploaded file */
        var file_name = this.openedFiles[0].name;
        /* Location where we want to copy the uploaded file */
        var new_location = 'c:/localhost/nodejs/';
 
        fs.copy(temp_path, new_location + file_name, function(err) {  
            if (err) {
                console.error(err);
            } else {
                console.log("success!")
            }
        });
    });
 
    return;
  }
 
  /* Display the file upload form. */
  res.writeHead(200, {'content-type': 'text/html'});
  res.end(
    '<form action="/upload" enctype="multipart/form-data" method="post">'+
    '<input type="text" name="title"><br>'+
    '<input type="file" name="upload" multiple="multiple"><br>'+
    '<input type="submit" value="Upload">'+
    '</form>'
  );
 
}).listen(8080);

Changing default options

There are some upload options you can modify according to your project requirements. For example you can change the default location to where Formidable saves the temporary file. This can be accomplished in the fileBegin event, emitted whenever a field / value pair has been received.

form.on('fileBegin', function(name, file) {
        file.path = 'c:/localhost/nodejs/uploads/' + file.name;
    });

If you are using the above method, you can do away with the additional copying process and directly upload the files to the desired location. However it is recommended to set a different temporary location other the uploads folder.

This site is a digital habitat of Sameer Borate, a freelance web developer working in PHP, MySQL and WordPress. I also provide web scraping services, website design and development and integration of various Open Source API's. Contact me at metapix[at]gmail.com for any new project requirements and price quotes.

Your thoughts

Sign up for fresh content in your email