In this post I will be discussing how to stream CSV to JSON via FTP , which will enable you to integrate your application with any number of legacy applications to extract data.
It is a common requirement for applications to be able to download and parse a
Comma Separated Value (CSV) file from a
FTP location. Usually this is to enable some kind of data interaction between systems.
CSV is a common, relatively simple file format that is widely supported by consumer, business, and scientific applications. Among its most common uses is moving tabular data between programs that natively operate on incompatible (often proprietary and/or undocumented) formats. This works because so many programs support some variation of CSV at least as an alternative import/export format.
We’ll be making use of two Node modules to implement this functionality:
The two modules we’re going to use are available through the Node Package Manager (npm) so installing them is pretty easy using the command line
The following module is my straw man proposal , to establish whether I could achieve this functionality quite simply in node.js. The aim of the application is quite simply access an FTP folder, stream a CSV file on the server, and transform that file into JSON objects so eventually I could import them into a database.
For the purpose of this module, we’ll only be printing the
JSON objects out the console for review.
Despite the fact that Douglas Crockford, author of
This is another fine example of, just how extensible node.js is , and how by making use of some pre-existing libraries we are able to quickly and efficiently knock up solutions to common problems.