I need to write an application with Node JS which given a link to a json file e.g .json (The link doesn't open the file it opens a download), the program simply downloads the object and then prints it out. How can this be achieved? This is what I have so far and it doesn't seem to be working:

var checkIfPhishing = function(urlToPrint){
    var http = require('http');
    var fs = require('fs');

    var file = fs.createWriteStream("SiteObject.json");
    var request = http.get(urlToPrint, function(response) {
    response.pipe(file);});

    var siteObj= fs.readFileSync("SiteObject.json");

    console.log(siteObj);

};

Thank you!

I need to write an application with Node JS which given a link to a json file e.g http://data.phishtank./data/online-valid.json (The link doesn't open the file it opens a download), the program simply downloads the object and then prints it out. How can this be achieved? This is what I have so far and it doesn't seem to be working:

var checkIfPhishing = function(urlToPrint){
    var http = require('http');
    var fs = require('fs');

    var file = fs.createWriteStream("SiteObject.json");
    var request = http.get(urlToPrint, function(response) {
    response.pipe(file);});

    var siteObj= fs.readFileSync("SiteObject.json");

    console.log(siteObj);

};

Thank you!

• javascript
• node.js

• 1 At first put the requires out of the function. That will double your performance... – Jonas Wilms Commented May 23, 2017 at 19:24

1 Answer 1

You cannot mix up async and sync reads and writes. In your case you start Streaming the data from the other server to yours, but then you already start the sync read. Which blocks the thread so the stream will be processed after you've read your 0 byte file...

So you need to store the stream in a variable first, then on finish log that stream.

So, simply, do something like this:

let data = "";

const request = http.get( urlToPrint, function(response) {

  response
  .on("data", append => data += append )
  .on("error", e => console.log(e) )
  .on("end", ()=> console.log(data) );

});

Store the asynchronously provided chunks of the stream in a variable, if the stream finishes, log that string.

If you want to store it too:

const http = require('http');
const fs = require('fs');

function checkIfPhishing(urlToPrint){

    const file = fs.createWriteStream("SiteObject.json");
    const request = http.get(urlToPrint, function(response) {
      response.on("finish",function(){
        console.log( fs.readFileSync("SiteObject.json",{encoding:"utf8"}));
      }).pipe(file);
    });
}

This is exactly like your code, but it waits for the stream to finish before it reads synchronously...

However, note that the sync read will slow down the whole thing, so you might directly stream to the console/a browser..