I'm trying to scrap a webpage using node js.I think I've written the code and was able to run it without any errors but the problem is the console doesn't print anything no matter what I do.It is not showing any errors. What's the reason?

Here is the content that I want to scrap:

var fs         = require('fs');
var request    = require('request');
var cheerio    = require('cheerio');

var htmlString = fs.readFileSync('scrap.html').toString();
var $          = cheerio.load(htmlString);
var json = [];

$('body > table:nth-child(1) > tbody > tr:nth-child(3) > td > table > tbody > tr > td:nth-child(2) > table > tbody > tr > td > table > tbody > tr').each(function(i, element) {
    json.push({});
    json[i].range = $(this).text().trim();

});


console.log(json);

I'm trying to scrap a webpage using node js.I think I've written the code and was able to run it without any errors but the problem is the console doesn't print anything no matter what I do.It is not showing any errors. What's the reason?

Here is the content that I want to scrap: https://paste.ee/r/b3yrn

var fs         = require('fs');
var request    = require('request');
var cheerio    = require('cheerio');

var htmlString = fs.readFileSync('scrap.html').toString();
var $          = cheerio.load(htmlString);
var json = [];

$('body > table:nth-child(1) > tbody > tr:nth-child(3) > td > table > tbody > tr > td:nth-child(2) > table > tbody > tr > td > table > tbody > tr').each(function(i, element) {
    json.push({});
    json[i].range = $(this).text().trim();

});


console.log(json);

• javascript
• node.js
• web-scraping
• cheerio

• 4 Probably json is empty. You should check if your selector finds anything. Or try to do a console.log(json.length); to see if the array is empty or not. – t.niese Commented Oct 18, 2015 at 9:49
• 1 If the selector won't find anything then a console.log inside of the each won't be called because there is no element. So the first test would be to do a console.log(json.length); instead of a console.log(json); – t.niese Commented Oct 18, 2015 at 9:57
• 1 Is is great that it does work in the browser, but the browser is not node.js and not cheerio. So why don't you just replace the console.log(json); with console.log(json.length); and tell us if you see a 0 or another number in the console, so that we can narrow down the problem? – t.niese Commented Oct 18, 2015 at 10:03
• 1 In the source there a two body elements, and I guess cheerio won't correct that error and while browser does. But then my assumption that your selector won't find anything (in cheerio) is still the same. – t.niese Commented Oct 18, 2015 at 10:09
• 3 I asked you to do one simple thing to narrow down the problem and you are not able to do that, so it is not possible to help you. And because of that I already wasted to much time with this question, that I could have used with other questions to help. – t.niese Commented Oct 18, 2015 at 10:40

1 Answer 1

To be sure that your console.log work correctly just try it like that :

console.log('starting');//<--------------------------------------- added line

var fs         = require('fs');
var request    = require('request');
var cheerio    = require('cheerio');

var htmlString = fs.readFileSync('scrap.html').toString();
var $          = cheerio.load(htmlString);
var json = [];

console.log('htmlString : ' + htmlString );//<-------------------- added line


$('body > table:nth-child(1) > tbody > tr:nth-child(3) > td > table > tbody > tr > td:nth-child(2) > table > tbody > tr > td > table > tbody > tr').each(function(i, element) {
    json.push({});
    json[i].range = $(this).text().trim();

});

console.log('Elements in json : ' + json.length);//<-------------- added line
console.log(json);

If this don't produce anything on the server-side, so yes we can confirm that your console.log don't work as expected, else it works and the problem e from other things !

Hope this will help you.