Is there a node / npm module which would scrape the contents of a web page or a given URL and execute and evaluate it's JavaScript as a real-life web browser, including XMLHttpRequest object and alike?
I thought that scraperjs was the solution, but sadly it isn't.
All help very appreciated!
Is there a node / npm module which would scrape the contents of a web page or a given URL and execute and evaluate it's JavaScript as a real-life web browser, including XMLHttpRequest object and alike?
I thought that scraperjs was the solution, but sadly it isn't.
All help very appreciated!
Share Improve this question edited Feb 13, 2021 at 14:14 DisappointedByUnaccountableMod 6,8264 gold badges20 silver badges23 bronze badges asked Oct 11, 2016 at 7:05 Zlatan OmerovicZlatan Omerovic 4,0974 gold badges46 silver badges70 bronze badges 2- 3 PhantomJS? I think the term you may be looking for is a headless browser, of which Phantom JS is one. – James Hay Commented Oct 11, 2016 at 7:06
- That's the one! Thank you <3 – Zlatan Omerovic Commented Oct 11, 2016 at 7:07
1 Answer
Reset to default 7Headless browsers based on rendering engine:
- Chrome (blink) => Playwright or Puppeteer
- Firefox (gecko) => Playwright
- Safari (webkit) => Playwright
Puppeteer is the official automation tool for Chrome. It only supports Chrome/Chromium browsers, (with experimental support for Firefox).
Playwright is an alternative to Puppeteer, with support for more browsers, written by the original creators of puppeteer, who jumped ship from Google to Microsoft. It supports Chrome, Firefox and Safari browsers
Previous versions of my answer, included PhantomJS, SlimerJS,CasperJS and Nightmare all of which are no longer maintaned. For modern browsers, just use one of the above