I want to disable JavaScript while scraping using scrapy and selenium. Moto of doing that is to increase scraping speed. I found the preference for Firefox driver but not PhantomJS.
firefox_profile = webdriver.FirefoxProfile()
firefox_profile.set_preference("javascript.enabled", False)
driver = webdriver.Firefox(firefox_profile=firefox_profile)
driver.get('/')
How can this be done for PhantomJS webdriver?
I want to disable JavaScript while scraping using scrapy and selenium. Moto of doing that is to increase scraping speed. I found the preference for Firefox driver but not PhantomJS.
firefox_profile = webdriver.FirefoxProfile()
firefox_profile.set_preference("javascript.enabled", False)
driver = webdriver.Firefox(firefox_profile=firefox_profile)
driver.get('http://www.quora./')
How can this be done for PhantomJS webdriver?
Share Improve this question edited Aug 20, 2015 at 10:31 Artjom B. 62k26 gold badges135 silver badges230 bronze badges asked Aug 20, 2015 at 10:23 amanaman 1,9954 gold badges20 silver badges29 bronze badges 02 Answers
Reset to default 8The WebDriver protocol in PhantomJS is a pure JavaScript implementation that is known as Ghostdriver. It makes heavy use of page.evaluate() to access the DOM and there is really no other way to access the DOM, interact with the page or do anything meaningful with PhantomJS. You shouldn't do this.
If you still want to go through with it, this should work:
cap = webdriver.DesiredCapabilities.PHANTOMJS
cap["phantomjs.page.settings.javascriptEnabled"] = False
driver = webdriver.PhantomJS(desired_capabilities=cap)
If the site does not require JavaScript, just use scrapy alone. There is no need for selenium. Scrapy is extremely fast for non JavaScript pages.