I've been using crawlee for python recently to build reliable web scrapers, but sometimes I needed to execute some code before visiting the page url like:
- blocking specific resources (ex: imaged, media, etc)
- passing the created page to
playwright-stealth
like this:await stealth_async(page)
I searched the Docs for a parameter or a class with a similar functionality & found that PlaywrightCrawler
has a param called pre_navigation_hook
. However, I couldn't find how to use it or a tutorial on their site demonstrating that.
Please provide an example how to use it or any other way to achieve the points above.
Note: pre_navigation_hook
accepts a PlaywrightPreNavCrawlingContext
as mentioned in docs not PlaywrightCrawlingContext
like almost all other request_handler
s if this makes any hint.