I am trying to perform a controlled scroll on the home feed of a lazy loading website (similar to LinkedIn). But playwright does not support auto-scroll. So I have used the following methods:
- Find the "Show more results" button and click it
await page.locator('button:has-text("Show more results")').click();
The issue in this method is that the click goes on an endless loop and times out after default 30 seconds. If I set timeout:0 then it doesn't time out, it goes on for an infinite loop.
- While trying to control the click with
waitForSelectorasshown below:
for (let i = 0; i < 20; i++) {
let loadMore = await page.waitForSelector(('button:has-text("Show more results")');
await loadMore.click({delay: 1000, timeout: 0});
}
It throws Element is not attached to DOM error. Log as below:
elementHandle.click: Element is not attached to the DOM
=========================== logs ===========================
attempting click action
waiting for element to be visible, enabled and stable
element is visible, enabled and stable
scrolling into view if needed
done scrolling
performing click action
<html lang="en" class="theme theme--mercado artdeco os…>…</html> intercepts pointer events
retrying click action, attempt #1
waiting for element to be visible, enabled and stable
============================================================
The only possible way which could see working is:
for (let i = 0; i < 20; i++) {
await page.keyboard.down('PageDown');
}
Any suggestions on better controlled scroll with Playwright library would be helpful :) Thank you.
I am trying to perform a controlled scroll on the home feed of a lazy loading website (similar to LinkedIn.). But playwright does not support auto-scroll. So I have used the following methods:
- Find the "Show more results" button and click it
await page.locator('button:has-text("Show more results")').click();
The issue in this method is that the click goes on an endless loop and times out after default 30 seconds. If I set timeout:0 then it doesn't time out, it goes on for an infinite loop.
- While trying to control the click with
waitForSelectorasshown below:
for (let i = 0; i < 20; i++) {
let loadMore = await page.waitForSelector(('button:has-text("Show more results")');
await loadMore.click({delay: 1000, timeout: 0});
}
It throws Element is not attached to DOM error. Log as below:
elementHandle.click: Element is not attached to the DOM
=========================== logs ===========================
attempting click action
waiting for element to be visible, enabled and stable
element is visible, enabled and stable
scrolling into view if needed
done scrolling
performing click action
<html lang="en" class="theme theme--mercado artdeco os…>…</html> intercepts pointer events
retrying click action, attempt #1
waiting for element to be visible, enabled and stable
============================================================
The only possible way which could see working is:
for (let i = 0; i < 20; i++) {
await page.keyboard.down('PageDown');
}
Any suggestions on better controlled scroll with Playwright library would be helpful :) Thank you.
Share Improve this question asked Apr 30, 2022 at 14:27 CaptV89CaptV89 711 silver badge5 bronze badges2 Answers
Reset to default 2You could try to use
await page.keyboard.press('End');
to scroll to the end of the page and then try it again with your locator on the show more results button.
Alternately you could also try:
await page.locator(('button:has-text("Show more results")').scrollIntoViewIfNeeded();
or more directly
await page.locator('button:has-text("Show more results")').click()
because playwright click should automatically scroll the elements into view before trying to click.
I created this auto-scroll function to scroll to the bottom of the page for Shopee (shopee.sg). My app was using a proxy and a VPN.
async function autoScroll(page) {
const maxScrolls = 100;
const scrollDelay = 4000;
let previousHeight = 0;
let scrollAttempts = 0;
while (scrollAttempts < maxScrolls) {
await page.keyboard.down('End');
await page.waitForTimeout(scrollDelay);
const currentHeight = await page.evaluate(() => document.body.scrollHeight);
if (currentHeight === previousHeight) {
break;
}
previousHeight = currentHeight;
scrollAttempts++;
}
}