I want to webscrape the following website:

But when my code clicks on the "Search button", I get the error:

ElementClickInterceptedException: element click intercepted: Element is not clickable at point (320, 1395)

I am not sure if it's because the Search button is not pletely visible and needs scrolling down. If so, how do I scroll down the page using Selenium?

Code trials:

from bs4 import BeautifulSoup as bs 
from selenium import webdriver
import time

driver = webdriver.Chrome(executable_path="C:/Users/chromedriver.exe")
page_url = ''
driver.get(page_url)
# Get search box and fill it up
search = driver.find_element_by_css_selector('#divmpscreen2 > div.row > div:nth-child(1) > div > div:nth-child(1) > input')
search.send_keys('COS')
# This will select the 13th parliament
session = driver.find_element_by_xpath("//select/option[contains(text(), '13th')]")
session.click()
time.sleep(10)
# Find submit element and click
submit = driver.find_element_by_xpath("//button[@label='Search']/em")
submit.click()

I want to webscrape the following website: https://sprs.parl.gov.sg/search/home

But when my code clicks on the "Search button", I get the error:

ElementClickInterceptedException: element click intercepted: Element is not clickable at point (320, 1395)

I am not sure if it's because the Search button is not pletely visible and needs scrolling down. If so, how do I scroll down the page using Selenium?

Code trials:

from bs4 import BeautifulSoup as bs 
from selenium import webdriver
import time

driver = webdriver.Chrome(executable_path="C:/Users/chromedriver.exe")
page_url = 'https://sprs.parl.gov.sg/search/home'
driver.get(page_url)
# Get search box and fill it up
search = driver.find_element_by_css_selector('#divmpscreen2 > div.row > div:nth-child(1) > div > div:nth-child(1) > input')
search.send_keys('COS')
# This will select the 13th parliament
session = driver.find_element_by_xpath("//select/option[contains(text(), '13th')]")
session.click()
time.sleep(10)
# Find submit element and click
submit = driver.find_element_by_xpath("//button[@label='Search']/em")
submit.click()

• javascript
• python
• selenium
• selenium-webdriver
• webdriverwait

4 Answers 4

We can try click by using moving to element or by using java script executor

Actions action = new Actions(webdriver);
WebElement we = webdriver.findElement(By.xpath("path here"));
action.moveToElement(we).click().build().perform();

using javascript

WebElement element = driver.findElement(By.id("gbqfd"));
JavascriptExecutor executor = (JavascriptExecutor)driver;
executor.executeScript("arguments[0].click();", element);

This is one way to select the desired info and click that button (EDIT: a better way without ActionChains, and working on smaller sized windows as well):

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.mon.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
import time as t


chrome_options = Options()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument('disable-notifications')
chrome_options.add_argument("window-size=1280,720")

webdriver_service = Service("chromedriver/chromedriver") ## path to where you saved chromedriver binary
browser = webdriver.Chrome(service=webdriver_service, options=chrome_options)

url = 'https://sprs.parl.gov.sg/search/home'

browser.get(url) 
searchbox = WebDriverWait(browser, 20).until(EC.presence_of_element_located((By.XPATH, "//label[contains(text(), 'By Keyword')]/following-sibling::input")))
print(searchbox.location_once_scrolled_into_view)
searchbox.send_keys('COS')
govt_nr = Select(WebDriverWait(browser, 20).until(EC.presence_of_element_located((By.XPATH, "//label[contains(text(), 'By Parliament')]/following-sibling::select"))))
govt_nr.select_by_index(13) 
browser.execute_script("window.scrollTo(0,document.body.scrollHeight);")
t.sleep(1)
submit_button = WebDriverWait(browser, 20).until(EC.presence_of_element_located((By.CSS_SELECTOR, "button[label = 'Search']")))
submit_button.click()

Setup is linux/chromedriver, you just have to observe the imports, and the code after defining the browser/driver.

Selenium docs: https://www.selenium.dev/documentation/

One more thing to keep in mind: results will open in a new tab, so you will need to switch to it using something to the tune of switch_to.window(driver.window_handles[-1]).

This is how you suppose to click an element using Selenium:

def clickLastFoundElement(my_element):       
    try:
        builder = Actions(browser)
        builder.moveToElement(my_element).click().build().perform()
    except ElementNotInteractableException:
        try:
            my_element.click()          
        except Exception:
            hardClickElement(my_element)

def hardClickElement(element):
    executor = (JavascriptExecutor)browser
    executor.executeScript('arguments[0].click();', element)

Sometimes an element won't be clicked, and you will have to "hard" click it by injecting JavaScript code into it.

To click on the Search button you need to induce WebDriverWait for the element_to_be_clickable() and you can use the following locator strategy:

driver.get('https://sprs.parl.gov.sg/search/home')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#divmpscreen2 > div.row > div:nth-child(1) > div > div:nth-child(1) > input"))).send_keys('COS')
Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//label[text()='By Parliament']//following-sibling::select[1]")))).select_by_value('13: 13')
driver.execute_script("arguments[0].click();", WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@class='btn btn-black' and @label='Search']/em"))))

Browser Snapshot:

Note: You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.mon.by import By
from selenium.webdriver.support import expected_conditions as EC