I've tried all logical divs to get the dropdown list, but could not make it '.click'. The URL I am trying to get the information from is /. I don't get any mistakes but when the code is running Chrome opens the site and just closes without clicking the link to make dropdown visible, so I could scrape data from it.

from selenium import webdriver
from selenium.webdrivermon.by import By
import csv

driver = webdriver.Chrome()
url = f''
driver.get(url)

div = driver.find_element(By.ID, value='region-selected-value')
div.click()
#selector = div.find_element(By.TAG_NAME,value='select')

print(div.text)

I've tried all logical divs to get the dropdown list, but could not make it '.click'. The URL I am trying to get the information from is https://krisha.kz/. I don't get any mistakes but when the code is running Chrome opens the site and just closes without clicking the link to make dropdown visible, so I could scrape data from it.

from selenium import webdriver
from selenium.webdrivermon.by import By
import csv

driver = webdriver.Chrome()
url = f'https://krisha.kz'
driver.get(url)

div = driver.find_element(By.ID, value='region-selected-value')
div.click()
#selector = div.find_element(By.TAG_NAME,value='select')

print(div.text)

• python
• selenium-webdriver
• web-scraping

• Your code looks just fine. Do you see any errors/exceptions in the console? Just apply a few minutes sleep at the end of your code like this time.sleep(20). You will notice that the dropdown is clicked and can see the dropdown window. – Shawn Commented Nov 19, 2024 at 9:33

1 Answer 1

As I have mentioned in the comment, your code to click on that dropdown element is just fine. I have tested it and it clicks the dropdown element opening the dropdown options. You just need to put a time.sleep(20) at the end so that you see it before the browser closes down.

Having said that, as I understand you are looking to scrape all the dropdown values. If that is correct, see the code below:

from selenium import webdriver
from selenium.webdrivermon.by import By
from selenium.webdriver.support.ui import Select

driver = webdriver.Chrome()
url = f'https://krisha.kz'
driver.get(url)
select = Select(driver.find_element(By.NAME, "das[live.rooms]"))
options = select.options
for option in options:
    print(option.text)

Result:

1 - комн.
1-2 - комн.
2 - комн.
2-3 - комн.
3 - комн.
3-4 - комн.
4 - комн.
4-5 - комн.
5 и более комн.

Process finished with exit code 0