This is a code that used Selenium to crawl the web, but .get() does not seem to work after updating Chrome and Chrome Driver.

Chrome version is "133.0.6943.99" and the Chrome Driver version is "133.0.6943.98".

# No.1 
# This code works.
driver = webdriver.Chrome()

url = ";
driver.get(url)

# No.2
# This code runs, but does not connect to the specified URL.
options = Options()
options.add_argument(r"USER_DATA_PATH") # "USER_DATA_PATH" is the path where the actual Chrome profile account is saved.

driver = webdriver.Chrome(options=options)

url = ";
driver.get(url)

There are some operations that can only be done using a Chrome profile account. Is there anyone who is experiencing the same problem or knows a solution?

For tasks that do not require a Chrome profile account, the URL was accessed normally when the first code was used, and crawling was completed without any problems.

However, when the second code is executed, the Chrome instance and profile are loaded normally, but it does not go to the specified URL. No error message or error appeared when executing the code. Code execution terminated normally.

This is a code that used Selenium to crawl the web, but .get() does not seem to work after updating Chrome and Chrome Driver.

Chrome version is "133.0.6943.99" and the Chrome Driver version is "133.0.6943.98".

# No.1 
# This code works.
driver = webdriver.Chrome()

url = "https://www.google"
driver.get(url)

# No.2
# This code runs, but does not connect to the specified URL.
options = Options()
options.add_argument(r"USER_DATA_PATH") # "USER_DATA_PATH" is the path where the actual Chrome profile account is saved.

driver = webdriver.Chrome(options=options)

url = "https://www.google"
driver.get(url)

There are some operations that can only be done using a Chrome profile account. Is there anyone who is experiencing the same problem or knows a solution?

For tasks that do not require a Chrome profile account, the URL was accessed normally when the first code was used, and crawling was completed without any problems.

However, when the second code is executed, the Chrome instance and profile are loaded normally, but it does not go to the specified URL. No error message or error appeared when executing the code. Code execution terminated normally.

• python
• selenium-webdriver
• web-scraping
• get
• selenium-chromedriver

1 Answer 1

Without specifying the capability, providing only the path will not work.

Check here for the chromedriver capabilities and examples of how to use them.

You have to use the user-data-dir capability which takes the path.

Try the following:

options = Options()
options.add_argument(r"user-data-dir=USER_DATA_PATH") # "USER_DATA_PATH" is the profile path 

driver = webdriver.Chrome(options=options)

url = "https://www.google"
driver.get(url)

Update:

The profile resides inside the directory used for user-data-dir. So for default chrome installation in windows the main profile directory would be

C:\Users\UserName\AppData\Local\Google\Chrome\User Data

And the default profile directory would be

C:\Users\Lenovo\AppData\Local\Google\Chrome\User Data\Default

So to use the default profile, you have to use the first one as the value of user-data-dir. If you have multiple profiles and you want to specify a profile, Then you have to use the capability profile-directory with user-data-dir. You have to provide the profile name with profile0directory

It would look like

options = Options()
options.add_argument(r"--user-data-dir=USER_DATA_PATH")
options.add_argument("--profile-directory=DesiredProfile")

If you set the profile directory to the user-data-dir, it will create a new data directory inside the profile directory ,and inside that there will be a new default profile. That profile will not be logged in on any google account as it is newly created.