I'm encountering a PermissionError: [Errno 13] Permission denied when trying to initialize a Selenium ChromeDriver in an Airflow task (using docker). The error occurs specifically when the script attempts to patch the ChromeDriver executable located at /usr/local/bin/chromedriver. Here's the relevant part of the log:
[2025-01-18, 22:30:22 UTC] {logging_mixin.py:190} INFO - Failed to initialize WebDriver: [Errno 13] Permission denied: '/usr/local/bin/chromedriver'
What I've Tried: Checked File Permissions:
Verified that /usr/local/bin/chromedriver exists and has execute permissions (chmod +x /usr/local/bin/chromedriver).
Confirmed that the file is owned by the user running the Airflow task (chown $USER /usr/local/bin/chromedriver).
Verified PATH:
Ensured that /usr/local/bin/ is included in the PATH environment variable.
Reinstalled ChromeDriver:
Downloaded and reinstalled ChromeDriver from the official site to ensure compatibility with the installed Chrome version.
Ran with Elevated Permissions:
Temporarily ran the script with sudo to test if it was a permission issue, but this is not a viable long-term solution.
Updated Dependencies:
Reinstalled Selenium and undetected_chromedriver to ensure they are up to date.
Here's the relevant part of the code where the error occurs:
from undetected_chromedriver import Chrome as uc
def initialize_driver(headless=True):
driver = uc.Chrome(
headless=headless,
executable_path='/usr/local/bin/chromedriver'
)
return driver
The full traceback is as follows:
Traceback (most recent call last):
File "/opt/airflow/dags/job_scraper_dag.py", line 40, in scrape_and_store
driver = initialize_driver(headless=config["HEADLESS"])
File "/opt/airflow/dags/app/scrape.py", line 54, in initialize_driver
driver = uc.Chrome()
File "/home/airflow/.local/lib/python3.11/site-packages/undetected_chromedriver/__init__.py", line 258, in __init__
self.patcher.auto()
File "/home/airflow/.local/lib/python3.11/site-packages/undetected_chromedriver/patcher.py", line 150, in auto
return self.patch_exe()
File "/home/airflow/.local/lib/python3.11/site-packages/undetected_chromedriver/patcher.py", line 347, in patch_exe
with io.open(self.executable_path, "r+b") as fh:
PermissionError: [Errno 13] Permission denied: '/usr/local/bin/chromedriver'
Environment: Airflow Version: 2.10.4
Python Version: 3.11
ChromeDriver Version: 132.0.6834.83