I have a simple Python project with the following structure:
root/
│── src/
│ ├── package_name/
│ │ ├── __init__.py
│ │ ├── main.py
│ │ ├── submodules1/
│ │ │ ├── __init__.py
│ │ │ ├── base1.py
│ │ ├── submodules2/
│ │ │ ├── __init__.py
│ │ │ ├── base2.py
│── pyproject.toml
My primary concern is that while I can schedule a notebook or a single Python file in Databricks, I am unable to schedule an entire project that depends on `main.py`.
The project functions as a CLI application and requires arguments to run. I need guidance on how to properly schedule this complete Python project in Databricks. Any suggestions or best practices would be greatly appreciated!
Thanks in advance.