I have an AWS Glue 5.0 job where I am specifying --additional-python-modules s3://my-dev/other-dependencies /MyPackage-0.1.1-py3-none-any.whl
in my job options.
My glue job itself is just a print("hello")
job, because when I save and run this glue job, it runs for 10 minutes 7 seconds in AWS< then fails with this error
LAUNCH ERROR | Glue bootstrap failed. Please refer logs for details. caused by LAUNCH ERROR | Installation of Python modules timed out after 10 minutesPlease refer logs for details.
my packages setup.py does use thee libraires:
from setuptools import setup, find_packages
setup(
name="...",
version="0.1.1",
packages=find_packages(),
install_requires=[
'dask[array]',
'zarr',
'scipy',
'scikit-image',
'bioio',
'bioio-tifffile',
'tifffile',
'opencv-python',
'torch',
'pyyaml',
'xmltodict'
],
entry_points={...},
)
Is there anyway I can tell AWS with a flag in an AWS Glue 5.0 job that I need more then a 10 minute install timeout?