I created a Docker container by using a custom base image, and then I installed some other python packages by:

pip install -r requirements_base.txt

with this requirements_base.txt file (some packages are taken from a custom pip registry):

awscli==1.33.0
model_evaluation==1.17.3
torch==2.2.0
torchvision==0.17.0
tqdm==4.66.3
mlflow==2.16.0
numpy<2
opencv-python==4.8.0.74
imgaug==0.4.0
optuna==3.6.1
imagecorruptions==1.1.2
utilities==3.18.0
model-registry==0.1.4
scikit-learn==1.5.0

After the installation is done, from inside the container I ran:

pip freeze > requirements.txt

so that requirements.txt contains exactly what was installed in my container, which is:

aiobotocore==2.13.3
aiohappyeyeballs==2.4.4
aiohttp==3.11.11
aiohttp-retry==2.9.1
aioitertools==0.12.0
aiosignal==1.3.2
albumentations==1.3.1
alembic==1.14.1
amqp==5.3.1
annotated-types==0.7.0
antlr4-python3-runtime==4.9.3
anyio==4.4.0
appdirs==1.4.4
argon2-cffi==23.1.0
argon2-cffi-bindings==21.2.0
arrow==1.3.0
asttokens==2.4.1
async-lru==2.0.4
asyncssh==2.19.0
atpublic==5.1
attrs==23.2.0
audioread==3.0.1
awscli==1.33.0
Babel==2.15.0
beautifulsoup4==4.12.3
billiard==4.2.1
bleach==6.1.0
blinker==1.9.0
boto3==1.34.118
botocore==1.34.118
cachetools==5.5.1
celery==5.4.0
certifi==2024.6.2
cffi==1.17.1
cfgv==3.4.0
charset-normalizer==3.3.2
click==8.1.8
click-didyoumean==0.3.1
click-plugins==1.1.1
click-repl==0.3.0
cloudpickle==3.1.1
colorama==0.4.6
colorlog==6.9.0
comm==0.2.2
configobj==5.0.9
contourpy==1.2.1
coverage==7.5.3
cryptography==44.0.0
cv-inference-pipeline==3.4.0
cv-model-evaluation==1.17.3
cv-utilities==3.18.0
cvaws==1.1.1
cvlogging==1.0.0
cvpipeline==1.5.2
cycler==0.12.1
Cython==3.0.6
databricks-sdk==0.42.0
dbus-python==1.2.18
debugpy==1.8.1
decorator==5.1.1
defusedxml==0.7.1
Deprecated==1.2.18
dictdiffer==0.9.0
diskcache==5.6.3
distlib==0.3.8
distro==1.7.0
distro-info==1.1+ubuntu0.2
docker==7.1.0
docutils==0.16
dpath==2.2.0
dulwich==0.22.7
dvc==3.59.0
dvc-data==3.16.8
dvc-http==2.32.0
dvc-objects==5.1.0
dvc-render==1.0.2
dvc-s3==3.2.0
dvc-studio-client==0.21.0
dvc-task==0.40.2
editdistance==0.8.1
entrypoints==0.4
et_xmlfile==2.0.0
executing==2.0.1
fastjsonschema==2.20.0
filelock==3.15.1
flake8==6.1.0
Flask==3.1.0
flatten-dict==0.4.2
flufl.lock==8.1.0
fonttools==4.53.0
fqdn==1.5.1
frozenlist==1.5.0
fsspec==2024.12.0
funcy==2.0
fuzzywuzzy==0.18.0
gitdb==4.0.12
GitPython==3.1.44
google-auth==2.38.0
grandalf==0.8
graphene==3.4.3
graphql-core==3.2.6
graphql-relay==3.2.0
greenlet==3.1.1
grpcio==1.70.0
gto==1.7.2
gunicorn==23.0.0
h11==0.14.0
httpcore==1.0.5
httplib2==0.20.2
httpx==0.27.0
hydra-core==1.3.2
identify==2.5.36
idna==3.7
imagecorruptions==1.1.2
imageio==2.34.1
imageio-ffmpeg==0.6.0
imgaug==0.4.0
importlib_metadata==8.5.0
iniconfig==2.0.0
ipykernel==6.29.4
ipython==8.25.0
ipywidgets==8.1.1
isoduration==20.11.0
iterative-telemetry==0.0.9
itsdangerous==2.2.0
jedi==0.19.1
jeepney==0.7.1
Jinja2==3.1.4
jmespath==1.0.1
joblib==1.4.2
json5==0.9.25
jsonpickle==4.0.1
jsonpointer==3.0.0
jsonschema==4.22.0
jsonschema-specifications==2023.12.1
jupyter-events==0.10.0
jupyter-lsp==2.2.5
jupyter_client==8.6.2
jupyter_core==5.7.2
jupyter_server==2.14.1
jupyter_server_terminals==0.5.3
jupyterlab==4.2.2
jupyterlab_pygments==0.3.0
jupyterlab_server==2.27.2
jupyterlab_widgets==3.0.11
keyring==23.5.0
kiwisolver==1.4.5
kombu==5.4.2
lazr.uri==1.0.6
lazy_loader==0.4
Levenshtein==0.26.1
librosa==0.10.2.post1
llvmlite==0.44.0
Mako==1.3.8
Markdown==3.7
markdown-it-py==3.0.0
MarkupSafe==2.1.5
matplotlib==3.8.2
matplotlib-inline==0.1.7
mccabe==0.7.0
mdurl==0.1.2
mergedeep==1.3.4
mistune==3.0.2
mlflow==2.16.0
mlflow-skinny==2.16.0
model-registry==0.1.4
more-itertools==8.10.0
moviepy==2.1.2
mpmath==1.3.0
msgpack==1.1.0
multidict==6.1.0
mypy==1.7.1
mypy-extensions==1.0.0
nbclient==0.10.0
nbconvert==7.16.4
nbformat==5.10.4
nest-asyncio==1.6.0
networkx==3.3
nodeenv==1.9.1
notebook==7.0.6
notebook_shim==0.2.4
numba==0.61.0
numpy==1.26.2
nvidia-cublas-cu12==12.1.3.1
nvidia-cuda-cupti-cu12==12.1.105
nvidia-cuda-nvrtc-cu12==12.1.105
nvidia-cuda-runtime-cu12==12.1.105
nvidia-cudnn-cu12==8.9.2.26
nvidia-cufft-cu12==11.0.2.54
nvidia-curand-cu12==10.3.2.106
nvidia-cusolver-cu12==11.4.5.107
nvidia-cusparse-cu12==12.1.0.106
nvidia-nccl-cu12==2.19.3
nvidia-nvjitlink-cu12==12.8.61
nvidia-nvtx-cu12==12.1.105
omegaconf==2.3.0
opencv-python==4.8.0.74
opencv-python-headless==4.10.0.82
openpyxl==3.1.5
opentelemetry-api==1.29.0
opentelemetry-sdk==1.29.0
opentelemetry-semantic-conventions==0.50b0
optuna==3.6.1
orjson==3.10.15
overrides==7.7.0
packaging==23.2
pandas==2.1.4
pandocfilters==1.5.1
parso==0.8.4
path==17.1.0
pathspec==0.12.1
pexpect==4.9.0
Pillow==10.1.0
platformdirs==4.2.2
pluggy==1.5.0
pooch==1.8.2
pre-commit==3.6.0
proglog==0.1.10
prometheus_client==0.20.0
prompt_toolkit==3.0.47
propcache==0.2.1
protobuf==5.29.3
psutil==5.9.7
ptyprocess==0.7.0
pure-eval==0.2.2
pyarrow==17.0.0
pyasn1==0.6.1
pyasn1_modules==0.4.1
pycodestyle==2.11.1
pycparser==2.22
pydantic==2.5.2
pydantic_core==2.14.5
pydot==3.0.4
pyflakes==3.1.0
pygit2==1.17.0
Pygments==2.18.0
PyGObject==3.42.1
pygtrie==2.5.0
PyJWT==2.3.0
pyparsing==3.2.1
pyrsistent==0.20.0
pytest==7.2.2
pytest-cov==4.0.0
python-apt==2.4.0+ubuntu3
python-dateutil==2.9.0.post0
python-dotenv==1.0.1
python-json-logger==2.0.7
python-Levenshtein==0.26.1
pytz==2024.1
pyvis==0.3.2
PyYAML==6.0.1
pyzmq==25.1.2
qudida==0.0.4
RapidFuzz==3.11.0
referencing==0.35.1
requests==2.31.0
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
rich==13.9.4
rpds-py==0.18.1
rsa==4.7.2
ruamel.yaml==0.18.10
ruamel.yaml.clib==0.2.12
s3fs==2024.12.0
s3transfer==0.10.4
scikit-image==0.22.0
scikit-learn==1.5.0
scipy==1.11.4
scmrepo==3.3.9
SecretStorage==3.3.1
semver==3.0.4
Send2Trash==1.8.3
shapely==2.0.6
shellingham==1.5.4
shortuuid==1.0.13
shtab==1.7.1
six==1.16.0
smmap==5.0.2
sniffio==1.3.1
soundfile==0.13.1
soupsieve==2.5
soxr==0.5.0.post1
SQLAlchemy==2.0.37
sqlparse==0.5.3
sqltrie==0.11.1
stack-data==0.6.3
sympy==1.12
tabulate==0.9.0
terminado==0.18.1
threadpoolctl==3.5.0
tifffile==2024.5.22
tinycss2==1.3.0
tomlkit==0.13.2
torch==2.2.0
torchvision==0.17.0
tornado==6.4.1
tqdm==4.66.3
traitlets==5.14.3
triton==2.2.0
typer==0.15.1
types-python-dateutil==2.9.0.20240316
typing_extensions==4.12.2
tzdata==2024.1
unattended-upgrades==0.1
uri-template==1.3.0
urllib3==2.2.1
vine==5.1.0
virtualenv==20.26.2
voluptuous==0.15.2
wadllib==1.3.6
wcwidth==0.2.13
webcolors==24.6.0
webencodings==0.5.1
websocket-client==1.8.0
Werkzeug==3.1.3
widgetsnbextension==4.0.11
wrapt==1.17.2
yarl==1.18.3
zc.lockfile==3.0.post1
zipp==3.21.0

Then, I tried rebuilding the image and installing everything during the build process. So, I defined a new requirements_dev.txt:

 # these 2 entries have been moved here from the requirements.txt file generated with pip freeze, so they don't appear in requirements.txt anymore
pytest==7.2.2
pytest-cov==4.0.0.

-r requirements.txt

and built the image:

python3 -m pip install -r requirements_dev.txt --extra-index-url "https://user_name:token@gitlab_url1"

and I'm getting this error:

67.58 Collecting pycairo>=1.16.0 (from PyGObject==3.42.1->-r /workspace/requirements.txt (line 237))
[...]   
73.71       ../cairo/meson.build:31:12: ERROR: Dependency "cairo" not found, tried pkgconfig and cmake

The error is easily solvable, but what I'm trying to understand is WHY THIS IS HAPPENING TO BEGIN WITH.

Again, since what I did was:

• install stuff, letting pip solve dependencies of the secondary packages
• freeze the environment that was successfully installed
• try to recreate the environment all in one go, using the requirements file generated after the first step.

I would expect the result to be identical, where does the difference arise the second time I crate the environment? Specifically, the error is related to the installation of pycairo, which is a dependency of PyGObject (see the error I posted).
pycairo was not present in the initial container, and is not listed in the requirements.txt file, so I guess it wasn't needed during the first

pip install -r requirements_base.txt

then why is it an issue afterwards?