Following is an abridged version of the Dockerfile I'm using

FROM pangeo/pangeo-notebook:2024.04.08 AS base
USER root
# Install dependencies
RUN apt-get update && apt-get install -y vim gfortran sqlite3 libsqlite3-dev ...
RUN mamba install -c conda-fe libboost -y #<--------------------------- libboost installation

FROM base AS prebuild
...

FROM prebuild AS app_clone
WORKDIR /app
RUN git clone ...

FROM app_clone AS app_build
ENV PATH=${PATH}:/usr/bin/mpich
ENV PATH=/usr/bin:${PATH} #<-------------- Env variable not showing in docker image 
ENV CC=/usr/bin/gcc #<------------- Env variable not showing in docker image
WORKDIR /app

RUN cmake -G Ninja -B cmake_build_serial -S . && \
    cmake --build cmake_build_serial --target all -- -j $(nproc)

I installed libboost using mamba since I want at least version 1.79.0. There are two issues happening

1. When I try to build the image using docker build --no-cache --platform linux/amd64 -t test-app ., it shows following error

0.788 -- The C compiler identification is GNU 11.4.0                                                                                                                                
1.204 -- The CXX compiler identification is GNU 11.4.0
1.238 -- Detecting C compiler ABI info
1.803 -- Detecting C compiler ABI info - done
1.822 -- Check for working C compiler: /usr/bin/gcc - skipped
1.823 -- Detecting C compile features
1.824 -- Detecting C compile features - done
1.828 -- Detecting CXX compiler ABI info
2.345 -- Detecting CXX compiler ABI info - done
2.365 -- Check for working CXX compiler: /usr/bin/g++ - skipped
2.365 -- Detecting CXX compile features
2.366 -- Detecting CXX compile features - done
2.384 CMake Error at /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
2.384   Could NOT find Boost (missing: Boost_INCLUDE_DIR) (Required is at least
2.384   version "1.79.0")
2.384 Call Stack (most recent call first):
2.384   /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
2.384   /usr/share/cmake-3.22/Modules/FindBoost.cmake:2360 (find_package_handle_standard_args)
2.384   CMakeLists.txt:168 (find_package)
2.384 
2.384 -- Configuring incomplete, errors occurred!
------

However,

• if I comment the last line (cmake ...) in the dockerfile
• run container docker run -it test-app /bin/bash
• manually copy the cmake ... command

the compilation completes without any issue.

2. Also, when inside the container, the environment variables are not set i.e.

• CC points to old value /srv/conda/envs/notebook/bin/x86_64-conda-linux-gnu-cc
• PATH does not have /usr/bin as prefix

Can someone please point out what I'm missing here?

[SOLUTION] Thanks for all the comments here and on a related post which guided me to a solution posted here. What worked for me was to modify the Dockerfile to use/activate notebook environment during image building before cmake ... as follows

# Make RUN commands use the notebook environment
SHELL ["mamba", "run", "--no-capture-output", "-n", "notebook", "/bin/bash", "-c"]
RUN cmake -G Ninja -B cmake_build_serial -S . && \
    cmake --build cmake_build_serial --target all -- -j $(nproc)

Following is an abridged version of the Dockerfile I'm using

FROM pangeo/pangeo-notebook:2024.04.08 AS base
USER root
# Install dependencies
RUN apt-get update && apt-get install -y vim gfortran sqlite3 libsqlite3-dev ...
RUN mamba install -c conda-fe libboost -y #<--------------------------- libboost installation

FROM base AS prebuild
...

FROM prebuild AS app_clone
WORKDIR /app
RUN git clone ...

FROM app_clone AS app_build
ENV PATH=${PATH}:/usr/bin/mpich
ENV PATH=/usr/bin:${PATH} #<-------------- Env variable not showing in docker image 
ENV CC=/usr/bin/gcc #<------------- Env variable not showing in docker image
WORKDIR /app

RUN cmake -G Ninja -B cmake_build_serial -S . && \
    cmake --build cmake_build_serial --target all -- -j $(nproc)

I installed libboost using mamba since I want at least version 1.79.0. There are two issues happening

1. When I try to build the image using docker build --no-cache --platform linux/amd64 -t test-app ., it shows following error

0.788 -- The C compiler identification is GNU 11.4.0                                                                                                                                
1.204 -- The CXX compiler identification is GNU 11.4.0
1.238 -- Detecting C compiler ABI info
1.803 -- Detecting C compiler ABI info - done
1.822 -- Check for working C compiler: /usr/bin/gcc - skipped
1.823 -- Detecting C compile features
1.824 -- Detecting C compile features - done
1.828 -- Detecting CXX compiler ABI info
2.345 -- Detecting CXX compiler ABI info - done
2.365 -- Check for working CXX compiler: /usr/bin/g++ - skipped
2.365 -- Detecting CXX compile features
2.366 -- Detecting CXX compile features - done
2.384 CMake Error at /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
2.384   Could NOT find Boost (missing: Boost_INCLUDE_DIR) (Required is at least
2.384   version "1.79.0")
2.384 Call Stack (most recent call first):
2.384   /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
2.384   /usr/share/cmake-3.22/Modules/FindBoost.cmake:2360 (find_package_handle_standard_args)
2.384   CMakeLists.txt:168 (find_package)
2.384 
2.384 -- Configuring incomplete, errors occurred!
------

However,

• if I comment the last line (cmake ...) in the dockerfile
• run container docker run -it test-app /bin/bash
• manually copy the cmake ... command

the compilation completes without any issue.

2. Also, when inside the container, the environment variables are not set i.e.

• CC points to old value /srv/conda/envs/notebook/bin/x86_64-conda-linux-gnu-cc
• PATH does not have /usr/bin as prefix

Can someone please point out what I'm missing here?

[SOLUTION] Thanks for all the comments here and on a related post which guided me to a solution posted here. What worked for me was to modify the Dockerfile to use/activate notebook environment during image building before cmake ... as follows

# Make RUN commands use the notebook environment
SHELL ["mamba", "run", "--no-capture-output", "-n", "notebook", "/bin/bash", "-c"]
RUN cmake -G Ninja -B cmake_build_serial -S . && \
    cmake --build cmake_build_serial --target all -- -j $(nproc)

• docker
• cmake
• boost
• dockerfile
• mamba

2 Answers 2

The reason that cmake finds the library at runtime, but not whilst building the container is due to the way that the environment is set-up; the Docker image has some machinery configured to activate your mamba environment at runtime. But, during build time, the environment is not yet active, and so the dependencies are not visible.

The fix is to activate the mamba environment for the RUN step. However, it's really not advisable to mix conda and system packages like this. If you're needing to compile something, I'd suggest building boost yourself, or moving exclusively to conda.

The likely problem here is the use of ENV in a multi-stage build. It is important to note that ENV variables do not persist across build stages but they will all be available at container runtime. This is likely why you see different behavior running the build from a container instance vs how it runs during the build stage.

1. Specific things to check here:

• Are you explicitly setting Boost_INCLUDE_DIR in an env variable in a different stage?

• Do you need to activate a conda environment in the build stage which puts libboost components into standard search paths?

2. Are you changing some of these ENV variables in different build stages? Or perhaps have some other steps which are manipulating standard ENV variables with this unintended side effect?