linux - How to tell julia that it should use a locally installed glibc version for compilation?

I am working on a HPC3 and want to install CUDA.jl to utilize the GPU while working in Julia. While compiling CUDA.jl (via Pkg.build("CUDA")), I realized that my glibc version is too low (currently 2.17, >= 2.27 needed). The system admin suggested to install a newer glibc version locally in /home/user/. That's what I did:

$ ldd --version # check old version
ldd (GNU libc) 2.17
$ /home/user/glibc-install/lib/ld-linux-x86-64.so.2 --version # installed new glibc in glibc-install
ld.so (GNU libc) stable release version 2.41.

I am unable to figure out how to run julia with this new glibc version. I tried a bunch of different stuff. For example setting the LD_LIBRARY_PATH gives

$ export LD_LIBRARY_PATH=/home/user/glibc-install/lib:/home/user/julia/julia/lib/julia:$LD_LIBRARY_PATH
$ /home/user/glibc-install/lib/ld-linux-x86-64.so.2 --library-path /home/user/glibc-install/lib:/home/user/julia/julia/lib/julia/ $(which julia)
/usr/bin/which: /lib64/ld-linux-x86-64.so.2: version `GLIBC_2.35' not found (required by /home/user/glibc-install/lib/libc.so.6)
/home/user/glibc-install/lib/ld-linux-x86-64.so.2: missing program name
Try '/home/user/glibc-install/lib/ld-linux-x86-64.so.2 --help' for more information.

I did not encounter any issues while installing glibc-2.41 locally but I don't know how to proceed.

Question: How to tell julia that I want to use glibc-2.41 to compile CUDA.jl? Is there a way to open the julia REPL using this newer glibc version? Also, is there a way to really know for sure if there was not any mistake with the installation of the new glibc version?

I am happy for any answer, also just to boil down what the exact problem might be.

Thanks in advance!

$ ldd --version # check old version
ldd (GNU libc) 2.17
$ /home/user/glibc-install/lib/ld-linux-x86-64.so.2 --version # installed new glibc in glibc-install
ld.so (GNU libc) stable release version 2.41.

I am unable to figure out how to run julia with this new glibc version. I tried a bunch of different stuff. For example setting the LD_LIBRARY_PATH gives

$ export LD_LIBRARY_PATH=/home/user/glibc-install/lib:/home/user/julia/julia/lib/julia:$LD_LIBRARY_PATH
$ /home/user/glibc-install/lib/ld-linux-x86-64.so.2 --library-path /home/user/glibc-install/lib:/home/user/julia/julia/lib/julia/ $(which julia)
/usr/bin/which: /lib64/ld-linux-x86-64.so.2: version `GLIBC_2.35' not found (required by /home/user/glibc-install/lib/libc.so.6)
/home/user/glibc-install/lib/ld-linux-x86-64.so.2: missing program name
Try '/home/user/glibc-install/lib/ld-linux-x86-64.so.2 --help' for more information.

I did not encounter any issues while installing glibc-2.41 locally but I don't know how to proceed.

I am happy for any answer, also just to boil down what the exact problem might be.

Thanks in advance!

Share Improve this question asked Mar 28 at 9:45 julian2000P 1676 bronze badges

For example setting the LD_LIBRARY_PATH gives the error is about which failing. Do not use which, what happens when you hardcode the path. did you install a new which executable? Try /home/user/glibc-install/lib/ld-linux-x86-64.so.2 which. – KamilCuk Commented Mar 28 at 9:52
I did not (knowingly) install a new which version. /home/user/glibc-install/lib/ld-linux-x86-64.so.2 which gives which: error while loading shared libraries: which: cannot open shared object file – julian2000P Commented Mar 28 at 9:57
Could you build it in a container? (SingularityCE?) – Ian Abbott Commented Mar 28 at 12:20
@IanAbbott thanks! I did not try building it in SingularityCE (I don't think that is even available on our cluster as of now). Would you recommend installing SingularityCE and try to build it there? If yes, what is the benefit compared to what I was doing? – julian2000P Commented Mar 28 at 13:16
SingularityCE was just the first one I found when googling. UC Irvine's cluster seems to have it pre-installed. Other container systems with a "rootless" mode are available, such as "podman". They allow users to create and manage their own containers without being root. The advantage is that stuff running in the container has its own libc, not tied to the host. – Ian Abbott Commented Mar 28 at 14:55

| Show 1 more comment

1 Answer 1

Sorted by: Reset to default 1

There is a much nicer way to install system programs like glibc and managing them: conda.

I recommend you to install miniconda into your HPC.

https://medium/codex/learn-conda-in-10-minutes-57e4be319495?sk=9092a1f5ec216070fdd1a405ceab95c1

And then do:

# create a new conda environment
conda create --name juliaup

# activate the conda environment
conda activate juliaup

# install glibc 2.17
conda install fridh::glibc

# install juliaup
conda install -c conda-fe juliaup

And then, using Juliaup install your desired Julia version.

And then install Cuda.jl

Conda takes care that packages installed locally into the conda environment are seen "first".

I recommend you to install juliaup into your conda environment first, because that is my experience with conda and Julia. juliaup doesn't create problems when installing into the conda environment. And from then on, you can handle with Juliaup any version of Julia within that juliaup conda environment.

https://blog.devgenius.io/how-to-installinstalling-julia-with-juliaup-in-a-conda-environment-1e43cb5c3a65?sk=8e6d8a498c3ff46ab48d8ada25b761ae

Upgrading CUDA

Our discussion lead to the new status - that the problem is also CUDA.

conda create --name new-cuda
conda activate new-cuda
conda install nvidia/label/cuda-12.8.1::cuda

# test which cuda version by:
nvcc --version

In the above juliaup environment, you could try, too nvcc --version. Maybe through which glibc you would find out which LD_LIBRARY_PATH to set.

科技改变生活-雨落星辰 - 所有的伟大,都源于一个勇敢的开始

linux - How to tell julia that it should use a locally installed glibc version for compilation? - Stack Overflow

1 Answer 1

Upgrading CUDA

与本文相关的文章

评论列表(0)