Even though I solved this problem, I still want to post it here in case someone else faces the same problem.

I just did a completely fresh install of an archlinux based distro on a newly built PC, and on this completely fresh system, I installed miniconda3 via the instructions on the anaconda website. Then I installed the vscode from the code repository and the popular Python and Jupyter extensions inside of vscode.

I made a new conda environment where I installed the jupyter package.

Because I installed everything on a completely fresh system, with no superfluous or conflicting packages or any other way of breaking any dependencies/ missing any configs etc..., this is (as I believe) as "out of the box" as it gets. So I was very surprised that the jupyter notebooks inside of vscode did not work; I was not able to choose a kernel to run my code.

When selecting the "Select Kernel" in the top right, I was not able to select anything, it was just endlessly loading. Searching for similar problems on Google yielded a number of results, suggesting to use the "Select Interpreter to Start Jupyter Server" option, but this did not work for me.