Installing neurodeskapp in an azure enclave

[marcelzwiers]

Hi there,

I'm not sure if it is possible what I'm trying to accomplish, but here is the situation. I recently got involved in setting up some applications in a secure computing environment (https://andrea-cloud.com/). I set up a Ubuntu22 client that I can connect to with my web-browser (using Bastion -> rdp). The default XFCE environment is very basic and I have to whitelist all connectivity. I managed to install docker and the neurodeskapp, but when I try to launch the app I get:

(sorry for the png, I couldn't get clipboard transfer to work)

Any suggestions? Do you think it should be doable what I'm trying to do?

[stebo85]

In an Ubuntu 22 environment, I would suggest not using docker to run Neurodesktop but rather using the Neurocommand Linux install method (https://www.neurodesk.org/docs/getting-started/neurocommand/linux/) or CVMFS if that's possible in your secure environment (https://www.neurodesk.org/docs/getting-started/neurocontainers/cvmfs/)

The error in the neurodeskapp you see most likely comes from Docker not working correctly. If you want to continue following this path, then the first step would be testing, if you can actually run the plain docker command:

docker volume create neurodesk-home &&
sudo docker run \
  --shm-size=1gb -it --privileged --user=root --name neurodesktop \
  -v ~/neurodesktop-storage:/neurodesktop-storage \
  --mount source=neurodesk-home,target=/home/jovyan \
  -e NB_UID="$(id -u)" -e NB_GID="$(id -g)" \
  -p 8888:8888 \
  -e NEURODESKTOP_VERSION=2024-01-12 vnmd/neurodesktop:2024-01-12

Alternatively, you could try podman

[marcelzwiers]

Indeed, I cannot run that command:

I really don't like this enclave tech :-(

I will go the neurocommand route (whenever I manage to get apptainer installed). Thanks for the help

[stebo85]

It looks like you don't have sudo privileges in this machine? Can you install anything via apt? If yes, apptainer should be easy to install:

sudo apt-get install -y software-properties-common
sudo add-apt-repository -y ppa:apptainer/ppa
sudo apt-get update
sudo apt-get install -y apptainer 

Neurodesk should work in such enclave environments - we just need to figure out what's possible and what's not.

[marcelzwiers]

Wait, I added a sudo and fixed some typos, now I got a bit further:

[stebo85]

It seems to have trouble with

-e NB_UID="$(id -u)" -e NB_GID="$(id -g)" \

you can remove that for now. Later it would be good to figure out why there is no gid

[marcelzwiers]

I removed -e NB_GID="$(id -g)" and got:

[stebo85]

Yes, this now is CVMFS which probably doesn't have fuse mounting privileges. So, I think your best bet is to install apptainer and go the HPC route.

[stebo85]

Can you try if this works? https://www.neurodesk.org/docs/getting-started/neurocontainers/cvmfs/ - maybe they didn't completely restrict fuse mounting. Otherwise, try a plain apptainer and download a container from here (https://www.neurodesk.org/docs/getting-started/neurocontainers/singularity/) to see if that route works

[marcelzwiers]

Yes, this now is CVMFS which probably doesn't have fuse mounting privileges. So, I think your best bet is to install apptainer and go the HPC route.

Could it be that I need to whitelist a domain first?

[stebo85]

That should come at a later step, but you can try: These are cvmfs domains we use: http://cvmfs1.neurodesk.org/ http://cvmfs2.neurodesk.org/ http://cvmfs3.neurodesk.org/ Thank you Steffen

[marcelzwiers]

Installing cvmfs fails, so I think I will try the neurocontainers (if I can get apptainer to install)

[stebo85]

you can't whitelist that IP 11.5.0.4 so it could pull the CVMFS repo package?

Is there a way we could get access to this environment so we can try a few things on our side?

[marcelzwiers]

Here's an update on things. I gave up on neurodesktop on Ubuntu and tried to get the environment modules system installed. It failed to work normally (I had to manually source /etc/profile to get it to work, but it seemed unavailable in subshell, or at least neurocommand was giving errors about module commands not found). In our enclave, there was also an old CentOS 7.9 system available, which I tried next. I started off with neurocommands and the good news is that I now got it to work, just like that. I don't understand why people are so fond of Ubuntu

[marcelzwiers]

The neurodesktop gives users a nice menu structure to startup a terminal with pre-loaded module, are there other benefits about using this desktop instead of the default XFCE that comes with the CentOS VM?

[marcelzwiers]

I tested a few neurocommands that worked fine, but when I tried RStudio I got this error:

[marcelzwiers]

FYI, if I re-install RStudio I get:

[marcelzwiers]

Btw, I've seen this module: command not found also in Ubuntu. I think it's because I need to run the install script using sudo, which starts it's own (non-profile) shell. I even use sudo -E env PATH=$PATH (also to get the conda python environment working), which is the only way for me to get to python 3.6 (I have sudo permission, but I am not allowed to become root)

[marcelzwiers]

Ok, it gets even more weird. I went back to Ubuntu and manually sourced the module profile (somehow this seems the only way) and set the modulepath (using module use, somehow this also seems the only way), and then I managed to install the RStudio neurocontainer. And guess what, the rstudio neurocommand just works

[stebo85]

interesting problem indeed. I unfortunately can't replicate these issues, so it's difficult to troubleshoot on my side. Can you describe in more detail what you did? Maybe the only way to replicate these issues is running it in the secure enclave you are in? Is there a way to get access to this environment?

[marcelzwiers]

Yes, I'll send you a message on Mattermost@brainhack with details

[marcelzwiers]

In the meantime I made some more progress on Ubuntu, because I found out (somewhere in a Stackoverflow comment) that there is a /usr/bin/add.modules file that I could run. Now I finally got the module system to work and this in turn makes the rest a lot easier... :-)

[marcelzwiers]

Ok, I learned another important thing. I believe the module system under Ubuntu is not initializing properly because in Ubuntu \bin\sh is a dash shell, whereas in other distros it is bash

[stebo85]

@marcelzwiers, so this instruction here does not work? https://www.neurodesk.org/docs/getting-started/neurocontainers/cvmfs/#use-of-containers-in-the-module-system

[stebo85]

Another problem was that apptainer wasn’t running with user namespaces and the setuid was interfering with rrstudio