I’m part of the Notre Dame Quark Net center, and we’re working on using Python to analyze some of the nanoAOD-like data from 2010, primarily on Windows computers. While Jupyter Notebooks work fine, we find ourselves missing some of the features in Spyder (part of Anaconda Navagator) like the variable inspection tool.
Is there something we can use within the docker container that reproduces that functionality? Alternatively, is there any way in Windows to get pieces of Anaconda Navigator like Spyder to open in the python docker container? We’d use it directly outside of the docker container, but we’ve not been able to install XRootD properly to read the ROOT files.
Thanks for any assistance!
Hi Jill,
This is a bit beyond me (maybe @katilp or @mccauley have more ideas), but it seems like it might be possible the other way around, e.g. connecting Spyder to the container as a remote kernel:
https://docs.spyder-ide.org/current/panes/ipythonconsole.html#connect-to-a-remote-kernel
But depending on what you actually use Spyder to do, even that might be overkill.
We usually recommend to users to mount a file directory to their container so that they can edit files with whatever tools their like outside the container, and then run python or whatever inside the container. So if the Spyder tools you’re interested in are useful in the script editing / creation process, then the shared folder concept might be enough.
Setting up that sharing looks like this (for Windows, this would be run in the WSL Ubuntu terminal):
export workpath=$PWD
mkdir cms_open_data_python
chmod -R 777 cms_open_data_python
docker run -it --name my_python -P -p 8888:8888 -v ${workpath}/cms_open_data_python:/code gitlab-registry.cern.ch/cms-cloud/python-vnc:python3.10.5
Regards,
Julie
Thanks for your response, Julie!
Looking at the link you attached, it doesn’t look like it will work well for the Windows machines we’re using for most of our analysis: we can’t get python to run from the command line. I don’t remember if we tried doing it from the docker container, but it was definitely a struggle earlier today. We can try it, but it looks like your second suggestion might be the winner anyway.
The sharing setup looks mainly like the general setup for the python docker. The first line setting up the workpath is new and it looks like instead of directing to a second port the docker run now directs to the home directory (PWD). Is that correct?
If we did this, do we just need to remove the existing python docker and re-run it to get the sharing set up? Or should we create a new docker? Maybe that’s up to us, but I wonder if you have any recommendations.
And thanks for calling @katilp and @mccauley in to this. Between the three of you, you’ve been a great help over the years getting us to work out how to do our (admittedly rudimentary) analyses.
~Jill
Hi,
Yes, using a variable for the path is not required, you can certainly type the full path to the shared folder when you create the docker container. The important part is creating the folder and doing chmod.
Making a new container would be the way to get the sharing working. Try the instructions for python + Exercise 1 + Exercise 2 in this new lesson page – that should show how the sharing works.
https://cms-opendata-workshop.github.io/workshop2024-lesson-docker/03-docker-for-cms-opendata.html
Julie
Hello again,
Thanks for the link to the new version of the docker instructions. We’ve been working off last year’s version where you didn’t have the sharing set up. I’ve been intending to read through the new workshop instructions but haven’t gotten to it yet!
Quick additional question: do you have to do the path export before you create the new folder and do the chmod? Context suggests not and suggests we can just remove the old container (I remember there were instructions on how to do it, so I’ll find them!) and create the new one in the pre-existing, chmodded folder. I probably want us to default to using the variable for the path both because it makes more sense to me and because it’s in the new tutorial. We tend to generally be a year behind in the tutorials . . .
Thanks yet again,
~Jill
Thanks for the clarification on the docker container. We’re trying to learn how docker works in general as we’re working through other things and the info that the container isn’t created in the folder is useful. I’m guessing that it links to the folder names you specify in the tutorial, then?
We have an additional question about the sharing. We can see the folders in the WSL2 environment. We can’t see them through Windows File Explorer from the Windows desktop. Should we be able to? We were hoping to be able to save to the cms_open_data_python folder from Spyder running on the desktop but haven’t been able to do so yet. If we won’t be able to do it at all, that’s fine as long as we know (cut and paste is a thing!). If we have to remove the dockers, docker images, and folders and restart the whole process, that’s fine as well.
To be specific, we had created the folders and used the identical chmod command previous to posting here and we removed and re-started the docker with the sharing as you gave directions for above.
As always, thanks!
Hi,
For me, folders in the WSL Ubuntu are in my file explorer, whether or not they’re related to docker. On the left side menu if I scroll all the way to the bottom I see the Linux penguin icon and it says “Linux”. Like this image:
Do you see that?
Julie
Yes, we do see that! Thanks! We honestly apparently hadn’t thought to scroll that far down to see the Linux penguin.
While Spyder doesn’t directly interface with the Linux folder, we were successfully able to get a file into the cms_open_data_python folder using Windows File Explorer. That means we’ll be able to get them out as well with the same kind of method.
Thanks again for being so patient with us and our oversights!