Error in Pulling Docker Image using Singularity

CMS Containers
1

I am trying to get myself familiar with operations related to our cluster, so I tried using singularity there. I am having issues with pulling a specific docker image into singularity in the cluster. I have done the following steps:

cd /scratch/$USER
mkdir -p singularity/{cache,temp}
mkdir my_containers
module load singularity/3.7.2
export SINGULARITY_CACHEDIR=/scratch/$USER/singularity/cache
export SINGULARITY_TMPDIR=/scratch/$USER/singularity/temp
cd my_containers
singularity pull ./cms7poet.sif docker://cmsopendata/cmssw_7_6_7-slc6_amd64_gcc493

After the conversion of blob OCI to SIF, specifically saying

Writing manifest to image destination
Storing signatures
... warn xattr{mnt}
... warn xattr{lib}
... warn rootless{rootfs/}
...

it then throws a bunch of warnings and at the end of the day, returns the error

While making image from oci registry: error fetching image to cache: while building SIF from layers: packer failed to pack: while unpacking tmpfs: error unpacking rootfs: unpack layer: unpack entry: var/lib/yum/yumdb/m/d1e78f5adcafeb6c7d05cf71d17fb5196f487db5-mesa-libgbm-11.0.7-4.el6-x86_64/checksum_type: link: unpriv.link: unpriv.wrap target: too many links

I tried adding chmod -R 777 [directory] from the very beginning, but it did not work. I also tried googling first the issue, but never got a particular answer for it. Does anyone know about how to get around this kind of error? The available version of singularity that I am using is version 3.7.2, and the docker image is from CERN.

2

Hello!

Thanks for your message!
We do not have much experience yet using the CMS OD docker images with apptainer/singularity but we were able to do so in the recent workshop session in January (Analyzing Open Data at Scale)

Pulling the image took very long (and gave a long list of warning messages) but it finally succeeded. Does your SINGULARITY_CACHEDIR have enough space?

If you have the possibility of having the cvmfs file system mounted on your cluster, there might be more convenient options than converting the docker image.

Best, Kati

3

Hi! Thank you so much for your response. I think there is an option of having cvmfs mounted in the cluster. Can you share some resources that I read about tutorials on getting cmssw setup on it? Thank you!