I have been working on the Docker pre-exercises for the CMS-OPENDATA Workshop and I’ve come across a software issue I believe. On section 4 (Test and validate) I always obtain an error. I have triple checked and I strongly believe I am doing the steps correctly, and it has to do with my operating system. I am using MacOS-Monterey. System: MacBook Pro with M1 Pro chip.
I paste in continuation my terminal results:
[04:34:37] cmsusr@2bbbc365f437 ~/CMSSW_5_3_32/src $ cmsenv
[04:35:08] cmsusr@2bbbc365f437 ~/CMSSW_5_3_32/src $ mkdir Demo
[04:35:14] cmsusr@2bbbc365f437 ~/CMSSW_5_3_32/src $ cd Demo
[04:35:17] cmsusr@2bbbc365f437 ~/CMSSW_5_3_32/src/Demo $ mkedanlzr DemoAnalyzer
I: using skeleton: /opt/cms/slc6_amd64_gcc472/cms/cmssw/CMSSW_5_3_32/bin/slc6_amd64_gcc472/mkTemplates/EDAnalyzer/BuildFile.temp
I: authors name is: , determined by the gcos entry
I: creating file: DemoAnalyzer/BuildFile.xml
I: using skeleton: /opt/cms/slc6_amd64_gcc472/cms/cmssw/CMSSW_5_3_32/bin/slc6_amd64_gcc472/mkTemplates/EDAnalyzer/ConfFile_cfg.py
I: authors name is: , determined by the gcos entry
I: creating file: DemoAnalyzer/demoanalyzer_cfg.py
I: using skeleton: /opt/cms/slc6_amd64_gcc472/cms/cmssw/CMSSW_5_3_32/bin/slc6_amd64_gcc472/mkTemplates/EDAnalyzer/edanalyzer.cc
I: authors name is: , determined by the gcos entry
I: creating file: DemoAnalyzer/src/DemoAnalyzer.cc
I: using skeleton: /opt/cms/slc6_amd64_gcc472/cms/cmssw/CMSSW_5_3_32/bin/slc6_amd64_gcc472/mkTemplates/EDAnalyzer/CfiFile_cfi.py
I: authors name is: , determined by the gcos entry
I: creating file: DemoAnalyzer/python/demoanalyzer_cfi.py
[04:35:37] cmsusr@2bbbc365f437 ~/CMSSW_5_3_32/src/Demo $ cd …
[04:35:54] cmsusr@2bbbc365f437 ~/CMSSW_5_3_32/src $ cd …
[04:36:03] cmsusr@2bbbc365f437 ~/CMSSW_5_3_32 $ scram arch
slc6_amd64_gcc472
[04:39:38] cmsusr@2bbbc365f437 ~/CMSSW_5_3_32 $ scram b
****WARNING: No need to export library once you have declared your library as plugin. Please cleanup src/Demo/DemoAnalyzer/BuildFile by removing the section.
/bin/sh: line 5: 942 Killed config/SCRAM/updateToolMK.pl slc6_amd64_gcc472 > .SCRAM/slc6_amd64_gcc472/MakeData/Tools.mk.tmp
gmake: *** [.SCRAM/slc6_amd64_gcc472/MakeData/Tools.mk] Error 137
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2
As you can see I get 2 make errors that in continuation prevent me from executing my cmsRun as follows:
[04:47:02] cmsusr@2bbbc365f437 ~/CMSSW_5_3_32/src $ nano Demo/DemoAnalyzer/demoanalyzer_cfg.py
[04:48:11] cmsusr@2bbbc365f437 ~/CMSSW_5_3_32/src $ cmsRun Demo/DemoAnalyzer/demoanalyzer_cfg.py
220405 04:48:23 1465 Xrd: XrdClientConn: Error resolving this host’s domain name.
220405 04:48:26 1465 secgsi_InitProxy: cannot access private key file: /home/cmsusr/.globus/userkey.pem
220405 04:48:26 1465 Xrd: CheckErrorStatus: Server [eospublic.cern.ch] declared: (error code: 3005)
05-Apr-2022 04:48:26 CEST Initiating request to open file root://eospublic.cern.ch//eos/opendata/cms/Run2012B/DoubleMuParked/AOD/22Jan2013-v1/10000/1EC938EF-ABEC-E211-94E0-90E6BA442F24.root
05-Apr-2022 04:48:41 CEST Successfully opened file root://eospublic.cern.ch//eos/opendata/cms/Run2012B/DoubleMuParked/AOD/22Jan2013-v1/10000/1EC938EF-ABEC-E211-94E0-90E6BA442F24.root
Assertion failed: p_rcu_reader->depth != 0 (/qemu/include/qemu/rcu.h: rcu_read_unlock: 101)
ldd: exited with unknown exit code (134)
05-Apr-2022 04:49:03 CEST Closed file root://eospublic.cern.ch//eos/opendata/cms/Run2012B/DoubleMuParked/AOD/22Jan2013-v1/10000/1EC938EF-ABEC-E211-94E0-90E6BA442F24.root
----- Begin Fatal Exception 05-Apr-2022 04:49:03 CEST-----------------------
An exception of category ‘PluginNotFound’ occurred while
[0] Constructing the EventProcessor
Exception Message:
Unable to find plugin ‘DemoAnalyzer’. Please check spelling of name.
----- End Fatal Exception -------------------------------------------------
[04:49:04] cmsusr@2bbbc365f437 ~/CMSSW_5_3_32/src $
I have been looking over this a couple of days and have no idea what to do. I look foreword to your response and thank you!
thanks for reporting. The first error message you receive is Error 137 which usually indicates that it is out of memory. Can you try running docker stats in your host system terminal?
In addition, I’m not familiar with MacOS but I see that docker desktop comes in two versions. Do you have the one corresponding to the M1 chip installed?
The container stats as such look similar to what I see (apart from 2 in column PIDS where I have 1).
This is then probably related to the gcc compiler in the container which is used for compiling CMSSW. It has been reported that the compilation fails on M1 MacOS in the CMS open data containers and this indeed happens in your case. It works on other operating systems and, unfortunately, we do not yet have a solution for this particular case.
We’ll investigate and get back to you if we find a solution.
I was able to compile and run test.c! I will copy a section of my terminal in the following:
root@44370ed447ae:/# ls
bin boot dev etc home lib lib32 lib64 libx32 media mnt opt proc root run sbin srv sys test.c tmp usr var
root@44370ed447ae:/# gcc test.c -o test
root@44370ed447ae:/# ./test
Hello
root@44370ed447ae:/#
Please feel free to ask me anything, I’d be happy to help you help me.
Ok so I just ran the command, and it all worked except the apt update, I believe it is because of the amd architecture. I will add a copy of the procedure in continuation:
➜ ~ docker start -i sweet_bardeen
root@02a570f3d9c0:/# apt update
bash: apt: command not found
root@02a570f3d9c0:/# apt install gcc
bash: apt: command not found
root@02a570f3d9c0:/# cat > test.c
#include<stdio.h>
int main()
{
printf(“Hello\n”);
return 0;
}
root@02a570f3d9c0:/# gcc test.c -o test
root@02a570f3d9c0:/# ./test
Hello
root@02a570f3d9c0:/#
Thanks! The gcc compiler is preinstalled in the gcc container so indeed no need for the apt ... commands to install it. Would you also do
gcc --version
in that same container to be sure of the version (I do not see from your output whether your container is from the latest gcc image or from gcc:4.9.3)?
➜ ~ docker start -i sweet_bardeen
root@02a570f3d9c0:/# gcc --version
gcc (GCC) 4.9.3
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Looking for the solution of the error that Alex mentioned posts ago I found this forum.
I have the same issue as @alextintin007, the same error.
I run the following tutorial on my MacBook M1 Pro https://cms-opendata-workshop.github.io/workshop2021-lesson-docker/
and at the part Test and validate it says that we have to run the python script. I modified it with the text that says and it throw out the error.
It seems that @kati is working on it. Thanks a lot! @alextintin007 have you find the solution? Thanks for creating this forum!
Error 137 is an indication of the process being killed since there’s no memory left. I got this to work by increasing the memory available to Docker in the settings (I set it to 12 GB) and I also increased the swap space to 3 GB. However, this is not a very viable approach, since in principle the compilation of the software should only require memory at the order of MB…
I’d recommend you look into using a remote linux host. This works for Docker and Podman. Have a look at How to deploy on remote Docker hosts with docker-compose - Docker and How to Use a Remote Docker Server to Speed Up Your Workflow | DigitalOcean to get an idea.
Hi there,
Is increíble how just changing it has executed the python script. Thanks @clange.
When I run the script shows me this:
process = cms.Process("Demo")
process.load("FWCore.MessageService.MessageLogger_cfi")
process.maxEvents = cms.untracked.PSet( input = cms.untracked.int32(10) )
process.source = cms.Source("PoolSource",
# replace 'myfile.root' with the source file you want to use
fileNames = cms.untracked.vstring(
'root://eospublic.cern.ch//eos/opendata/cms/Run2012B/DoubleMuParked/AOD/22Jan2013-v1/10000/1EC938EF-ABEC-E211-94E0-90E6BA442F24.root'
)
)
process.demo = cms.EDAnalyzer('DemoAnalyzer'
)
process.p = cms.Path(process.demo)
[05:18:27] cmsusr@47516f27bfba ~/CMSSW_5_3_32/src/Demo/DemoAnalyzer $ ls
BuildFile.xml demoanalyzer_cfg.py doc interface python src test
[05:18:40] cmsusr@47516f27bfba ~/CMSSW_5_3_32/src/Demo/DemoAnalyzer $ cmsRun demoanalyzer_cfg.py
220409 05:19:00 2006 Xrd: XrdClientConn: Error resolving this host's domain name.
220409 05:19:01 2006 secgsi_InitProxy: cannot access private key file: /home/cmsusr/.globus/userkey.pem
220409 05:19:02 2006 Xrd: CheckErrorStatus: Server [eospublic.cern.ch] declared: (error code: 3005)
09-Apr-2022 05:19:02 CEST Initiating request to open file root://eospublic.cern.ch//eos/opendata/cms/Run2012B/DoubleMuParked/AOD/22Jan2013-v1/10000/1EC938EF-ABEC-E211-94E0-90E6BA442F24.root
09-Apr-2022 05:19:17 CEST Successfully opened file root://eospublic.cern.ch//eos/opendata/cms/Run2012B/DoubleMuParked/AOD/22Jan2013-v1/10000/1EC938EF-ABEC-E211-94E0-90E6BA442F24.root
Assertion failed: p_rcu_reader->depth != 0 (/qemu/include/qemu/rcu.h: rcu_read_unlock: 101)
ldd: exited with unknown exit code (134)
Begin processing the 1st record. Run 195013, Event 24425389, LumiSection 66 at 09-Apr-2022 05:19:35.117 CEST
Begin processing the 2nd record. Run 195013, Event 24546773, LumiSection 66 at 09-Apr-2022 05:19:35.132 CEST
Begin processing the 3rd record. Run 195013, Event 24679037, LumiSection 66 at 09-Apr-2022 05:19:35.134 CEST
Begin processing the 4th record. Run 195013, Event 24839453, LumiSection 66 at 09-Apr-2022 05:19:35.136 CEST
Begin processing the 5th record. Run 195013, Event 24894477, LumiSection 66 at 09-Apr-2022 05:19:35.138 CEST
Begin processing the 6th record. Run 195013, Event 24980717, LumiSection 66 at 09-Apr-2022 05:19:35.139 CEST
Begin processing the 7th record. Run 195013, Event 25112869, LumiSection 66 at 09-Apr-2022 05:19:35.141 CEST
Begin processing the 8th record. Run 195013, Event 25484261, LumiSection 66 at 09-Apr-2022 05:19:35.142 CEST
Begin processing the 9th record. Run 195013, Event 25702821, LumiSection 66 at 09-Apr-2022 05:19:35.143 CEST
Begin processing the 10th record. Run 195013, Event 25961949, LumiSection 66 at 09-Apr-2022 05:19:35.145 CEST
09-Apr-2022 05:19:35 CEST Closed file root://eospublic.cern.ch//eos/opendata/cms/Run2012B/DoubleMuParked/AOD/22Jan2013-v1/10000/1EC938EF-ABEC-E211-94E0-90E6BA442F24.root
=============================================
MessageLogger Summary
type category sev module subroutine count total
---- -------------------- -- ---------------- ---------------- ----- -----
1 fileAction -s file_close 1 1
2 fileAction -s file_open 2 2
type category Examples: run/evt run/evt run/evt
---- -------------------- ---------------- ---------------- ----------------
1 fileAction PostEndRun
2 fileAction pre-events pre-events
Severity # Occurrences Total Occurrences
-------- ------------- -----------------
System 3 3
[05:19:35] cmsusr@47516f27bfba ~/CMSSW_5_3_32/src/Demo/DemoAnalyzer $
It seems that is successfully executed but there is reaming the 134 error. Is kind of weird. What could be?
You’re right! On all the pre-exercises I’ve gotten the same error but it never got in my way of my data output… well till the [Trigger essentials (Basic physics objects)]. So a solution to the error to keep going will be needed. This is my output:
10-Apr-2022 01:43:09 CEST Initiating request to open file root://eospublic.cern.ch//eos/opendata/cms/Run2012B/TauPlusX/AOD/22Jan2013-v1/20000/0040CF04-8E74-E211-AD0C-00266CFFA344.root
10-Apr-2022 01:43:16 CEST Successfully opened file root://eospublic.cern.ch//eos/opendata/cms/Run2012B/TauPlusX/AOD/22Jan2013-v1/20000/0040CF04-8E74-E211-AD0C-00266CFFA344.root
Assertion failed: p_rcu_reader->depth != 0 (/qemu/include/qemu/rcu.h: rcu_read_unlock: 101)
ldd: exited with unknown exit code (134)
10-Apr-2022 01:43:43 CEST Closed file root://eospublic.cern.ch//eos/opendata/cms/Run2012B/TauPlusX/AOD/22Jan2013-v1/20000/0040CF04-8E74-E211-AD0C-00266CFFA344.root
----- Begin Fatal Exception 10-Apr-2022 01:43:43 CEST-----------------------
An exception of category ‘PluginLibraryLoadError’ occurred while
[0] Constructing the EventProcessor
Exception Message:
unable to load /home/cmsusr/CMSSW_5_3_32/lib/slc6_amd64_gcc472/pluginDemoDemoAnalyzer.so because /home/cmsusr/CMSSW_5_3_32/lib/slc6_amd64_gcc472/pluginDemoDemoAnalyzer.so: cannot open shared object file: No such file or directory
----- End Fatal Exception -------------------------------------------------