Unable to download simulated data files from CERN opendata

Hello,
When I try to directly download any data file from the CERN opendata website, I am getting a 504 Gateway Time-out The server didn’t respond in time error. I tried to download the files from my command prompt using the command: cernopendata-client download-files --recid (record number of file), which is working for some files such as the Jpsi data. However, it isn’t working for datasets like *Simulated dataset ttGG_8TeV-whizard-pythia6 in AODSIM format for 2012 collision data, Simulated dataset TTGJets_8TeV-madgraph in AODSIM format for 2012 collision data, Simulated dataset tGamma_8TeV_madgraph in AODSIM format for 2012 collision data. What should I do to fix this?
Thanks in advance

2 Likes

Hi! It looks there a problem with downloads from the portal in general. As the service is maintained by the CERN IT, we need to wait that the lab opens and people who can fix can have a look.

Thanks for your patience!

Hi @aarushi-tiwari

The download troubles that you observed were intermittent and I hope that you managed to download all the files of interest by now. (Apologies for my late reply!)

I’d like to share two news on this topic:

  1. We have deployed an improvement to the file serving infrastructure earlier this week so that you should be seeing much less gateway timeout problems than in December.

  2. If you do encounter additional download troubles, you may be perhaps interested in using our command-line client https://cernopendata-client.readthedocs.io/ that automatises downloads of large datasets with file checksum verifications etc.

Most notably, the client allows you to easily use the XRootD protocol for downloads, which is usually more robust than the HTTPS protocol due to larger available bandwidth.