Search code examples
wgetcdo-climate

ESGF `wget` scripts incorrectly generating; linking to a random unrelated file


About half the time when I click on WGET script following a CMIP6 data search on the ESGF (LLNL node), I get a wget script that only points to one, unrelated file. It's always the same one, too. Here's the relevant line that shows up in each wget file:

download_files="$(cat <<EOF--dataset.file.url.chksum_type.chksum
'famipc5_ne120_v0.3_00001_01_198001_198401_climo.nc' 'http://esgf.anl.gov/thredds/fileServer/esg_dataroot/ACME/climo/amip/v0_3/atm/mon/native/ne120/ens1/famipc5_ne120_v0.3_00001_01_198001_198401_climo.nc' 'SHA256' 'e5040c5df9d080437418943f02a41e84712dbe1c4a69982447712d7c7334241d'
EOF--dataset.file.url.chksum_type.chksum
)"

This happens with a wide variety of datasets. Here's one file where that happens, for example: CMIP6.CMIP.CCCma.CanESM5.amip.r1i1p1f1.day.pr.gn

I've been searching for a reason, so far without success. A workaround is to hit the "download HTML" button 1000 times for each individual needed file instead (or set up a Globus endpoint for the files where that's possible), but it's very inconvenient and doesn't provide the functionality of a bash script.

Does anyone know what may be causing this? Is there some sort of limit to how many wget scripts an ESGF user can generate per day and these are downloaded as placeholders afterwards instead?

Grateful for any insight!

PS: I apologize for the cdo tag; I know this isn't a cdo problem, but it's hard to find relevant tags for this, and I figured that community may know what's up.


Solution

  • Turns out this is a browser issue. Repeating the search with chrome fixed it.

    (Also that stack overflow may have not been the right venue to post this question; but I want this to be searchable somewhere at least)