IRSA-7344: Download Dialog Improvements and Bug fixes

[kpuriIpac]

Ticket:

• See IFE PR
• The code changes are small, but aim to fix all existing bugs and make important improvements to the download dialog
• Bug fixes:
  • The datalink table (that you get to on Euclid, for instance, by clicking on Show table of all Euclid products) was displaying two download buttons
  • I think this happened because our code to detect ObsCore like tables recently got better, and recognized datalink tables as obscore-ish tables and added a download button to them. Now, adding a check in the DefaultSearchActions' showDatalinkTable to ensure it doesn't already have a download button fixes this.
• If no checkboxes are selected in the download dialog, and user attempts to click on the download button, they will now be prompted with an error message asking them to make at least one selection.
• The cutout size changes were not being respected in some instances in Spherex and Euclid (for example, if you did a TAP search or a Euclid catalogs search, the cutout size updates would not be reflected in the download dialog).
• Euclid & Spherex now use Generate Download Script throughout (even for TAP / Catalog searches), except for Spherex's Spectrophotometry tool.

Note: The default mode for checkboxes in our download dialogs is that all products will be unselected. If you'd like me to change it to rather make everything selected by default, let me know.

Testing:

• Firefly, DCE, Euclid and Spherex
• The main purpose of testing below is to try and see if you can find any bugs. Hence the instructions to make sure we cover all our bases.
• Firefly: Use CADC IVOA search to find tables where the Prepare Download button shows up. If cutouts exist, attempt to change the cutout size and ensure that you see the updated cutout size in the download dialogs when you select the cutout checkbox.
• DCE: Regression testing to ensure Prepare Download works as expected.
• Euclid & Spherex:
  • Do an Image, Inspect Objects and TAP (CADC ivoa) and Euclid Catalogs searches on Euclid.
    • For all of these, try changing the cutout size and ensure it's reflected in the download dialog when cutout is selected.
    • Ensure you only see the Generate Download Button everywhere, including on TAP and Euclid Catalog results.
  • Do a Spectral Image, Spectrophotometry, and TAP (CADC ivoa) searches on Spherex
    • Repeat the steps above.
    • The only difference is that for the Spectrophotometry search, you should see a Prepare Download button
• For any of the search results above, click on the Show table of all Euclid products (or spherex) to see the products table. Ensure you only see one download (Generate Download Script) button.
• And for some of the searches above, attempt to actually to do package or script download to ensure it works as expected.

[lrebull]

DCE is broken for me immediately:

[lrebull]

Firefly build -> SIAv2 searches -> CADC ObsCore -> m16, IRSA -> picked four rows, asked it for all data, including 10 arcmin cutouts, got this in the README, which makes it look like the cutouts failed, but I got cutouts with my download which appear to be correct..?:
Successfully packaged 16 files: 5,599,720 B
Errors were encountered when packaging 4 files:
IRIS-IRAS-f189h000/asynccutout
IRIS-IRAS-f189h000-1/asynccutout
IRIS-IRAS-f189h000-2/asynccutout
IRIS-IRAS-f189h000-3/asynccutout

[lrebull]

cutout is not a choice in this spherex build (see there is no icon on the right above the image), so i cannot change it to test what it does.

[lrebull]

For SPHEREx:

• For the calibration files, I'm getting 31 copies of each of them. Surely we can do a better job than that from the packaging side -- if we're asking x times for the same file, just package one copy of them.
• I ran the shell script it sent me all the way through and I'm not sure I've ever done that before. I have three copies of the main directory and counting, which I did not expect - I expected SPHERExLVFSurveyData to contain one subdir for each of the rows i selected for download, but if it's giving me one SPHERExLVFSurveyData directory for each row, I'll live with that. The first two directories each have 324 files, most of which are 31 copies each of 9 calibration files. The third directory seems to not contain any calibration files, only science files? (900 and counting!! and only 2 actual cutouts) but see next item.
• I'm getting errors, e.g.:

downloading https://irsadev.ipac.caltech.edu/ibe/cutout?ra=184.578987982502&dec=-39.86101153510683&size=0.01&path=spherex/qr2/level2/2025W22_1B/l2b-v20-2025-250/3/level2_2025W22_1B_0093_3D3_spx_l2b-v20-2025-250.fits ...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 54 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (22) The requested URL returned error: 422
ERROR: failed to download https://irsadev.ipac.caltech.edu/ibe/cutout?ra=184.578987982502&dec=-39.86101153510683&size=0.01&path=spherex/qr2/level2/2025W22_1B/l2b-v20-2025-250/3/level2_2025W22_1B_0093_3D3_spx_l2b-v20-2025-250.fits

[lrebull]

tried SPHEREx-> TAP -> CADC ObsCore -> prepare download -> Curl script, got these errors:

downloading https://ws.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/raven/files/cadc:MACHO/cal123583b.fits.fz ...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 17.4M 100 17.4M 0 0 2835k 0 0:00:06 0:00:06 --:--:-- 2968k
Saved as /Users/rebull/Downloads/MACHO-MACHO-123583/cal123583b.fits.fz
downloading https://ws.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/caom2ops/sync?CIRCLE=184.10460290953273+-37.94573006670811+0.01&ID=cadc:MACHO/cal123583b.fits.fz ...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 32 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (22) The requested URL returned error: 400
ERROR: failed to download https://ws.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/caom2ops/sync?CIRCLE=184.10460290953273+-37.94573006670811+0.01&ID=cadc:MACHO/cal123583b.fits.fz
downloading https://ws.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/caom2ops/async?CIRCLE=184.10460290953273+-37.94573006670811+0.01&ID=cadc:MACHO/cal123583b.fits.fz ...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 36 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (22) The requested URL returned error: 403
ERROR: failed to download https://ws.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/caom2ops/async?CIRCLE=184.10460290953273+-37.94573006670811+0.01&ID=cadc:MACHO/cal123583b.fits.fz
[staryu:~/Downloads] rebull%

[lrebull]

I just killed the SPHEREx download script because it was in the process of downloading 20,000 files. That doesn't seem reasonable for selecting three rows from an image search. Something is really, really wrong here.

[lrebull]

SpectralImages-curl.sh
here is the download script i got for the SPHEREx search

[lrebull]

Euclid seems to be working ok; it just takes a long time because the images are large. . .