Support Globus 4.0

[forsyth2]

Summary

Objectives:

• Make any changes necessary to support Globus 4.0

Issue resolution:

• Closes Support Globus 4.0 #395

Select one: This pull request is...

• a bug fix: increment the patch version
• a small improvement: increment the minor version
• a new feature: increment the minor version
• an incompatible (non-backwards compatible) API change: increment the major version

Small Change

• To merge, I will use "Squash and merge". That is, this change should be a single commit.
• Logic: I have visually inspected the entire pull request myself.
• Pre-commit checks: All the pre-commits checks have passed.

[forsyth2]

Output log using Globus 4.0:

> ./globus_auth.bash test_globus4.0 chrysalis /home/ac.forsyth2/ez/zstash /home/ac.forsyth2/zstash_tests /global/homes/f/forsyth/zstash_tests /home/f/forsyth/zstash_tests /compyfs/fors729/zstash_tests
You may wish to clear your dst directories for a fresh start:
Chrysalis: rm -rf /home/ac.forsyth2/zstash_tests/test_globus_auth*
Perlmutter: rm -rf /global/homes/f/forsyth/zstash_tests/test_globus_auth*
Compy: rm -rf /compyfs/fors729/zstash_tests/test_globus_auth*
This won't work on HPSS, because -rf flags are unsupported:
NERSC HPSS: rm -rf /home/f/forsyth/zstash_tests/test_globus_auth*

It is therefore advisable to just set a unique_id to avoid directory conflicts.
Currently, unique_id=test_globus4.0
Is the unique_id correct? (y/n): y
Go to https://app.globus.org/file-manager?two_pane=true > For Collection, choose each of the following endpoints and, if needed, authenticate:
LCRC Improv DTN, NERSC Perlmutter, NERSC HPSS, pic#compy-dtn
Have you authenticated into all endpoints? (y/n): y
Primary tests: single authentication code tests for each endpoint
If a test hangs, check if https://app.globus.org/activity reports any errors on your transfers.
Testing transfer to LCRC Improv DTN ####################################
Reset Globus consents:
https://auth.globus.org/v2/web/consents > Globus Endpoint Performance Monitoring > rescind all
Have you revoked Globus consents? (y/n): y
Running test_single_auth_code
Exit codes: 0 -- success, 1 -- zstash failed, 2 -- grep check failed
##########################################################################################################
Testing: run1
For help, please see https://e3sm-project.github.io/zstash. Ask questions at https://github.com/E3SM-Project/zstash/discussions/categories/q-a.
INFO: /home/ac.forsyth2/.zstash.ini does NOT exist. This means we won't be able to read the local endpoint ID from it.
INFO: Token file /home/ac.forsyth2/.zstash_globus_tokens.json does NOT exist. This means we won't be able to load tokens from it.
INFO: Writing to empty /home/ac.forsyth2/.zstash.ini
INFO: Setting local_endpoint_id based on FQDN chrlogin2.lcrc.anl.gov: 15288284-7006-4041-ba1a-6b52501e49f1
INFO: No stored tokens found - starting authentication
Please go to this URL and login:
https://auth.globus.org/v2/oauth2/authorize?client_id=6c1629cf-446c-49e7-af95-323c6412397f&redirect_uri=https%3A%2F%2Fauth.globus.org%2Fv2%2Fweb%2Fauth-code&scope=urn%3Aglobus%3Aauth%3Ascope%3Atransfer.api.globus.org%3Aall%5B%2Ahttps%3A%2F%2Fauth.globus.org%2Fscopes%2F15288284-7006-4041-ba1a-6b52501e49f1%2Fdata_access+%2Ahttps%3A%2F%2Fauth.globus.org%2Fscopes%2F15288284-7006-4041-ba1a-6b52501e49f1%2Fdata_access%5D&state=_default&response_type=code&code_challenge=PW-1XMi1zRpbSNoYcgkbhEXEPdjjJh3SRXvodgr2UtY&code_challenge_method=S256&access_type=offline
Please enter the code you get after login here:
INFO: Tokens saved successfully
Traceback (most recent call last):
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/bin/zstash", line 7, in <module>
    sys.exit(main())
             ~~~~^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/main.py", line 65, in main
    create()
    ~~~~~~^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/create.py", line 60, in create
    globus_activate(hpss)
    ~~~~~~~~~~~~~~~^^^^^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/globus.py", line 51, in globus_activate
    r = transfer_client.endpoint_autoactivate(ep_id, if_expires_in=600)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'TransferClient' object has no attribute 'endpoint_autoactivate'
run1 completed successfully. Checking run1.log now.
##########################################################################################################
Testing: run2
For help, please see https://e3sm-project.github.io/zstash. Ask questions at https://github.com/E3SM-Project/zstash/discussions/categories/q-a.
INFO: /home/ac.forsyth2/.zstash.ini exists. We can try to read the local endpoint ID from it.
INFO: Token file /home/ac.forsyth2/.zstash_globus_tokens.json exists. We can try to load tokens from it.
INFO: Setting local_endpoint_id based on /home/ac.forsyth2/.zstash.ini: 
INFO: Setting local_endpoint_id based on FQDN chrlogin2.lcrc.anl.gov: 15288284-7006-4041-ba1a-6b52501e49f1
INFO: Found stored refresh token - using it
Traceback (most recent call last):
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/bin/zstash", line 7, in <module>
    sys.exit(main())
             ~~~~^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/main.py", line 65, in main
    create()
    ~~~~~~^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/create.py", line 60, in create
    globus_activate(hpss)
    ~~~~~~~~~~~~~~~^^^^^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/globus.py", line 51, in globus_activate
    r = transfer_client.endpoint_autoactivate(ep_id, if_expires_in=600)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'TransferClient' object has no attribute 'endpoint_autoactivate'
run2 completed successfully. Checking run2.log now.
For help, please see https://e3sm-project.github.io/zstash. Ask questions at https://github.com/E3SM-Project/zstash/discussions/categories/q-a.
INFO: 20251125_201056_181418: in hpss_transfer, prev_transfers is starting as []
INFO: Transferring file from HPSS: zstash/index.db
INFO: 20251125_201056_182749: DIVING: hpss calls globus_transfer(name=index.db)
INFO: 20251125_201056_182790: Entered globus_transfer() for name = index.db
INFO: /home/ac.forsyth2/.zstash.ini exists. We can try to read the local endpoint ID from it.
INFO: Token file /home/ac.forsyth2/.zstash_globus_tokens.json exists. We can try to load tokens from it.
INFO: Setting local_endpoint_id based on /home/ac.forsyth2/.zstash.ini: 
INFO: Setting local_endpoint_id based on FQDN chrlogin2.lcrc.anl.gov: 15288284-7006-4041-ba1a-6b52501e49f1
INFO: Found stored refresh token - using it
Traceback (most recent call last):
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/bin/zstash", line 7, in <module>
    sys.exit(main())
             ~~~~^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/main.py", line 75, in main
    ls()
    ~~^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/ls.py", line 34, in ls
    matches: List[FilesRow] = ls_database(args, cache)
                              ~~~~~~~~~~~^^^^^^^^^^^^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/ls.py", line 105, in ls_database
    hpss_get(hpss, get_db_filename(cache), cache)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/hpss.py", line 166, in hpss_get
    hpss_transfer(hpss, file_path, "get", cache, False)
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/hpss.py", line 111, in hpss_transfer
    globus_status = globus_transfer(
        endpoint, url_path, name, transfer_type, non_blocking
    )
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/globus.py", line 82, in globus_transfer
    globus_activate("globus://" + remote_ep)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/globus.py", line 51, in globus_activate
    r = transfer_client.endpoint_autoactivate(ep_id, if_expires_in=600)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'TransferClient' object has no attribute 'endpoint_autoactivate'
Expected grep 'file_empty.txt' not found in run1_ls.log. Test failed.

This matches the error noted in #395.

[forsyth2]

0f63f4c implements Claude's suggestion in #395, but it unfortunately doesn't seem to use a valid command:

> ./globus_auth.bash test_globus4.0_try2 chrysalis /home/ac.forsyth2/ez/zstash /home/ac.forsyth2/zstash_tests /global/homes/f/forsyth/zstash_tests /home/f/forsyth/zstash_tests /compyfs/fors729/zstash_tests
You may wish to clear your dst directories for a fresh start:
Chrysalis: rm -rf /home/ac.forsyth2/zstash_tests/test_globus_auth*
Perlmutter: rm -rf /global/homes/f/forsyth/zstash_tests/test_globus_auth*
Compy: rm -rf /compyfs/fors729/zstash_tests/test_globus_auth*
This won't work on HPSS, because -rf flags are unsupported:
NERSC HPSS: rm -rf /home/f/forsyth/zstash_tests/test_globus_auth*

It is therefore advisable to just set a unique_id to avoid directory conflicts.
Currently, unique_id=test_globus4.0_try2
Is the unique_id correct? (y/n): y
Go to https://app.globus.org/file-manager?two_pane=true > For Collection, choose each of the following endpoints and, if needed, authenticate:
LCRC Improv DTN, NERSC Perlmutter, NERSC HPSS, pic#compy-dtn
Have you authenticated into all endpoints? (y/n): y
Primary tests: single authentication code tests for each endpoint
If a test hangs, check if https://app.globus.org/activity reports any errors on your transfers.
Testing transfer to LCRC Improv DTN ####################################
Reset Globus consents:
https://auth.globus.org/v2/web/consents > Globus Endpoint Performance Monitoring > rescind all
Have you revoked Globus consents? (y/n): y
Running test_single_auth_code
Exit codes: 0 -- success, 1 -- zstash failed, 2 -- grep check failed
##########################################################################################################
Testing: run1
For help, please see https://e3sm-project.github.io/zstash. Ask questions at https://github.com/E3SM-Project/zstash/discussions/categories/q-a.
INFO: /home/ac.forsyth2/.zstash.ini does NOT exist. This means we won't be able to read the local endpoint ID from it.
INFO: Token file /home/ac.forsyth2/.zstash_globus_tokens.json does NOT exist. This means we won't be able to load tokens from it.
INFO: Writing to empty /home/ac.forsyth2/.zstash.ini
INFO: Setting local_endpoint_id based on FQDN chrlogin2.lcrc.anl.gov: 15288284-7006-4041-ba1a-6b52501e49f1
INFO: No stored tokens found - starting authentication
Please go to this URL and login:
https://auth.globus.org/v2/oauth2/authorize?client_id=6c1629cf-446c-49e7-af95-323c6412397f&redirect_uri=https%3A%2F%2Fauth.globus.org%2Fv2%2Fweb%2Fauth-code&scope=urn%3Aglobus%3Aauth%3Ascope%3Atransfer.api.globus.org%3Aall%5B%2Ahttps%3A%2F%2Fauth.globus.org%2Fscopes%2F15288284-7006-4041-ba1a-6b52501e49f1%2Fdata_access+%2Ahttps%3A%2F%2Fauth.globus.org%2Fscopes%2F15288284-7006-4041-ba1a-6b52501e49f1%2Fdata_access%5D&state=_default&response_type=code&code_challenge=QOPokFT_b9oS5DC3ERjzlWec341bJz7lNuYTPzIwTw8&code_challenge_method=S256&access_type=offline
Please enter the code you get after login here:
INFO: Tokens saved successfully
Traceback (most recent call last):
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/bin/zstash", line 7, in <module>
    sys.exit(main())
             ~~~~^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/main.py", line 65, in main
    create()
    ~~~~~~^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/create.py", line 60, in create
    globus_activate(hpss)
    ~~~~~~~~~~~~~~~^^^^^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/globus.py", line 51, in globus_activate
    r = transfer_client.endpoint_activate(ep_id, if_expires_in=600)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'TransferClient' object has no attribute 'endpoint_activate'. Did you mean: 'endpoint_acl_list'?
run1 completed successfully. Checking run1.log now.
##########################################################################################################
Testing: run2
For help, please see https://e3sm-project.github.io/zstash. Ask questions at https://github.com/E3SM-Project/zstash/discussions/categories/q-a.
INFO: /home/ac.forsyth2/.zstash.ini exists. We can try to read the local endpoint ID from it.
INFO: Token file /home/ac.forsyth2/.zstash_globus_tokens.json exists. We can try to load tokens from it.
INFO: Setting local_endpoint_id based on /home/ac.forsyth2/.zstash.ini: 
INFO: Setting local_endpoint_id based on FQDN chrlogin2.lcrc.anl.gov: 15288284-7006-4041-ba1a-6b52501e49f1
INFO: Found stored refresh token - using it
Traceback (most recent call last):
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/bin/zstash", line 7, in <module>
    sys.exit(main())
             ~~~~^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/main.py", line 65, in main
    create()
    ~~~~~~^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/create.py", line 60, in create
    globus_activate(hpss)
    ~~~~~~~~~~~~~~~^^^^^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/globus.py", line 51, in globus_activate
    r = transfer_client.endpoint_activate(ep_id, if_expires_in=600)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'TransferClient' object has no attribute 'endpoint_activate'. Did you mean: 'endpoint_acl_list'?
run2 completed successfully. Checking run2.log now.
For help, please see https://e3sm-project.github.io/zstash. Ask questions at https://github.com/E3SM-Project/zstash/discussions/categories/q-a.
INFO: 20251125_201659_572120: in hpss_transfer, prev_transfers is starting as []
INFO: Transferring file from HPSS: zstash/index.db
INFO: 20251125_201659_572731: DIVING: hpss calls globus_transfer(name=index.db)
INFO: 20251125_201659_572773: Entered globus_transfer() for name = index.db
INFO: /home/ac.forsyth2/.zstash.ini exists. We can try to read the local endpoint ID from it.
INFO: Token file /home/ac.forsyth2/.zstash_globus_tokens.json exists. We can try to load tokens from it.
INFO: Setting local_endpoint_id based on /home/ac.forsyth2/.zstash.ini: 
INFO: Setting local_endpoint_id based on FQDN chrlogin2.lcrc.anl.gov: 15288284-7006-4041-ba1a-6b52501e49f1
INFO: Found stored refresh token - using it
Traceback (most recent call last):
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/bin/zstash", line 7, in <module>
    sys.exit(main())
             ~~~~^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/main.py", line 75, in main
    ls()
    ~~^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/ls.py", line 34, in ls
    matches: List[FilesRow] = ls_database(args, cache)
                              ~~~~~~~~~~~^^^^^^^^^^^^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/ls.py", line 105, in ls_database
    hpss_get(hpss, get_db_filename(cache), cache)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/hpss.py", line 166, in hpss_get
    hpss_transfer(hpss, file_path, "get", cache, False)
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/hpss.py", line 111, in hpss_transfer
    globus_status = globus_transfer(
        endpoint, url_path, name, transfer_type, non_blocking
    )
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/globus.py", line 82, in globus_transfer
    globus_activate("globus://" + remote_ep)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zstash-globus-4.0/lib/python3.13/site-packages/zstash/globus.py", line 51, in globus_activate
    r = transfer_client.endpoint_activate(ep_id, if_expires_in=600)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'TransferClient' object has no attribute 'endpoint_activate'. Did you mean: 'endpoint_acl_list'?
Expected grep 'file_empty.txt' not found in run1_ls.log. Test failed.

[forsyth2]

The latest commit (9059672) allows us to get through the Chrysalis test of ./globus_auth.bash, which suggests Globus 4.0 is working alright now. (Will need to run full test suite for a more thorough confirmation)

[forsyth2]

This was set to ==4.0 to specifically test Globus 4.0, but before merging, we should change it to simply drop the <4.0 constraint.

[forsyth2]

Well actually, we'd need to test if this change works using a version below 4.0. I think it's a breaking change, so it probably will not work. That is, we'd likely want to constrain this as globus-sdk >= 4.0