datasets summary virus genome taxon 10239 fails on gateway errors

[jo-delia-faraguna]

Before opening an issue, please:

• Make sure you are using the latest version using datasets --version
• Review our documentation

Describe the bug
Data report request for entire 10239 virus taxonomic tree fails for the virus data package.

For context, we maintain a virus database built here by first enumerating viral assembly accessions, and then downloading and rehydrating them in chunks. We'd like to switch to using the virus data package, since it seems like there are no new non-Flu assemblies as of early 2025.

Indicate what operating system you're using
Amazon Linux 2023, datasets version: 18.33.1

To Reproduce

# fails
NCBI_API_KEY=<redacted> datasets summary virus genome taxon 10239 --complete-only --released-after 2024-01-01 --debug > fail.log 2>&1
# succeeds
NCBI_API_KEY=<redacted> datasets summary genome taxon 10239

Expected behavior
I would expect to return the data report for all virus metadata.

Debug log

2026/07/23 20:08:18 
POST /datasets/v2/taxonomy/taxon_suggest HTTP/1.1
Host: api.ncbi.nlm.nih.gov
User-Agent: OpenAPI-Generator/1.0.0/go
Content-Length: 128
Accept: application/json
Api-Key: c442c934c6f712a32c4f4ce8c75dd652f108
Content-Type: application/json
Ncbi-Phid: 82105ECBAE67541C2C9D5A8C
X-Datasets-Client: datasets-cli
X-Datasets-Client-Arch: amd64
X-Datasets-Client-Cmd: summary virus genome taxon 10239 --complete-only --released-after 2024-01-01 --debug
X-Datasets-Client-Os: linux
X-Datasets-Client-Version: 18.33.1
Accept-Encoding: gzip

{"exact_match":true,"tax_rank_filter":"higher_taxon","taxon_query":"10239","taxon_resource_filter":"TAXON_RESOURCE_FILTER_ALL"}

2026/07/23 20:08:18 
HTTP/2.0 200 OK
Access-Control-Expose-Headers: X-RateLimit-Limit,X-RateLimit-Remaining
Alt-Svc: clear
Content-Security-Policy: upgrade-insecure-requests
Content-Type: application/json
Date: Thu, 23 Jul 2026 20:08:18 GMT
Grpc-Metadata-Via: h2 linkerd
Ncbi-Phid: 82105ECBAE67541C2C9D5A8C.1.1.1
Server: Finatra
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Via: 1.1 google
X-Datasets-Version: 18.33.1
X-Ratelimit-Limit: 10
X-Ratelimit-Remaining: 9
X-Ua-Compatible: IE=Edge
X-Xss-Protection: 1; mode=block


2026/07/23 20:08:18 
POST /datasets/v2/taxonomy/dataset_report HTTP/1.1
Host: api.ncbi.nlm.nih.gov
User-Agent: OpenAPI-Generator/1.0.0/go
Content-Length: 51
Accept: application/json
Api-Key: c442c934c6f712a32c4f4ce8c75dd652f108
Content-Type: application/json
Ncbi-Phid: 82105ECBAE67541C2C9D5A8C
X-Datasets-Client: datasets-cli
X-Datasets-Client-Arch: amd64
X-Datasets-Client-Cmd: summary virus genome taxon 10239 --complete-only --released-after 2024-01-01 --debug
X-Datasets-Client-Os: linux
X-Datasets-Client-Version: 18.33.1
Accept-Encoding: gzip

{"returned_content":"METADATA","taxons":["10239"]}

2026/07/23 20:08:18 
HTTP/2.0 200 OK
Access-Control-Expose-Headers: X-RateLimit-Limit,X-RateLimit-Remaining
Alt-Svc: clear
Content-Security-Policy: upgrade-insecure-requests
Content-Type: application/json
Date: Thu, 23 Jul 2026 20:08:18 GMT
Grpc-Metadata-Via: h2 linkerd
Ncbi-Phid: 82105ECBAE67541C2C9D5A8C.2.1.1
Server: Finatra
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Via: 1.1 google
X-Datasets-Version: 18.33.1
X-Ncbi-Total-Count: 1
X-Ratelimit-Limit: 10
X-Ratelimit-Remaining: 8
X-Ua-Compatible: IE=Edge
X-Xss-Protection: 1; mode=block


2026/07/23 20:08:18 
POST /datasets/v2/virus HTTP/1.1
Host: api.ncbi.nlm.nih.gov
User-Agent: OpenAPI-Generator/1.0.0/go
Content-Length: 291
Accept: application/json
Api-Key: c442c934c6f712a32c4f4ce8c75dd652f108
Content-Type: application/json
Ncbi-Phid: 82105ECBAE67541C2C9D5A8C
X-Datasets-Client: datasets-cli
X-Datasets-Client-Arch: amd64
X-Datasets-Client-Cmd: summary virus genome taxon 10239 --complete-only --released-after 2024-01-01 --debug
X-Datasets-Client-Os: linux
X-Datasets-Client-Version: 18.33.1
Accept-Encoding: gzip

{"filter":{"annotated_only":false,"complete_only":true,"geo_location":"","host":"","pangolin_classification":"","refseq_only":false,"released_since":"2024-01-01T00:00:00Z","taxons":["10239"],"updated_since":"0001-01-01T00:00:00Z","usa_state":""},"page_size":1,"returned_content":"COMPLETE"}

Error: [gateway] Post "https://api.ncbi.nlm.nih.gov/datasets/v2/virus": POST https://api.ncbi.nlm.nih.gov/datasets/v2/virus giving up after 11 attempt(s)

[ericcox1]

Hi @jo-delia-faraguna,

Thanks for opening this issue. We have been able to reproduce this error and we are investigating. I will post here with any updates.

Best,
Eric