OSError: Could not resolve hostname when using overturemaps Python API or CLI behind corporate proxy

[jaime-srr]

Summary

Both the Python API (record_batch_reader) and the CLI (overturemaps download) fail with the same low‑level error:

curlCode: 6, Could not resolve hostname
Details: Could not resolve host: overturemaps-us-west-2.s3.us-west-2.amazonaws.com

The failure occurs after the request reaches the proxy, even though a simple curl request to the bucket URL succeeds (returns the XML bucket listing).

Environment

Item	Value
OS	Ubuntu 22.04 (JEO‑Desk)
Python	3.10.12
overturemaps version	0.3.5 (installed via pip install overturemaps)
pyarrow version	24.0.0
Proxy environment variables (set in the script)	http_proxy, https_proxy, HTTP_PROXY, HTTPS_PROXY all point to the above URL

Commands that were run

1️⃣ Python script (client API)

import os
os.environ['http_proxy'] = 'http//corporate.proxy:port'
os.environ['https_proxy'] = 'http//corporate.proxy:port'

from overturemaps import record_batch_reader
from overturemaps.writers import copy, get_writer

bbox = (8.47276, 50.015353, 8.800405, 50.227143)
reader = record_batch_reader(
    "place",
    bbox=bbox,
    release="2025-05-20.0",
    stac=False,
)

with get_writer("geojson", "frankfurt_test.geojson", schema=reader.schema) as writer:
    copy(reader, writer)

Output / traceback

OSError: When getting information for key 'release/2025-05-20.0/theme=places/type=place' in bucket 'overturemaps-us-west-2':
AWS Error NETWORK_CONNECTION during HeadObject operation: curlCode: 6, Could not resolve hostname;
Details: Could not resolve host: overturemaps-us-west-2.s3.us-west-2.amazonaws.com

2️⃣ CLI command

overturemaps download \
    --bbox=8.47276,50.015353,8.800405,50.227143 \
    -f geojson \
    --type=place \
    -o frankfurt_places_test.geojson

Output / traceback

OSError: Error creating dataset. Could not read schema from
'overturemaps-us-west-2/release/2026-05-20.0/theme=places/type=place/part-00013-98c8497f-6c85-53c2-9f2e-015f0e135935-c000.zstd.parquet'.
Is this a 'parquet' file?: When reading information for key
'release/2026-05-20.0/theme=places/type=place/part-00013-98c8497f-6c85-53c2-9f2e-015f0e135935-c000.zstd.parquet'
in bucket 'overturemaps-us-west-2':
AWS Error NETWORK_CONNECTION during HeadObject operation: curlCode: 6, Could not resolve hostname;
Details: Could not resolve host: overturemaps-us-west-2.s3.us-west-2.amazonaws.com

Debugging steps already performed

curl through the proxy
curl -v -x 'http//corporate.proxy:port' \
     https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/

Result: 200 OK and the XML bucket listing (the same XML that appears in the error output).
This shows that the proxy can resolve the hostname and retrieve the bucket listing.

Requested help

We are not sure that overturemaps client handles proxy settings. Does it automatically respect the usual http_proxy / https_proxy environment variables for all S3 operations—including the HEAD and GET requests for individual objects? If the client doesn’t pick up those variables on its own, could you point me to the right way to pass proxy configuration explicitly? Any work‑arounds or tips that have helped you get downloads working through a proxy would be greatly appreciated. Thanks a lot!

[Dana Bauer (danabauer)]

Hi jaime-srr. Thanks for writing this up. You're right that this is a proxy configuration problem, not necessarily an Overture issue. Arrow's S3 client doesn't inherit standard proxy env vars, so you'll need to configure it at the AWS/Arrow level. You could try adding the proxy url to your AWS config file.

[jaime-srr]

Hi Dana, I solved it by changing the calls to S3FileSystem Arrow function in the core.py file by a crafted function that now passes the proxy configuration to S3FileSystem function. Thank you for the support!

[Dana Bauer (danabauer)]

You could also make a feature request to overturemaps-py for how you'd like our tooling to handle this!

[jaime-srr]

I'll have a look, thanks!