Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Brest, FR elevation data sources #32

Open
XioNoX opened this issue Aug 23, 2023 · 5 comments · May be fixed by #35
Open

Add Brest, FR elevation data sources #32

XioNoX opened this issue Aug 23, 2023 · 5 comments · May be fixed by #35
Labels
enhancement New feature or request

Comments

@XioNoX
Copy link

XioNoX commented Aug 23, 2023

Opening this ticket to start a conversation.

My urban community (https://en.wikipedia.org/wiki/Brest_M%C3%A9tropole) have elevation data as contour data exposed in their open data portal: https://geo.brest-metropole.fr/arcgis/rest/services/public/GPB_REF/MapServer/2210001

I used pyesridump to "scrape" all of it as GeoJSON. I temporarily hosted the file for 30 days in https://transfert.free.fr/NhszRPT

For the datasources.json file:

I'm thinking something like:

{
	"name": "Brest 50cm contours",
	"url": "TBD",
	"filename": "brest_contours.geojson",
	"crs": "ESRI:102110",
	"bbox": [-4.552179, 48.355777, -4.434689727004813, 48.44768660963146],
	"download_method": "http",
	"lookup_method": "contour_lines",
	"lookup_field": "ELEVATION",
	"units": "meters",
	"recheck_interval_days": null
},
  • url:
    • Where can I host that ~1Gb file long term?
    • Could "elevation_lookups" use "pyesridump" to download data from the arcgis server directly (it takes some time, especially at it pauses for 10s between each queries, I guess to not overload the server)?
  • crs: the page says "Spatial Reference: 102110 (2154)", https://spatialreference.org/ref/esri/102110/
  • bbox: later on we can probably enlage the box to include the whole metro area, unless you think it's better to do it now.

Let me know what you think,

Thanks
@XioNoX XioNoX added the enhancement New feature or request label Aug 23, 2023
@XioNoX XioNoX mentioned this issue Aug 23, 2023
Merged
@dabreegster
Copy link
Collaborator

Where can I host that ~1Gb file long term?

Happy to use the A/B Street S3 bucket; we do it for other large files with similar hosting / reproducibility problems. I'll upload a copy hopefully tonight when I'm in a spot with a faster connection and post the URL here

@eldang
Copy link
Owner

eldang commented Aug 23, 2023

Thank you, both!

@dabreegster much appreciated on the storage.

@XioNoX :

  • I think incorporating pyesridump or something like it is a great idea, because I can imagine many situations in which temporary storage for one region at a time is easy to arrange, but long term storage for everywhere is cost-prohibitive. I don't think I'll have the time to do that in the near future, but if you do then I would love a PR for it. And if not, then I'm happy to write an issue and see if anyone else comes forward.
  • For the CRS, I think PROJ supports the ESRI: namespace so it should work. I suggest testing it empirically: if that reference gets you sensible results then keep it, and if not then it may be worth trying EPSG:2154 or EPSG:102110 to see if either of those do.
  • The bounding box for this one is a bit of a difficult call! The trouble is that if we use the extent of the whole file, we'll get weird results in the NW and NE corners, where the script should be falling back to SRTM data but won't know that so it will either return zeroes or whatever the nearest contour happens to be. So I think for general use, the ideal bbox for now is the largest rectangle that fits within the dataset + the sea / estuary. Unfortunately that would ignore some of the real data, so for Brest-specific uses it may make sense to use a larger one.

In the long run, between this and #31 (comment) I am seeing a need to add the ability to use a real dataset boundary instead of a bbox as appropriate.

@eldang eldang mentioned this issue Aug 23, 2023
Open
dabreegster added a commit to a-b-street/abstreet that referenced this issue Aug 23, 2023
@dabreegster
Save a copy of Brest elevation data in S3. See eldang/elevation_looku…
e8acee6 
…ps#32 for source info
@dabreegster
Copy link
Collaborator

https://play.abstreet.org/dev/data/input/shared/elevation/brest.geojson.gz only 200MB compressed, FWIW

XioNoX added a commit to XioNoX/elevation_lookups that referenced this issue Aug 23, 2023
@XioNoX
Add Brest, FR 50cm contours
dd45a28 
Original data source: https://geo.brest-metropole.fr/portal/home/item.html?id=c0e498ab9b2547659b5a76509c99bc00

Fixes eldang#32
@XioNoX XioNoX mentioned this issue Aug 23, 2023
Closed
@XioNoX
Copy link
Author

XioNoX commented Aug 23, 2023

Thanks for your feedback. I made a tentative with the linked PR but I'm a bit lost on what are the next steps to have it show up in ABStreet (my goal eventually :) ).
I don't have time to incorporate pyesridump (even though it doesn't seem too hard). Opening a separate task about it makes sens to me.
For the CRS, noted. Let's see if that one works as expected.

@dabreegster dabreegster mentioned this issue Aug 29, 2023
Open
@XioNoX XioNoX linked a pull request Aug 29, 2023 that will close this issue
Open
@XioNoX
Copy link
Author

XioNoX commented Nov 10, 2023

I noticed that my city's public ESRI can't be reached from outside of France. They're looking at removing that restriction but it makes the "pyesridump" option less straightforward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
3 participants
@XioNoX @eldang @dabreegster