Add esrunner sample project with how-to guide. #169

joshhvulcan · 2025-07-24T18:28:43Z

No description provided.

data/esrunner_sample/checkpoint.ckpt

hunterp · 2025-07-25T00:26:00Z

data/esrunner_sample/README.md

+
+## Setting up your environment
+
+- Install `esrunner` (earth-system-run) in your development environment (Or clone the repository and add to your `PYTHONPATH`. If you go this route, ensure you install the packages listed in `earth-system-run/requirements.txt`)


I do not think anybody should clone the repo.

can you provide the exact command to install esrunner.

Could it not be part of the project-specific requirements.txt, pointing to a git repo at a specific tag? that way when we're installing deps for the docker image we can reliably just say pip install rslp-projects/thingy/requirements.txt (or requirements.frozen.txt).

They should probably be running a separate venv per project, so this doesn't seem like a hard sell?

Per our conversation toady, we intend to continue to have one set of requirements for all of rslearn_projects. We use one environment and build one Docker container for all projects. Projects (as in different applications for which we want to fine tune models) share the vast majority of requirements and those that are not shared are typically specific to individual model architectures like TerraMind vs OLMo-Earth so even then they would not be project-specific, unless "experiments to compare OLMo-Earth against TerraMind / DINOv2" is considered a project which is not really how we think about it. There are some requirements like prometheus-client that are only used for specific projects like vessel detection but I think the intention is to make things more consistent across projects e.g. using the same system for observability.

hunterp · 2025-07-25T00:27:01Z

data/esrunner_sample/README.md

+- `partition_strategies.yaml`: 
+- `postprocessing_strategies.yaml`: This file defines how the esrunner will post-process the predictions.  
+- `requirements.txt`: This file contains the additional Python packages required for the pipeline. It should include any dependencies that are not part of the base environment.
+- `prediction/test-request1.geojson`: This directory contains the prediction requests in GeoJSON format. Each file represents a set of prediction requests for a specific region or time period.  Many different prediction requests can be defined within a single file as separate features in the feature collection. The esrunner will partition these requests into smaller tasks based on the partition strategies defined in `partition_strategies.yaml`.


why is this a directory of geojson files vs just a single feature collection?

This came from a conversation with Henry where he mentioned wanting to be able to work with different input geometries. I figured providing a pattern for managing these different inputs would work better than saying "you must only have one input file".

data/esrunner_sample/README.md

hunterp · 2025-07-25T00:30:12Z

data/esrunner_sample/README.md

+## Setting up your environment
+
+- Install `esrunner` (earth-system-run) in your development environment (Or clone the repository and add to your `PYTHONPATH`. If you go this route, ensure you install the packages listed in `earth-system-run/requirements.txt`)
+- Following the project structure below, create a directory in the `rslearn-projects/data/` directory. This directory will contain all the necessary files for your prediction or fine-tuning pipeline.


if we're telling folks to have the rslp/data/{my_project}/ directory match this structure; why do we need to have each filename passed in? vs just passing in rslp/data/{my_project}/

This is one of those things I want to validate with the ML folks. There are a lot of cases where they store a variety of model configs for different experiments in the same directory. I am assuming they will want to continue doing that to some degree. Perhaps a happy medium would be to read the prescribed names but also allow them to be overridden in the EsPredictionRunner.__init__() for flexibility.

data/esrunner_sample/README.md

hunterp · 2025-07-25T00:57:27Z

rslp/espredict_runner.py

this appears to be the same as in esrun. why?

oh i need to delete this. I had it here first and then copied to es run and forgot to remove this.

hunterp · 2025-07-25T00:58:01Z

pyproject.toml

+file = ["requirements.txt"]
+
+[tool.setuptools.dynamic.optional-dependencies]
+ai2 = { file = ["ai2_requirements.txt"] }


I don't see ai2_requirements.txt here.

Can we chat with @StephenWithPH about the path forward here as well.

it already existed. I'm just wiring it up to be accessible via pip install rslearn_projects[ai2]. Totally agree on the Stephen thing.

hunterp · 2025-08-08T03:42:17Z

.github/workflows/build-esrunner.yml

+        uses: actions/checkout@v4
+        with:
+          repository: allenai/helios
+          ref: josh/split-evals


favyen2 · 2025-08-21T15:39:13Z

I think we should merge this but I'm not sure about mixing it with the requirements change. If we need to change the requirements, can we remove the part that prevents it from installing with Python 3.12+ and make it so rslearn[extra,dev] can be installed? Currently only extra appears in pyproject.toml and it is called all instead of extra, but may be more clear to match the name of the file. Also if this format is desired then ai2_requirements.txt should be renamed requirements-ai2.txt.

joshhvulcan · 2025-08-21T16:27:22Z

@favyen2 I am probably going to close this one and reimplement from master so that its all clean. A lot has changed since I opened this so its probably best just to start fresh and build back up. I will put that on my list for today.

favyen2 · 2025-08-21T16:30:41Z

Sounds good maybe can have separate PRs for adding the example versus building the Docker container for esrun (which may involve updates to how dependencies are split up).

hunterp reviewed Jul 24, 2025

View reviewed changes

data/esrunner_sample/checkpoint.ckpt Outdated Show resolved Hide resolved

hunterp reviewed Jul 25, 2025

View reviewed changes

data/esrunner_sample/README.md Outdated Show resolved Hide resolved

hunterp reviewed Jul 25, 2025

View reviewed changes

data/esrunner_sample/README.md Outdated Show resolved Hide resolved

hunterp reviewed Jul 25, 2025

View reviewed changes

data/esrunner_sample/README.md Outdated Show resolved Hide resolved

hunterp reviewed Jul 25, 2025

View reviewed changes

joshhvulcan force-pushed the josh/sample-esrunner branch from 34eab54 to ae936d3 Compare August 8, 2025 00:20

hunterp reviewed Aug 8, 2025

View reviewed changes

joshhvulcan and others added 10 commits August 13, 2025 11:32

Add harness example

617427e

Split requirements out

70271d0

Add README.md

210bb73

Updates

fd38dfa

Initial CI attempt of rslearn+esrunner

25253ce

Functional state of things

b9c7df8

Faster docker builds

d73fdbe

Use github token

1c556d4

test

2f10396

Change ref for helios.

691d32a

joshhvulcan force-pushed the josh/sample-esrunner branch from ae936d3 to 691d32a Compare August 13, 2025 18:33

skwash added 6 commits August 13, 2025 11:55

Linting for esrunner

8355aa3

Use GIT_TOKEN?

9280218

Use different secret for cloning non-public repos

6fabdfe

Fix tag name

c555088

Use the correct dockerfile

ff47bae

Fix image name

9c3841d

favyen2 closed this pull request by merging all changes into master in 2e8f28c Aug 26, 2025

favyen2 deleted the josh/sample-esrunner branch August 26, 2025 16:45


		## Setting up your environment

		- Install `esrunner` (earth-system-run) in your development environment (Or clone the repository and add to your `PYTHONPATH`. If you go this route, ensure you install the packages listed in `earth-system-run/requirements.txt`)

Add esrunner sample project with how-to guide. #169

Add esrunner sample project with how-to guide. #169

Uh oh!

Conversation

joshhvulcan commented Jul 24, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cmwilhelm Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hunterp Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

favyen2 commented Aug 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joshhvulcan commented Aug 21, 2025

Uh oh!

favyen2 commented Aug 21, 2025

Uh oh!

Uh oh!

cmwilhelm Jul 25, 2025 •

edited

Loading

hunterp Jul 25, 2025 •

edited

Loading

favyen2 commented Aug 21, 2025 •

edited

Loading