Skip to content

Feature/json io #1013

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 33 commits into
base: main
Choose a base branch
from
Open

Conversation

flohorovicic
Copy link
Member

Description

This PR introduces JSON input/output functionality to GemPy, allowing users to save and load geological models in JSON format. This new feature provides:

  1. Initial implementation of JSON serialization and deserialization for GemPy models
  2. Complete support for all essential model components:
    • Surface points and orientations
    • Series and structural relationships
    • Fault relations
    • Grid settings
    • Interpolation options
    • Model metadata
    • Element colors and styling

The JSON format offers several advantages:

  • Human-readable and editable model representation
  • Language-agnostic data format for interoperability
  • Structured validation of model data
  • Easy version control and model sharing

The implementation includes:

  • JsonIO class with save_model_to_json and load_model_from_json methods
  • Schema validation for JSON data
  • Sensible defaults for optional parameters
  • Comprehensive error handling

Key improvements in latest commits:

  • Colors are now correctly preserved when loading models from JSON
  • Structural relationships (ERODE, ONLAP) are properly handled
  • Fault relations matrix is correctly saved and restored
  • Metadata (creation_date, last_modification_date) is properly managed
  • Default values are provided for optional parameters
  • Comprehensive error handling for invalid data structures

Relates to #1003

Checklist

  • My code uses type hinting for function and method arguments and return values.
    • Added type hints in JsonIO class methods (load_model_from_json, save_model_to_json)
    • All helper methods include proper type annotations
  • I have created tests which cover my code.
    • Added 19 comprehensive tests in test_json_io.py
    • Tests cover both success and failure cases
    • Edge cases are properly tested
  • The test code either:
    1. demonstrates at least one valuable use case (e.g. integration tests)
      • test_multiple_series_relationships demonstrates real-world usage with multiple series
      • test_combination_model shows complex model with faults and unconformities
      • test_fault_relationships verifies fault relation handling
    2. verifies that outputs are as expected for given inputs (e.g. unit tests)
      • Unit tests for surface points loading/saving
      • Unit tests for orientations handling
      • Unit tests for metadata preservation
      • Unit tests for color preservation
  • New tests pass locally with my changes.
    • All 19 tests pass successfully
    • Only expected warnings are present (deprecation notices and known basement color warning)

- Add JSON I/O module with schema definitions

- Implement surface points loading functionality

- Add comprehensive test suite for surface points I/O

- Create tutorial demonstrating JSON I/O usage
- Add orientation schema and loading methods

- Implement orientation data validation and polarity handling

- Add comprehensive test suite for orientation loading

- Fix polarity handling in test data
- Create tutorial demonstrating JSON I/O with horizontal stratigraphic model

- Add sample JSON file with model data

- Include surface points and orientations for two horizontal layers
…id_to_name mapping from surface IDs to names - Update surface points and orientations loading to use actual surface names - Fix issue with surface names not matching between structural frame and series
…odel computation - Update surface points and orientations to match original CSV files - Add model computation step before plotting - Fix plot display using Agg backend
…s work, some problems with fault relations still remaining.
…ns requried; standard stack created by default based on point/ orientation ids.
…lization - Fixed series mapping, fault orientation, and plotting
…r visualization, fix surface points plotting, add colorbar and labels
…ervation, fixed color assignment to structural elements
…r assertions in test_multiple_series_relationships and test_combination_model
@javoha
Copy link
Member

javoha commented Apr 11, 2025

Hi,
looks good to me and tests pass. Just two notes (I also pushed them as TODOs)

  1. In 04_combination_model.py you wrote code for saving the model after computation - the resulting file is exactly the same as the one before. So this is not implemted yet - question would be if we want to do that before merging.
  2. As far as I can tell in the current version the last modification date ist nor actually updated. There is an example for getting the current date in 02_horizontal_stratigraphic.py but I am not sure if that is the final version.

Cheers,
Jan

@flohorovicic
Copy link
Member Author

Thanks a lot for the checks and the comments, @javoha!

  1. Good point - is left from a check I had done before. But it would make sense to include it, again - the functionality itself is now included in a test.
  2. True, but not required - this was mostly to show how it can be set. Maybe more something to describe in a bit more detail in an example.

Anyway, I'd say both are rather minor aspects, can be adjusted after the merge :-)

Copy link
Member

@javoha javoha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I think this feature should be part of the core gempy functionality. In the future we should include the possibility to store results. Thanks @flohorovicic!

@flohorovicic flohorovicic removed the request for review from NilsChudalla April 22, 2025 09:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants