Add waterdata-client dataframe transformers#307
Conversation
christophertubbs
left a comment
There was a problem hiding this comment.
It might have been better to set TransformedResponse_co to something like:
TransformedResponse = Sequence[Mapping[str, Any]]
in order to keep the interface uniform where the transformer's duties are just to modify what will eventually get passed into other constructors. May also be worth coming up with an alias for list[dict[str, Any]] anyways since it sounds like it fits Rows or something.
The only thing I consider a tad shaky is the vagueness of what TransformedResponse_co might be, but I'm not looking at this in an editor and the possibility of IDE confusion doesn't qualify as a blocker.
from typing import Any
from hydrotools.waterdata_client import LatestContinuousClient
# Example custom transformer
def my_custom_transformer(data: list[dict[str, Any]]) -> int:
return 42
# Instantiate client with custom transformer
client = LatestContinuousClient(
transformer=my_custom_transformer
)
# Call client.get
value = client.get(
monitoring_location_id="USGS-02146470"
)
# Look at result
print(value) # 42 |
|
Actually, I think I'll rename the type variable to |
|
WRT a type alias for https://developmentseed.org/geojson-pydantic/intro/ |
This PR introduces new functionality to transform deserialized GeoJSON into alternative formats. Prior to this PR, all classes defined in
clients.pycould only returndict. This PR adds the ability to further process these results into more useful formats for data scientists. This PR also adds column label mapping from WaterData labels to HydroTools canonical labels for some backwards compatibility withhydrotools.nwis_client.Changes
pyproject.tomlAddedgeopandasas a dependency.base_client.pyAdded newtransformerattribute (aCallabletype) and_handle_responsemethod that calls thetransformer.clients.pyRegenerated from template.constants.pyRegenerated from template.transformers.pyNew module that defines aResponseTransformerprotocol andTransformedResponse_coTypeVar. Also includes three example transformer methods:check_features,to_geodataframe, andto_dataframe.clients.py.j2Updates Jinja2 template forclients.pyto include application of optional tranformer.constants.py.j2Updates Jinja2 template with column enums and mapping from waterdata labels to hydrotools canonical columns.README.mdAdded example using newto_geodataframetransformer.Testing
test_clients.pyUpdated to account for new transformer kwarg. Set to None to maintain current tests.test_transformers.pyAdd testing for transformers.Checklist