Skip to content

[v0/v1 migration] Move metadata modals in tools to V2#6090

Merged
nick-nlb merged 10 commits intodatacommonsorg:masterfrom
nick-nlb:metadata_v2_tools
Mar 17, 2026
Merged

[v0/v1 migration] Move metadata modals in tools to V2#6090
nick-nlb merged 10 commits intodatacommonsorg:masterfrom
nick-nlb:metadata_v2_tools

Conversation

@nick-nlb
Copy link
Contributor

Issue

b/491842059

Description

This is part two of the conversion of the website metadata handling from V1 to V2.

See 6078 for part 1, which describes the overall purpose.

This part converts metadata calls related to the three tools (timeline, scatter and map) to use the V2 metadata endpoint.

Solution

The primary change was to send entities up through into the metadata modal and the chart embed for each of the tools.

However, this update (and the general move to V2) highlighted some changes that needed to be made to all three of the tools in order to supply the precisely used numerator and demoninator facets in each of the charts. This is what makes up the majority of the changes in this PR.

Testing

For the most part, the metadata modal should show the same information as it does in production. However, there are certain areas where we can expect divergences. The production metadata modals do not recognize denominator values (and so will not display the chosen denomator. Additionally, the production metadata modals will sometimes show only a single facet when more facets were actually used (ultimately for the same reason).

The following chart demonstrates both of these issues:

Literacy in India

Upcoming

This is the second change in a series. Pending changes coming in later PRs are:

  • The moving of facet selection dialog metadata to V2.
  • Final cleanup and removal of the non V2 endpoints.

…dal and chart embed. Note that this required some work to correctly resolve the used denom facets
…with a reworking of the facet plumbing within the tool to ensure that the correct (actually used) facets are supplied to the metadata modal.
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request represents the second phase of migrating the website's metadata infrastructure from V1 to V2, specifically targeting the map, scatter, and timeline tools. The core objective is to enhance the precision of metadata presented to users by ensuring that the exact numerator and denominator facets used in chart calculations are correctly identified and passed to the metadata modals. This update improves data transparency and accuracy within these visualization tools.

Highlights

  • V2 Metadata Migration for Tools: This pull request continues the migration to V2 metadata handling by converting the timeline, scatter, and map tools to use the new V2 metadata endpoint.
  • Precise Facet Handling: The changes ensure that the precisely used numerator and denominator facets are supplied for each chart, improving the accuracy of metadata displayed in the tools.
  • Entity Information for Modals: The pull request introduces the passing of entity DCIDs to metadata modals and chart embeds, allowing for more context-aware metadata displays.
Changelog
  • static/js/tools/map/chart.tsx
    • Imported useMemo hook.
    • Introduced a useMemo hook to compile a deduplicated list of entity DCIDs from map regions, map points, and breadcrumbs.
    • Passed the newly compiled entities list to the ToolChartFooter component.
  • static/js/tools/map/chart_loader.tsx
    • Modified the logic for compiling facets to prioritize numerFacets from chartStore.mapValuesDates.data for the main statistical variable.
    • Added functionality to explicitly include denominator facets (denomFacets) when a statistical variable is displayed per capita and a denominator is present.
    • Updated the dependency array for the useMemo hook to include new facet-related dependencies.
    • Compiled a list of entities from various chart data sources (mapValues, mapPointValues, breadcrumbValues).
    • Passed the entities list to the ChartEmbed component.
  • static/js/tools/map/chart_store.ts
    • Added denomFacets and numerFacets properties to the mapValuesDates interface to store sets of facet IDs.
  • static/js/tools/map/compute/map_value_dates.ts
    • Initialized denomFacets and numerFacets as new Set objects.
    • Populated denomFacets and numerFacets based on the denomFacet in placeChartData and the facet in wantedFacetData.
    • Included denomFacets and numerFacets in the payload object returned for mapValuesDates.
  • static/js/tools/map/util.ts
    • Added an optional denomFacet property to the PlaceChartData interface.
    • Initialized denomFacet to null within the getPlaceChartData function.
    • Assigned the popFacetId to denomFacet when population data is used for per capita calculations.
    • Returned the denomFacet as part of the PlaceChartData object.
  • static/js/tools/scatter/chart.tsx
    • Imported useMemo hook.
    • Introduced a useMemo hook to compile a deduplicated list of entity DCIDs from the chart's data points.
    • Passed the compiled entities list to the ToolChartFooter component.
  • static/js/tools/scatter/chart_loader.tsx
    • Added xDenomFacets, yDenomFacets, xNumerFacets, and yNumerFacets properties to the ChartData type.
    • Refactored the facet processing logic into a new processFacets helper function for reusability.
    • Utilized chartData.xNumerFacets, chartData.yNumerFacets, chartData.xDenomFacets, and chartData.yDenomFacets to populate the statVarToFacets and facets maps.
    • Updated the dependency array for the useMemo hook to include chartData and per capita/denominator-related properties.
    • Passed the keys of chartData.points as entities to the ChartEmbed component.
    • Included populationData.facets in the metadataMap during the data loading process.
    • Initialized xDenomFacets, yDenomFacets, xNumerFacets, and yNumerFacets as new Set objects.
    • Populated these new facet sets based on placeChartData and statistical data for X and Y axes.
    • Returned the new facet sets as part of the getChartData function's result.
  • static/js/tools/shared/vis_tools/tool_chart_footer.tsx
    • Added an optional entities array property to the ToolChartFooterProps interface.
    • Passed the props.entities to the TileSources component.
  • static/js/tools/timeline/chart.tsx
    • Modified the logic for embedStatVarToFacets to directly use this.state.statData.denomFacets instead of findFirstAvailableFacet.
    • Passed the keys of this.props.placeNameMap as entities to both the ToolChartFooter and ChartEmbed components.
  • static/js/tools/timeline/data_fetcher.ts
    • Added an optional denomFacets property to the StatData interface.
    • Combined rawData.statAllData.facets and rawData.denomData.facets into a single facets object.
    • Initialized denomFacets as a new Set object.
    • Populated result.denomFacets with the denomFacet obtained from rawData.denomData.
  • static/js/utils/scatter_data_utils.ts
    • Added an optional denomFacet property to the PlaceAxisChartData interface.
    • Initialized denomFacet to null within the getPlaceAxisChartData function.
    • Assigned denomSeries.facet to denomFacet when a denominator series is used.
    • Assigned popSeries.facet to denomFacet when population data is used for per capita calculations.
    • Returned the denomFacet as part of the PlaceAxisChartData object.
    • Added optional xDenomFacet and yDenomFacet properties to the PlaceScatterData interface.
    • Returned xDenomFacet and yDenomFacet as part of the getPlaceScatterData function's result.
Activity
  • This pull request is part of a larger series of changes, following up on PR 6078, to migrate the website's metadata handling from V1 to V2.
  • The author has provided a detailed description outlining the issue, solution, testing notes, and upcoming changes in subsequent PRs.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request continues the migration of website metadata handling from V1 to V2, focusing on the timeline, scatter, and map tools. The changes primarily involve passing entity DCIDs and accurately tracking numerator and denominator facets for metadata display. The implementation is solid, but I've identified a few areas for improvement regarding code duplication and simplification, which would enhance maintainability and readability.

@nick-nlb nick-nlb marked this pull request as ready for review March 17, 2026 20:35
@nick-nlb nick-nlb changed the title [V2 Migration] Move metadata modals in tools to V2 [v0/v1 migration] Move metadata modals in tools to V2 Mar 17, 2026
@nick-nlb nick-nlb requested a review from juliawu March 17, 2026 20:48
Copy link
Contributor

@juliawu juliawu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hooray! Thank you for also fixing denominator facet handling in the tools!

@nick-nlb nick-nlb merged commit ac6fbb1 into datacommonsorg:master Mar 17, 2026
12 checks passed
nick-nlb added a commit that referenced this pull request Mar 19, 2026
## Issue
[b/491842059](https://buganizer.corp.google.com/issues/491842059)

## Description

This is part three of the conversion of the website metadata handling
from V1 to V2.

See [6078](#6078) for part
1, which describes the overall purpose.
See [6090](#6090) for part
2.

This third part:
* adds a new `api/metadata/facet` endpoint that enriches the facets
using only V2 calls (replicating what was previously done using V1 in
the frontend). This is parallel to the recently added `api/metadata`
call.
* Updates the front end so that entities (or enclosed places and place
types) are sent through to the facet selector.

## Notes

The use of the observation endpoint to get earliest and latest dates
threw a challenge here, because in certain cases (such as with US
counties and multiple stat vars), there were too many series and the
mixer refused to handle the request.

This is parallel to what happens in `series.py`, where this is
compensated for with very highly specific triggers that result in
batching.

We have replicated the same functionality (adjusted to fit our
situation) here.

A note that I left a TODO to merge the constants between the two file.
That can be a small follow-up PR to this later (I didn't want to touch
series.py in this PR).

Another note is that the changes to metadata.py may appear more
significant than they really are (there was some refactoring to use
reuse code between the two endpoints.

## Testing

For the most part, the facet modal should show the same information as
it does in production. The primary point of divergence will be dates.
The old facet modal used facet-wide dates, whereas the new V2 endpoints
are more accurate, as they incorporate entities.

Pages that demonstrate the new facet modal are:
*
[scatter](http://localhost:8080/tools/scatter#svx%3DDifferenceRelativeToBaseDate2006_Max_Temperature_RCP45%26dx%3DCount_Person%26svy%3DPercent_Person_WithCoronaryHeartDisease%26dy%3DCount_Person%26epd%3Dcountry%2FUSA%26ept%3DCounty%26qd%3D1%26dd%3D1)
- with batching!
* [explore
page](http://localhost:8080/explore?enable_feature=enable_stat_var_autocomplete#q=Jobs%20in%20Texas)
*
[maps](http://localhost:8080/tools/map#%26sv%3DCount_Person_EducationalAttainmentBachelorsDegreeOrHigher%26pc%3D1%26denom%3DCount_Person%26pd%3DgeoId%2F08%26ept%3DCounty)
*
[timeline](http://localhost:8080/tools/timeline#&place=geoId/0606000,geoId/2511000,geoId/2603000,geoId/1777005,geoId/1225175,geoId/4815976&statsVar=Median_Income_Person)
* Download Tool

## Upcoming

This is the second change in a series. Pending changes coming are:
* Final cleanup and removal of the non V2 endpoints.
nick-nlb added a commit that referenced this pull request Mar 19, 2026
## Issue
[b/491842059](https://buganizer.corp.google.com/issues/491842059)

## Description

This is part four of the conversion of the website metadata handling
from V1 to V2.

See [6078](#6078) for part
1, which describes the overall purpose.
See [6090](#6090) for part
2.
See [6104](#6104) for part
3.

This fourth part:
* removes the V1 metadata endpoints completely from the frontend.
* wires the surface through to the new facet / metadata endpoints.

## Testing

This should have no visual or functional effect on the website.

## Codacy Note

This PR is failing codacy but in a way we cannot resolve. If we add the
explicit type, then ESLint complains. We can ignore codacy here.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants