Improved postcode picker - #2049
Draft
ajparsons wants to merge 13 commits into
Draft
Conversation
Needed for the committee popolo files in parlparse members/posts: the committee organisation classification, organisation description/links/ parent_id fields, the relaxed post id pattern, and get_localised_value() for the bilingual Senedd data.
The organization table holds committee details (name, description, url, tags, parent) with one row per organisation per language, so the bilingual Senedd committees can be stored without _cy columns everywhere. moffice gains org_id to join against it, post_type so committees can be distinguished from government posts without string-matching moffice_id, parliament, position_cy for the Welsh role, and loader so an importer can prune only its own rows. position widens to 255 because Westminster roles reach 187 characters. Additive only - nothing reads the new columns yet.
Reads the four popolo files in parlparse members/posts into the organization and moffice tables, so the Scottish Parliament, the Senedd and the NI Assembly get committee memberships alongside Westminster. A membership is either a seat on a committee defined in the file, or a role held directly in the chamber - a minister, a whip, a spokesperson. Chamber roles get an empty dept so prettify_office() shows the role on its own, which is what the Perl loader did by special-casing the name "House of Commons". Senedd organisations are written once per language, and Senedd roles carry the Welsh form in position_cy. NI ministerial roles hang off a post rather than an organisation and are skipped for now, with a count in the output. table_sync does a three-way create/update/delete so the tables are never emptied mid-import, and takes over rows an older loader wrote under the same id rather than duplicating them. Not wired into cron yet.
Member::offices() took two booleans and told committees from government posts by looking for the string "Committee" inside the moffice id. That depended on an id format only the MNIS importer produced, so every pre-2010 committee membership was treated as a government post, and no devolved membership could be classified at all. It now takes a list of post types, with Office::COMMITTEE_TYPES and Office::POST_TYPES for the two cases the callers actually want. load_extra_info joins the organization table in twice, once in the reader's language and once in English as the fallback, so an Office arrives with its committee description, url and tags already attached, and Welsh readers get Welsh committee names and roles. The memcache key moves to extra_info2 so cached rows from the old shape are not reused. pretty_dates() no longer claims a start in the year 1000 for the devolved memberships that arrive without dates, and getCommittee's date listing filters on post_type rather than a source value that has been empty for years.
Committee descriptions and links came from a nightly scrape cached as uk_committees_groups.json, matched onto memberships by comparing the committee name string. Any drift in naming between the two sources silently dropped the description, and the lookup only ran for current committees, so past memberships never had one. Both now come from the organization table, joined on the organisation id the membership already carries, which also means the devolved committees get descriptions and the past memberships get them too. The MiniGroup data classes had no other user and go with it.
The memberships page described Westminster committees to everybody and only showed cross-party groups to MPs. It now takes its introductory copy from the member's house, and shows Cross-Party Groups to MSPs and MSs under that name - the postcode cards already made that distinction, and the group data for both has been imported since e710c10. Past committee memberships get the description and external link that only current ones used to have. Public bill committees are left out of the committee lists: the profile page already shows them from the pbc data, which also knows which bill was being scrutinised. The Welsh strings for the new copy still need translating.
load-people's load_moffices() read ministers.json and ministers-2010.json into moffice. Both are superseded by parlparse members/posts, which the new importer reads, so drop the Perl loader and run scripts/posts after load-people in morningupdate and quick-populate. 0038 clears the rows the old loader wrote that the importer has not claimed. It has to test loader rather than the id alone, because the new Westminster file reuses the same uk.parliament.data ids. Anything ministers-2010.json held that the new Westminster file does not goes with this - in particular Westminster offices from before 2010. That gap belongs upstream in parlparse rather than in a second loader here. check-bad-ministers diffed moffice against those two files and had no caller; the importer's own prune replaces it.
Covers offices() filtering by post type rather than by the moffice id, the organization details arriving on the Office through the join, the join not duplicating an office when its committee has both an English and a Welsh organization row, the unknown-start-date wording, and public bill committee detection.
ajparsons
force-pushed
the
improved-postcode-picker
branch
from
August 14, 2026 13:35
95f0bc6 to
9aa43aa
Compare
ajparsons
marked this pull request as draft
August 14, 2026 13:36
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.