Import cover images & external references by mihow · Pull Request #838 · RolnickLab/antenna

mihow · 2025-05-08T07:16:40Z

Summary

Adds management command to update any field on existing taxa

List of Changes

New management command update_taxa
Update example image title
Allow manual URL filter param include_observed to show all species, regardless of occurrence count
Allow sorting by cover image URL, so it's easier to demo taxa that have images

Deployment

Run this command in each environment to import images to known Panama species

docker compose  run --rm django python manage.py update_taxa --format csv https://docs.google.com/spreadsheets/d/e/2PACX-1vTH0BwanooyMUWWTkxttCqUyLwDDB4WYy10FqW53eprMcfSs1hiNxej3bLegHZpduPU4YwyCx1fGf8Y/pub\?gid\=1624392492\&single\=true\&output\=csv --list panama-cover-images

Part of #813

netlify · 2025-05-08T07:16:50Z

✅ Deploy Preview for antenna-preview ready!

Name	Link
🔨 Latest commit	`15fe076`
🔍 Latest deploy log	https://app.netlify.com/sites/antenna-preview/deploys/681c5a5bdd511c0008289f6a
😎 Deploy Preview	https://deploy-preview-838--antenna-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
Lighthouse	1 paths audited Performance: 75 (🔴 down 5 from production) Accessibility: 89 (no change from production) Best Practices: 92 (no change from production) SEO: 100 (no change from production) PWA: 80 (no change from production) View the detailed breakdown and full score reports

To edit notification comments on pull requests, go to your Netlify site configuration.

Copilot

Pull Request Overview

This PR enhances the taxa import and image reference functionality by adding new management commands and updating models, API views, serializers, and admin interfaces to support cover image URLs, clustering, and external references. Key changes include:

Introducing the new management command “update_taxa” for bulk updating Taxon records.
Updating models and migrations to add fields such as cover_image_url, cover_image_credit, unknown_species, and ood_score.
Enhancing clustering and API endpoints to support new demo functionality and improving admin actions.

Reviewed Changes

Copilot reviewed 68 out of 68 changed files in this pull request and generated no comments.

File	Description
ami/ml/clustering_algorithms/agglomerative.py	Improved clustering functionality and minor logging string cleanup.
ami/main/management/commands/update_taxa.py	New management command with CSV processing improvements.
ami/jobs/models.py	Added DetectionClusteringJob with redundant status updates to review.
... (other files)	Various migrations, API view, serializer, and admin interface updates.

Copilot

Pull Request Overview

This PR adds a new management command (update_taxa) for updating existing taxa data, enables new taxa filters and improves taxa import functionality. Key changes include the addition of a new filter in the UI, updating label text in species details, and modifications to the taxa import and update commands to support new fields such as cover image URLs and credits.

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
ui/src/utils/useFilters.ts	Added a new filter option with label "Show unobserved taxa".
ui/src/pages/species-details/species-details.tsx	Updated occurrence label from "Representative occurrence" to "Example occurrence".
ami/main/management/commands/update_taxa.py	Introduced a new management command for updating taxa using CSV input.
ami/main/management/commands/import_taxa.py	Enhanced taxon creation to include new fields and lookup by gbif_taxon_key.
ami/main/api/views.py	Extended API view to include cover image URL in the output.

Comments suppressed due to low confidence (1)

ui/src/utils/useFilters.ts:115

[nitpick] The filter name 'include_unobserved' may be inconsistent with the documented URL parameter 'include_observed'; please verify that the naming reflects the intended behavior.

label: 'Show unobserved taxa',

Copilot · 2025-05-08T07:20:42Z

+    reader = csv.DictReader(open(fname))
+    taxa = [row for row in reader]


[nitpick] Consider using a context manager (e.g., 'with open(fname) as csvfile:') for reading the CSV to ensure the file is properly closed after use.

Suggested change

reader = csv.DictReader(open(fname))

taxa = [row for row in reader]

with open(fname) as csvfile:

reader = csv.DictReader(csvfile)

taxa = [row for row in reader]

Copilot · 2025-05-08T07:20:42Z

            # Assume ranks are in order of rank
            if rank.name.lower() in taxon_data.keys() and taxon_data[rank.name.lower()]:
                name = taxon_data[rank.name.lower()]
+                gbif_taxon_key = taxon_data.get("gbif_taxon_key", None)


Ensure that 'gbif_taxon_key' is validated (i.e., not None) before including it in the query filter, as passing None might lead to unintended matches in the lookup logic.

@mihow remove the changes from import_taxa.py

* feat: management command for updating fields on existing Taxa from CSV * feat: try another title for example occurrence image * feat: allow manual filter for showing taxa without occurrences * feat: allow sorting by reference image for better demos

* Import cover images & external references (#838) * feat: management command for updating fields on existing Taxa from CSV * feat: try another title for example occurrence image * feat: allow manual filter for showing taxa without occurrences * feat: allow sorting by reference image for better demos * Fields for Taxon reference images (#822) * feat: add fields for Taxon cover images and Fieldguide ID * feat: add cover image fields to Taxon model for the UI * fix: look up existing taxa by name only * fix: update existing genus parents if their rank is wrong, don't crash * fix: don't allow empty CSV columns to clear existing data * fix: ensure all taxa in import list are added to taxa list in DB * fix: only keep changes related to importing taxa in this branch * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix: missing migration after last commit in #946 this is only a configuration change, it does not alter the DB schema --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Import cover images & external references (#838) * feat: management command for updating fields on existing Taxa from CSV * feat: try another title for example occurrence image * feat: allow manual filter for showing taxa without occurrences * feat: allow sorting by reference image for better demos * Fields for Taxon reference images (#822) * feat: add fields for Taxon cover images and Fieldguide ID * feat: add cover image fields to Taxon model for the UI * fix: look up existing taxa by name only * fix: update existing genus parents if their rank is wrong, don't crash * fix: don't allow empty CSV columns to clear existing data * fix: ensure all taxa in import list are added to taxa list in DB

mihow added 4 commits May 8, 2025 00:05

feat: management command for updating fields on existing Taxa from CSV

4bd7ebe

feat: try another title for example occurrence image

fd755b7

feat: allow manual filter for showing taxa without occurrences

d84beab

feat: allow sorting by reference image for better demos

15fe076

Copilot AI review requested due to automatic review settings May 8, 2025 07:16

mihow changed the base branch from main to deployments/ood.antenna.insectai.org May 8, 2025 07:16

Copilot AI reviewed May 8, 2025

View reviewed changes

mihow added this to the OOD Integration milestone May 8, 2025

mihow self-assigned this May 8, 2025

mihow requested a review from Copilot May 8, 2025 07:19

mihow merged commit 92a24d1 into deployments/ood.antenna.insectai.org May 8, 2025
6 checks passed

mihow deleted the feat/import-cover-images branch May 8, 2025 07:20

Copilot AI reviewed May 8, 2025

View reviewed changes

This was referenced May 12, 2025

Integrate reference examples from Fieldguide #523

Closed

Clean up global vs. project taxa & API responses #853

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Import cover images & external references#838

Import cover images & external references#838
mihow merged 4 commits intodeployments/ood.antenna.insectai.orgfrom
feat/import-cover-images

mihow commented May 8, 2025 •

edited

Loading

Uh oh!

netlify Bot commented May 8, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI May 8, 2025

Uh oh!

Copilot AI May 8, 2025

Uh oh!

mihow May 8, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		reader = csv.DictReader(open(fname))
		taxa = [row for row in reader]

Conversation

mihow commented May 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

List of Changes

Deployment

Uh oh!

netlify Bot commented May 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for antenna-preview ready!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI May 8, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI May 8, 2025

Choose a reason for hiding this comment

Uh oh!

mihow May 8, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mihow commented May 8, 2025 •

edited

Loading

netlify Bot commented May 8, 2025 •

edited

Loading