Restructure priority calculation to use group-based extremity #2079
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Remove priority calculation from math pipeline (conversation.py) * Delete _importance_metric and _priority_metric methods * Remove priority computation from recompute() method * Math pipeline now focuses on PCA, clustering, and representativeness - Add dedicated priority calculation script (502_calculate_priorities.py) * Implements PriorityCalculator class with group-based extremity * Matches Clojure priority formula: (importance * scaling_factor)^2 * Retrieves extremity values from Delphi_CommentExtremity table * Updates priorities in Delphi_CommentRouting table - Update pipeline execution order (run_delphi.py, run_delphi.sh) * Math pipeline → UMAP pipeline → Extremity calculation → Priority calculation * Ensures priorities use group-based extremity instead of PCA-based * Maintains separation of concerns between mathematical and priority calculations This fixes the priority calculation bug where all priorities were 0 due to missing extremity values, and implements proper group-based extremity usage as requested for the Pakistan conversation analysis. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Extract priority formulas into polismath/conversation/priority.py * Create PriorityCalculator class with static methods for core formulas * Port importance_metric and priority_metric from Clojure implementation * Add convenience methods: calculate_comment_priority, validate_inputs, explain_priority * Pure mathematical logic with no I/O dependencies for better testability - Refactor umap_narrative/502_calculate_priorities.py to use extracted formulas * Rename class from PriorityCalculator to PriorityService (clearer distinction) * Import and use PriorityCalculator.calculate_comment_priority() * Remove duplicate formula implementations (38 lines removed) * Focus service on DynamoDB operations and data orchestration Benefits: - Better separation of concerns: formulas vs data processing - Improved testability: mathematical logic can be unit tested independently - Enhanced reusability: priority formulas can be used in other contexts - Cleaner maintainability: formula changes only need to happen in one place Tested successfully with conversation 36324: - 807 comments processed - Priority statistics: min=0, max=3, avg=2.36 - All priorities calculated using group-based extremity values 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Remove priority calculation from math pipeline (conversation.py)
Add dedicated priority calculation script (502_calculate_priorities.py)
Update pipeline execution order (run_delphi.py, run_delphi.sh)
This fixes the priority calculation bug where all priorities were 0 due to missing extremity values, and implements proper group-based extremity usage as requested for the Pakistan conversation analysis.
🤖 Generated with Claude Code