feat(detections): ✨ paligemma segmentation support added - #1663
Draft
onuralpszr wants to merge 2 commits into
Draft
onuralpszr wants to merge 2 commits into
onuralpszr wants to merge 2 commits into
Conversation
onuralpszr
force-pushed
the
feat/paligemma_segmentation
branch
from
November 8, 2024 06:52
6c4bd86 to
1aeb573
Compare
Signed-off-by: Onuralp SEZER <thunderbirdtr@gmail.com>
onuralpszr
force-pushed
the
feat/paligemma_segmentation
branch
from
November 8, 2024 06:59
1aeb573 to
5fdb3fb
Compare
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Segmentation tokens currently produce only empty masks, malformed records may be accepted, and the new behavior lacks tests.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds parsing of PaliGemma segmentation outputs into Detections.
Changes:
- Parses segmentation tokens and returns masks.
- Passes parsed masks into
Detections.
File summaries
| File | Description |
|---|---|
supervision/detection/lmm.py |
Adds segmentation parsing and mask output. |
supervision/detection/core.py |
Propagates masks into Detections. |
Review details
Suppressed comments (1)
supervision/detection/lmm.py:105
- The parsed
seg_tokensnever affectmasks, so every valid segmentation is returned as an all-false image-sized mask. Decode each 16-token code with the PaliGemma segmentation decoder and map the decoded mask into its bounding box before returning it; otherwise this feature silently reports empty segmentations.
seg_tokens = matches[:, 4:-1].astype(int)
masks = [np.zeros((h, w), dtype=bool) for tokens in seg_tokens]
- Files reviewed: 2/2 changed files
- Comments generated: 2
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| class_name = np.char.strip(class_name.astype(str)) | ||
| class_id = None | ||
| segmentation_pattern = re.compile( | ||
| r"<loc(\d{4})><loc(\d{4})><loc(\d{4})><loc(\d{4})>\s*" |
Comment on lines
+99
to
+100
| segmentation_matches = segmentation_pattern.findall(result) | ||
| if segmentation_matches: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adding PaliGemma Segmentation parse ability to conver to detections masks
Type of change