Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running image segmenter on codepen gives different results than running image segmenter on mediapipe studio on ios chrome #5862

Open
kaneishere opened this issue Feb 18, 2025 · 6 comments
Assignees
Labels
platform:ios MediaPipe IOS issues task:image segmentation Issues related to image segmentation: Locate objects and create image masks with labels type:performance Execution Time and memory heap, stackoverflow and garbage collection related type:support General questions

Comments

@kaneishere
Copy link

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

None

OS Platform and Distribution

ios 18.3.1

Mobile device if the issue happens on mobile device

iphone 14 pro max

Browser and version if the issue happens on browser

133.0.6943.84

Programming Language and version

javascript

MediaPipe version

0.10.21

Bazel version

No response

Solution

ImageSegmenter

Android Studio, NDK, SDK versions (if issue is related to building in Android environment)

No response

Xcode & Tulsi version (if issue is related to building for iOS)

No response

Describe the actual behavior

Codepen example modified to use multiclass selfie segmenter and paint 'others' (line 158) region does not paint 'others' region when run on my ios chrome. However, the multiclass segmenter on mediapipe studio works just fine, showing me the expected 'others' region.

Describe the expected behaviour

Expect codepen example (https://codepen.io/Kane-Wu/pen/bNGNrPw) modified to use multiclass selfie segmenter (modified to show 'Others' region only) to show the same results as mediapipe studio example

Standalone code/steps you may have used to try to get what you need

https://codepen.io/Kane-Wu/pen/bNGNrPw

Other info / Complete Logs

@kaneishere kaneishere added the type:bug Bug in the Source Code of MediaPipe Solution label Feb 18, 2025
@google-ml-butler google-ml-butler bot added the type:others issues not falling in bug, perfromance, support, build and install or feature label Feb 18, 2025
@kaneishere
Copy link
Author

Running on mediapipe studio (my glasses are highlighted)

Image

Running on codepen (note that my glasses are not highlighted, the rest of the image is not painted over since I only filtered for 'others' region)

Image

@kuaashish kuaashish assigned kuaashish and unassigned kalyan2789g Feb 18, 2025
@kuaashish kuaashish added type:performance Execution Time and memory heap, stackoverflow and garbage collection related type:support General questions platform:ios MediaPipe IOS issues task:image segmentation Issues related to image segmentation: Locate objects and create image masks with labels and removed type:bug Bug in the Source Code of MediaPipe Solution type:others issues not falling in bug, perfromance, support, build and install or feature labels Feb 18, 2025
@kaneishere
Copy link
Author

some other devices I have tested on

works
Iphone 15 pro
chrome 133.0.6943.84

doesn't work
pixel 8 pro
chrome 133.0.6943.50

@kuaashish
Copy link
Collaborator

Hi @kaneishere,

We are unable to determine the cause of the detection issue with the current information. Could you please share any error logs you are encountering? These logs will help us diagnose the problem.

Thank you!!

@kuaashish kuaashish added the stat:awaiting response Waiting for user response label Feb 25, 2025
Copy link

github-actions bot commented Mar 5, 2025

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

@github-actions github-actions bot added the stale label Mar 5, 2025
@kaneishere
Copy link
Author

Hi @kaneishere,

We are unable to determine the cause of the detection issue with the current information. Could you please share any error logs you are encountering? These logs will help us diagnose the problem.

Thank you!!

These are the logs I get when I run the codepen link. No errors but I can't visualize 'Others'

LOGI0307 01:00:38.651000 1922656 segmentation_postprocessor_gl.cc:382] SOFTMAX activation function chosen on GPU, but only category mask produced, so not applying.
LOGGraph finished closing successfully.
LOGI0307 01:00:38.788000 1922656 gl_context_webgl.cc:140] Successfully destroyed WebGL context with handle 1
LOGI0307 01:00:38.802000 1922656 gl_context.cc:386] GL version: 3.0 (OpenGL ES 3.0 (WebGL 2.0)), renderer: WebKit WebGL
LOGGraph successfully started running.
LOGI0307 01:00:39.088000 1922656 segmentation_postprocessor_gl.cc:382] SOFTMAX activation function chosen on GPU, but only category mask produced, so not applying.

@google-ml-butler google-ml-butler bot removed stale stat:awaiting response Waiting for user response labels Mar 7, 2025
@kaneishere
Copy link
Author

It's not an error per se, the problem is that I'm seeing an inconsistency in the segmentation results across devices. The call to the API still returns, but the result is incorrect

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
platform:ios MediaPipe IOS issues task:image segmentation Issues related to image segmentation: Locate objects and create image masks with labels type:performance Execution Time and memory heap, stackoverflow and garbage collection related type:support General questions
Projects
None yet
Development

No branches or pull requests

3 participants