From 20fe522cfe2f5a9462a6d235a25cf64033b36b71 Mon Sep 17 00:00:00 2001 From: QuinnHe Date: Thu, 1 Feb 2024 07:53:46 -0600 Subject: [PATCH] Quinn.update handpose model page (#38) * update ml5 lib path * update model description --- docs/reference/facemesh.md | 5 ++- docs/reference/handpose.md | 41 +++++++++++++++++-- .../iframes/handpose-keypoints/main.html | 2 +- .../iframes/pose-estimation/main.html | 2 +- 4 files changed, 43 insertions(+), 7 deletions(-) diff --git a/docs/reference/facemesh.md b/docs/reference/facemesh.md index 0cc8b10..8056477 100644 --- a/docs/reference/facemesh.md +++ b/docs/reference/facemesh.md @@ -12,7 +12,10 @@ Facemesh is a machine-learning model that allows for facial landmark detection i The ml5.js Facemesh model is ported from the [TensorFlow.js Facemesh implementation](https://github.com/tensorflow/tfjs-models/tree/master/face-landmarks-detection). -## Quickstart +## Getting Started +Ready to give it a try? Just follow our simple instructions, and you'll be on your way to creating your very own Facemesh project in no time! + +### Quickstart ```js let predictions = []; diff --git a/docs/reference/handpose.md b/docs/reference/handpose.md index efe3ab4..52c6825 100644 --- a/docs/reference/handpose.md +++ b/docs/reference/handpose.md @@ -8,12 +8,43 @@ ## Description -Handpose is a machine-learning model that allows for palm detection and hand-skeleton finger tracking in the browser. It can detect a maximum of one hand at a time and provides 21 3D hand keypoints that describe important locations on the palm and fingers. +Handpose is a machine-learning model that allows for palm detection and hand-skeleton finger tracking in the browser. It can detect multiple hands at a time and for each hand, provides 21 3D hand keypoints that describe important locations on the palm and fingers. -The ml5.js Handpose model is ported from the [TensorFlow.js Handpose implementation](https://github.com/tensorflow/tfjs-models/tree/master/handpose). +The ml5.js Handpose model is ported from the [TensorFlow.js Mediapipe Handpose implementation](https://github.com/tensorflow/tfjs-models/tree/master/hand-pose-detection). + +#### Key Features +- **Palm Detection**: Handpose can detect the palm of a hand and provide the 2D and 3D coordinates of 21 keypoints on the hand. +- **Finger Tracking**: Handpose can track the 3D coordinates of the tips and joints of the fingers. +- **Handedness**: Handpose can determine the handedness (left or right) of the detected hand. +- **Multiple Hands**: Handpose can detect multiple hands at the same time. + +#### Output Example +Imagine we detected two hand poses in a frame. The output would look like this: + +```javascript +[ + { + score: 0.86, + handedness: "Left", + keypoints: [ + { x: 623.57, y: 374.79, score: 0.85, name: "wrist" }, + // Additional keypoints here... + ], + keypoints3D: [ + { x: 0.0024, y: 0.070, z: 0.035, score: 0.85, name: "wrist" }, + // Additional 3D keypoints here... + ], + index_finger_dip: { x: /* value */, y: /* value */, x3D: /* value */, y3D: /* value */, z3D: /* value */ }, + index_finger_mcp: { x: /* value */, y: /* value */, x3D: /* value */, y3D: /* value */, z3D: /* value */ }, + // Additional finger properties here... + }, + // Additional objects here... +] + +``` ## Getting Started -Ready to give it a try? Feel free to follow our instructions to build your first Handpose project! +Ready to give it a try? Our demo is here to give you a sneak peek into what Handpose can do! Don't hesitate to follow along with our instructions to kickstart your very own Handpose project! ### Demo [p5 Web Editor](iframes/handpose-keypoints ':include :type=iframe width=100% height=550px') @@ -191,4 +222,6 @@ handpose.detect(media, ?callback); - **callback(output, error)**: OPTIONAL. A callback function to handle the output of the estimation, see output example above. **Returns:** -A promise that resolves to the estimation output. \ No newline at end of file +A promise that resolves to the estimation output. + +
diff --git a/docs/reference/iframes/handpose-keypoints/main.html b/docs/reference/iframes/handpose-keypoints/main.html index fc9afb1..764d6aa 100644 --- a/docs/reference/iframes/handpose-keypoints/main.html +++ b/docs/reference/iframes/handpose-keypoints/main.html @@ -2,7 +2,7 @@ - + diff --git a/docs/reference/iframes/pose-estimation/main.html b/docs/reference/iframes/pose-estimation/main.html index fc9afb1..764d6aa 100644 --- a/docs/reference/iframes/pose-estimation/main.html +++ b/docs/reference/iframes/pose-estimation/main.html @@ -2,7 +2,7 @@ - +