Skip to content

Commit

Permalink
Quinn.update handpose model page (#38)
Browse files Browse the repository at this point in the history
* update ml5 lib path

* update model description
  • Loading branch information
QuinnHe authored Feb 1, 2024
1 parent 2e364e8 commit 20fe522
Show file tree
Hide file tree
Showing 4 changed files with 43 additions and 7 deletions.
5 changes: 4 additions & 1 deletion docs/reference/facemesh.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,10 @@ Facemesh is a machine-learning model that allows for facial landmark detection i

The ml5.js Facemesh model is ported from the [TensorFlow.js Facemesh implementation](https://github.com/tensorflow/tfjs-models/tree/master/face-landmarks-detection).

## Quickstart
## Getting Started
Ready to give it a try? Just follow our simple instructions, and you'll be on your way to creating your very own Facemesh project in no time!

### Quickstart

```js
let predictions = [];
Expand Down
41 changes: 37 additions & 4 deletions docs/reference/handpose.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,43 @@

## Description

Handpose is a machine-learning model that allows for palm detection and hand-skeleton finger tracking in the browser. It can detect a maximum of one hand at a time and provides 21 3D hand keypoints that describe important locations on the palm and fingers.
Handpose is a machine-learning model that allows for palm detection and hand-skeleton finger tracking in the browser. It can detect multiple hands at a time and for each hand, provides 21 3D hand keypoints that describe important locations on the palm and fingers.

The ml5.js Handpose model is ported from the [TensorFlow.js Handpose implementation](https://github.com/tensorflow/tfjs-models/tree/master/handpose).
The ml5.js Handpose model is ported from the [TensorFlow.js Mediapipe Handpose implementation](https://github.com/tensorflow/tfjs-models/tree/master/hand-pose-detection).

#### Key Features
- **Palm Detection**: Handpose can detect the palm of a hand and provide the 2D and 3D coordinates of 21 keypoints on the hand.
- **Finger Tracking**: Handpose can track the 3D coordinates of the tips and joints of the fingers.
- **Handedness**: Handpose can determine the handedness (left or right) of the detected hand.
- **Multiple Hands**: Handpose can detect multiple hands at the same time.

#### Output Example
Imagine we detected two hand poses in a frame. The output would look like this:

```javascript
[
{
score: 0.86,
handedness: "Left",
keypoints: [
{ x: 623.57, y: 374.79, score: 0.85, name: "wrist" },
// Additional keypoints here...
],
keypoints3D: [
{ x: 0.0024, y: 0.070, z: 0.035, score: 0.85, name: "wrist" },
// Additional 3D keypoints here...
],
index_finger_dip: { x: /* value */, y: /* value */, x3D: /* value */, y3D: /* value */, z3D: /* value */ },
index_finger_mcp: { x: /* value */, y: /* value */, x3D: /* value */, y3D: /* value */, z3D: /* value */ },
// Additional finger properties here...
},
// Additional objects here...
]

```

## Getting Started
Ready to give it a try? Feel free to follow our instructions to build your first Handpose project!
Ready to give it a try? Our demo is here to give you a sneak peek into what Handpose can do! Don't hesitate to follow along with our instructions to kickstart your very own Handpose project!

### Demo
[p5 Web Editor](iframes/handpose-keypoints ':include :type=iframe width=100% height=550px')
Expand Down Expand Up @@ -191,4 +222,6 @@ handpose.detect(media, ?callback);
- **callback(output, error)**: OPTIONAL. A callback function to handle the output of the estimation, see output example above.
**Returns:**
A promise that resolves to the estimation output.
A promise that resolves to the estimation output.
<br>
2 changes: 1 addition & 1 deletion docs/reference/iframes/handpose-keypoints/main.html
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<html lang="en">

<head>
<script src="https://unpkg.com/ml5@latest/dist/ml5.min.js"></script>
<script src="https://unpkg.com/ml5@alpha/dist/ml5.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.0/p5.js"></script>
<!-- <script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.0/addons/p5.sound.min.js"></script> -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/dat-gui/0.7.9/dat.gui.min.js"></script>
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/iframes/pose-estimation/main.html
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<html lang="en">

<head>
<script src="https://unpkg.com/ml5@latest/dist/ml5.min.js"></script>
<script src="https://unpkg.com/ml5@alpha/dist/ml5.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.0/p5.js"></script>
<!-- <script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.0/addons/p5.sound.min.js"></script> -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/dat-gui/0.7.9/dat.gui.min.js"></script>
Expand Down

0 comments on commit 20fe522

Please sign in to comment.