Skip to content

Conversation

@jasondaming
Copy link
Member

@jasondaming jasondaming commented Oct 8, 2025

Summary

Creates a comprehensive guide for using AprilTags for robot pose estimation, addressing the lack of documentation on obtaining vision measurements for pose estimators.

Changes

New file: apriltag-pose-estimation.rst in vision processing section

  • Explains the complete process: detect tags → get tag pose → calculate robot pose → apply measurement
  • Documents PhotonVision's PhotonPoseEstimator with code examples in Java, C++, and Python
  • Documents Limelight's MegaTag with NetworkTables API usage
  • Important sections covering:
    • Timestamp handling (using capture time, not processing time)
    • Standard deviation scaling based on distance, tag count, and ambiguity
    • Rejecting bad measurements (no tags, high ambiguity, unrealistic poses)
  • Guidance for custom vision implementations

Modified: state-space-pose-estimators.rst

  • Replaced detailed vision measurement section with concise seealso link
  • Keeps pose estimator docs focused on the API, refers to vision docs for obtaining measurements

Modified: apriltag/index.rst

  • Added new article to table of contents

Organization

Per review feedback, vision measurement content now lives in vision processing docs (where teams look for vision guidance) rather than pose estimator docs (which focus on the API).

Fixes #1413

Adds comprehensive "Obtaining Vision Measurements" section explaining how to get robot pose estimates from vision systems like PhotonVision and Limelight for use with AddVisionMeasurement().

Fixes wpilibsuite#1413
@sciencewhiz
Copy link
Collaborator

Thinking about this more, I think most of this probably belongs in the vision processing docs, rather then here.

jasondaming and others added 3 commits October 13, 2025 20:32
Per review feedback, moved the detailed vision measurement information from the pose estimator docs to a new article in the vision processing section.

Changes:
- Created new apriltag-pose-estimation.rst in vision-processing/apriltag/
- Comprehensive guide covering PhotonVision, Limelight, and custom solutions
- Detailed sections on timestamps, standard deviations, and rejecting bad measurements
- Code examples in Java, C++, and Python for both libraries
- Updated apriltag/index.rst to include new article
- Replaced detailed section in state-space-pose-estimators.rst with a seealso link

The pose estimator docs now focus on the API usage, while the vision processing docs explain how to obtain the measurements.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Changed links in "See Also" section to anonymous hyperlinks (double
underscores) to avoid conflicts with similar link text used earlier in
the document.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
…d-layout file

Removed references to apriltag-field-layout.rst which doesn't exist in this PR:
- Removed from toctree in index.rst
- Removed from "See Also" section in apriltag-pose-estimation.rst

The apriltag-field-layout article is being created in PR wpilibsuite#3138. Once that
PR merges, the references can be added back.

This fixes the build errors:
- "toctree contains reference to nonexisting document"
- "unknown document: 'apriltag-field-layout'"

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
- Add setup instructions for PhotonVision/Limelight AprilTag detection
- Fix RST formatting (blank line before bullet list)
- Fix standard deviation formula using AdvantageKit approach with proper
  baseline values (0.02m xy, 0.06rad theta) scaled by distance²/tagCount
- Add conditional to use MAX_VALUE for theta with single tag (ambiguous)
- Clarify that rejection example only shows field boundary checking
- Add 0.5m tolerance margin for edge-of-field measurements

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Copy link
Contributor

@Gold856 Gold856 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, just some minor changes to match current photonlib.


## Using Vision Libraries

Most teams use existing vision processing libraries that handle the complex mathematics for you. Before using these libraries in your robot code, you'll need to configure the vision coprocessor to detect AprilTags. See the `PhotonVision AprilTag documentation <https://docs.photonvision.org/en/latest/docs/apriltag-pipelines/index.html>`__ or `Limelight AprilTag documentation <https://docs.limelightvision.io/docs/docs-limelight/pipeline-apriltag/apriltag-introduction>`__ for setup instructions.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LL link is broken.

Comment on lines +32 to +44
camera, // PhotonCamera
robotToCam // Transform3d from robot to camera
);
// In your periodic method
var result = photonPoseEstimator.update();
if (result.isPresent()) {
var estimatedPose = result.get();
poseEstimator.addVisionMeasurement(
estimatedPose.estimatedPose.toPose2d(),
estimatedPose.timestampSeconds
);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
camera, // PhotonCamera
robotToCam // Transform3d from robot to camera
);
// In your periodic method
var result = photonPoseEstimator.update();
if (result.isPresent()) {
var estimatedPose = result.get();
poseEstimator.addVisionMeasurement(
estimatedPose.estimatedPose.toPose2d(),
estimatedPose.timestampSeconds
);
}
robotToCam // Transform3d from robot to camera
);
// In your periodic method
for (var result : camera.getAllUnreadResults()) {
var estimate = photonPoseEstimator.update(result);
if (estimate.isPresent()) {
var estimatedPose = estimate.get();
poseEstimator.addVisionMeasurement(
estimatedPose.estimatedPose.toPose2d(),
estimatedPose.timestampSeconds
);
}
}

)
# In your periodic method
result = photon_pose_estimator.update()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll PR an update to this when we've updated the API.

Comment on lines +54 to +65
camera, // photon::PhotonCamera
robotToCam // frc::Transform3d from robot to camera
};
// In your periodic method
auto result = photonPoseEstimator.Update();
if (result) {
poseEstimator.AddVisionMeasurement(
result->estimatedPose.ToPose2d(),
result->timestamp
);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
camera, // photon::PhotonCamera
robotToCam // frc::Transform3d from robot to camera
};
// In your periodic method
auto result = photonPoseEstimator.Update();
if (result) {
poseEstimator.AddVisionMeasurement(
result->estimatedPose.ToPose2d(),
result->timestamp
);
}
robotToCam // frc::Transform3d from robot to camera
};
// In your periodic method
for (const auto& result : camera.GetAllUnreadResults()) {
auto estimate = photonPoseEstimator.Update(result);
if (estimate) {
poseEstimator.AddVisionMeasurement(
estimate->estimatedPose.ToPose2d(),
estimate->timestamp
);
}
}

Comment on lines +262 to +278
var result = photonPoseEstimator.update();
if (result.isPresent()) {
var estimatedPose = result.get();
// Check if pose is reasonable (within field boundaries with tolerance)
double margin = 0.5; // meters of tolerance for edge measurements
if (estimatedPose.estimatedPose.getX() >= -margin &&
estimatedPose.estimatedPose.getX() <= fieldLayout.getFieldLength() + margin &&
estimatedPose.estimatedPose.getY() >= -margin &&
estimatedPose.estimatedPose.getY() <= fieldLayout.getFieldWidth() + margin) {
poseEstimator.addVisionMeasurement(
estimatedPose.estimatedPose.toPose2d(),
estimatedPose.timestampSeconds
);
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
var result = photonPoseEstimator.update();
if (result.isPresent()) {
var estimatedPose = result.get();
// Check if pose is reasonable (within field boundaries with tolerance)
double margin = 0.5; // meters of tolerance for edge measurements
if (estimatedPose.estimatedPose.getX() >= -margin &&
estimatedPose.estimatedPose.getX() <= fieldLayout.getFieldLength() + margin &&
estimatedPose.estimatedPose.getY() >= -margin &&
estimatedPose.estimatedPose.getY() <= fieldLayout.getFieldWidth() + margin) {
poseEstimator.addVisionMeasurement(
estimatedPose.estimatedPose.toPose2d(),
estimatedPose.timestampSeconds
);
}
}
for (var result : camera.getAllUnreadResults()) {
var estimate = photonPoseEstimator.update(result);
if (estimate.isPresent()) {
var estimatedPose = estimate.get();
// Check if pose is reasonable (within field boundaries with tolerance)
double margin = 0.5; // meters of tolerance for edge measurements
if (estimatedPose.estimatedPose.getX() >= -margin &&
estimatedPose.estimatedPose.getX() <= fieldLayout.getFieldLength() + margin &&
estimatedPose.estimatedPose.getY() >= -margin &&
estimatedPose.estimatedPose.getY() <= fieldLayout.getFieldWidth() + margin) {
poseEstimator.addVisionMeasurement(
estimatedPose.estimatedPose.toPose2d(),
estimatedPose.timestampSeconds
);
}
}
}

Comment on lines +282 to +296
auto result = photonPoseEstimator.Update();
if (result) {
// Check if pose is reasonable (within field boundaries with tolerance)
units::meter_t margin = 0.5_m; // tolerance for edge measurements
if (result->estimatedPose.X() >= -margin &&
result->estimatedPose.X() <= fieldLayout.GetFieldLength() + margin &&
result->estimatedPose.Y() >= -margin &&
result->estimatedPose.Y() <= fieldLayout.GetFieldWidth() + margin) {
poseEstimator.AddVisionMeasurement(
result->estimatedPose.ToPose2d(),
result->timestamp
);
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
auto result = photonPoseEstimator.Update();
if (result) {
// Check if pose is reasonable (within field boundaries with tolerance)
units::meter_t margin = 0.5_m; // tolerance for edge measurements
if (result->estimatedPose.X() >= -margin &&
result->estimatedPose.X() <= fieldLayout.GetFieldLength() + margin &&
result->estimatedPose.Y() >= -margin &&
result->estimatedPose.Y() <= fieldLayout.GetFieldWidth() + margin) {
poseEstimator.AddVisionMeasurement(
result->estimatedPose.ToPose2d(),
result->timestamp
);
}
}
for (const auto& result : camera.GetAllUnreadResults()) {
auto estimate = photonPoseEstimator.Update(result);
if (estimate) {
// Check if pose is reasonable (within field boundaries with tolerance)
units::meter_t margin = 0.5_m; // tolerance for edge measurements
if (estimate->estimatedPose.X() >= -margin &&
estimate->estimatedPose.X() <= fieldLayout.GetFieldLength() + margin &&
estimate->estimatedPose.Y() >= -margin &&
estimate->estimatedPose.Y() <= fieldLayout.GetFieldWidth() + margin) {
poseEstimator.AddVisionMeasurement(
estimate->estimatedPose.ToPose2d(),
estimate->timestamp
);
}
}
}


Most vision libraries provide this timestamp:

- PhotonVision: ``result.timestampSeconds``
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- PhotonVision: ``result.timestampSeconds``
- PhotonVision: ``estimate.timestampSeconds``

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pose estimator article doesn't show usage

3 participants