-
Notifications
You must be signed in to change notification settings - Fork 287
Add section on obtaining vision measurements for pose estimation #3137
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Add section on obtaining vision measurements for pose estimation #3137
Conversation
Adds comprehensive "Obtaining Vision Measurements" section explaining how to get robot pose estimates from vision systems like PhotonVision and Limelight for use with AddVisionMeasurement(). Fixes wpilibsuite#1413
source/docs/software/advanced-controls/state-space/state-space-pose-estimators.rst
Outdated
Show resolved
Hide resolved
|
Thinking about this more, I think most of this probably belongs in the vision processing docs, rather then here. |
Per review feedback, moved the detailed vision measurement information from the pose estimator docs to a new article in the vision processing section. Changes: - Created new apriltag-pose-estimation.rst in vision-processing/apriltag/ - Comprehensive guide covering PhotonVision, Limelight, and custom solutions - Detailed sections on timestamps, standard deviations, and rejecting bad measurements - Code examples in Java, C++, and Python for both libraries - Updated apriltag/index.rst to include new article - Replaced detailed section in state-space-pose-estimators.rst with a seealso link The pose estimator docs now focus on the API usage, while the vision processing docs explain how to obtain the measurements. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Changed links in "See Also" section to anonymous hyperlinks (double underscores) to avoid conflicts with similar link text used earlier in the document. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
…d-layout file Removed references to apriltag-field-layout.rst which doesn't exist in this PR: - Removed from toctree in index.rst - Removed from "See Also" section in apriltag-pose-estimation.rst The apriltag-field-layout article is being created in PR wpilibsuite#3138. Once that PR merges, the references can be added back. This fixes the build errors: - "toctree contains reference to nonexisting document" - "unknown document: 'apriltag-field-layout'" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
source/docs/software/vision-processing/apriltag/apriltag-pose-estimation.rst
Outdated
Show resolved
Hide resolved
source/docs/software/vision-processing/apriltag/apriltag-pose-estimation.rst
Show resolved
Hide resolved
source/docs/software/vision-processing/apriltag/apriltag-pose-estimation.rst
Outdated
Show resolved
Hide resolved
source/docs/software/vision-processing/apriltag/apriltag-pose-estimation.rst
Outdated
Show resolved
Hide resolved
source/docs/software/vision-processing/apriltag/apriltag-pose-estimation.rst
Outdated
Show resolved
Hide resolved
source/docs/software/vision-processing/apriltag/apriltag-pose-estimation.rst
Outdated
Show resolved
Hide resolved
- Add setup instructions for PhotonVision/Limelight AprilTag detection - Fix RST formatting (blank line before bullet list) - Fix standard deviation formula using AdvantageKit approach with proper baseline values (0.02m xy, 0.06rad theta) scaled by distance²/tagCount - Add conditional to use MAX_VALUE for theta with single tag (ambiguous) - Clarify that rejection example only shows field boundary checking - Add 0.5m tolerance margin for edge-of-field measurements 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Gold856
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me, just some minor changes to match current photonlib.
|
|
||
| ## Using Vision Libraries | ||
|
|
||
| Most teams use existing vision processing libraries that handle the complex mathematics for you. Before using these libraries in your robot code, you'll need to configure the vision coprocessor to detect AprilTags. See the `PhotonVision AprilTag documentation <https://docs.photonvision.org/en/latest/docs/apriltag-pipelines/index.html>`__ or `Limelight AprilTag documentation <https://docs.limelightvision.io/docs/docs-limelight/pipeline-apriltag/apriltag-introduction>`__ for setup instructions. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LL link is broken.
| camera, // PhotonCamera | ||
| robotToCam // Transform3d from robot to camera | ||
| ); | ||
| // In your periodic method | ||
| var result = photonPoseEstimator.update(); | ||
| if (result.isPresent()) { | ||
| var estimatedPose = result.get(); | ||
| poseEstimator.addVisionMeasurement( | ||
| estimatedPose.estimatedPose.toPose2d(), | ||
| estimatedPose.timestampSeconds | ||
| ); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| camera, // PhotonCamera | |
| robotToCam // Transform3d from robot to camera | |
| ); | |
| // In your periodic method | |
| var result = photonPoseEstimator.update(); | |
| if (result.isPresent()) { | |
| var estimatedPose = result.get(); | |
| poseEstimator.addVisionMeasurement( | |
| estimatedPose.estimatedPose.toPose2d(), | |
| estimatedPose.timestampSeconds | |
| ); | |
| } | |
| robotToCam // Transform3d from robot to camera | |
| ); | |
| // In your periodic method | |
| for (var result : camera.getAllUnreadResults()) { | |
| var estimate = photonPoseEstimator.update(result); | |
| if (estimate.isPresent()) { | |
| var estimatedPose = estimate.get(); | |
| poseEstimator.addVisionMeasurement( | |
| estimatedPose.estimatedPose.toPose2d(), | |
| estimatedPose.timestampSeconds | |
| ); | |
| } | |
| } |
| ) | ||
| # In your periodic method | ||
| result = photon_pose_estimator.update() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll PR an update to this when we've updated the API.
| camera, // photon::PhotonCamera | ||
| robotToCam // frc::Transform3d from robot to camera | ||
| }; | ||
| // In your periodic method | ||
| auto result = photonPoseEstimator.Update(); | ||
| if (result) { | ||
| poseEstimator.AddVisionMeasurement( | ||
| result->estimatedPose.ToPose2d(), | ||
| result->timestamp | ||
| ); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| camera, // photon::PhotonCamera | |
| robotToCam // frc::Transform3d from robot to camera | |
| }; | |
| // In your periodic method | |
| auto result = photonPoseEstimator.Update(); | |
| if (result) { | |
| poseEstimator.AddVisionMeasurement( | |
| result->estimatedPose.ToPose2d(), | |
| result->timestamp | |
| ); | |
| } | |
| robotToCam // frc::Transform3d from robot to camera | |
| }; | |
| // In your periodic method | |
| for (const auto& result : camera.GetAllUnreadResults()) { | |
| auto estimate = photonPoseEstimator.Update(result); | |
| if (estimate) { | |
| poseEstimator.AddVisionMeasurement( | |
| estimate->estimatedPose.ToPose2d(), | |
| estimate->timestamp | |
| ); | |
| } | |
| } |
| var result = photonPoseEstimator.update(); | ||
| if (result.isPresent()) { | ||
| var estimatedPose = result.get(); | ||
| // Check if pose is reasonable (within field boundaries with tolerance) | ||
| double margin = 0.5; // meters of tolerance for edge measurements | ||
| if (estimatedPose.estimatedPose.getX() >= -margin && | ||
| estimatedPose.estimatedPose.getX() <= fieldLayout.getFieldLength() + margin && | ||
| estimatedPose.estimatedPose.getY() >= -margin && | ||
| estimatedPose.estimatedPose.getY() <= fieldLayout.getFieldWidth() + margin) { | ||
| poseEstimator.addVisionMeasurement( | ||
| estimatedPose.estimatedPose.toPose2d(), | ||
| estimatedPose.timestampSeconds | ||
| ); | ||
| } | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| var result = photonPoseEstimator.update(); | |
| if (result.isPresent()) { | |
| var estimatedPose = result.get(); | |
| // Check if pose is reasonable (within field boundaries with tolerance) | |
| double margin = 0.5; // meters of tolerance for edge measurements | |
| if (estimatedPose.estimatedPose.getX() >= -margin && | |
| estimatedPose.estimatedPose.getX() <= fieldLayout.getFieldLength() + margin && | |
| estimatedPose.estimatedPose.getY() >= -margin && | |
| estimatedPose.estimatedPose.getY() <= fieldLayout.getFieldWidth() + margin) { | |
| poseEstimator.addVisionMeasurement( | |
| estimatedPose.estimatedPose.toPose2d(), | |
| estimatedPose.timestampSeconds | |
| ); | |
| } | |
| } | |
| for (var result : camera.getAllUnreadResults()) { | |
| var estimate = photonPoseEstimator.update(result); | |
| if (estimate.isPresent()) { | |
| var estimatedPose = estimate.get(); | |
| // Check if pose is reasonable (within field boundaries with tolerance) | |
| double margin = 0.5; // meters of tolerance for edge measurements | |
| if (estimatedPose.estimatedPose.getX() >= -margin && | |
| estimatedPose.estimatedPose.getX() <= fieldLayout.getFieldLength() + margin && | |
| estimatedPose.estimatedPose.getY() >= -margin && | |
| estimatedPose.estimatedPose.getY() <= fieldLayout.getFieldWidth() + margin) { | |
| poseEstimator.addVisionMeasurement( | |
| estimatedPose.estimatedPose.toPose2d(), | |
| estimatedPose.timestampSeconds | |
| ); | |
| } | |
| } | |
| } |
| auto result = photonPoseEstimator.Update(); | ||
| if (result) { | ||
| // Check if pose is reasonable (within field boundaries with tolerance) | ||
| units::meter_t margin = 0.5_m; // tolerance for edge measurements | ||
| if (result->estimatedPose.X() >= -margin && | ||
| result->estimatedPose.X() <= fieldLayout.GetFieldLength() + margin && | ||
| result->estimatedPose.Y() >= -margin && | ||
| result->estimatedPose.Y() <= fieldLayout.GetFieldWidth() + margin) { | ||
| poseEstimator.AddVisionMeasurement( | ||
| result->estimatedPose.ToPose2d(), | ||
| result->timestamp | ||
| ); | ||
| } | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| auto result = photonPoseEstimator.Update(); | |
| if (result) { | |
| // Check if pose is reasonable (within field boundaries with tolerance) | |
| units::meter_t margin = 0.5_m; // tolerance for edge measurements | |
| if (result->estimatedPose.X() >= -margin && | |
| result->estimatedPose.X() <= fieldLayout.GetFieldLength() + margin && | |
| result->estimatedPose.Y() >= -margin && | |
| result->estimatedPose.Y() <= fieldLayout.GetFieldWidth() + margin) { | |
| poseEstimator.AddVisionMeasurement( | |
| result->estimatedPose.ToPose2d(), | |
| result->timestamp | |
| ); | |
| } | |
| } | |
| for (const auto& result : camera.GetAllUnreadResults()) { | |
| auto estimate = photonPoseEstimator.Update(result); | |
| if (estimate) { | |
| // Check if pose is reasonable (within field boundaries with tolerance) | |
| units::meter_t margin = 0.5_m; // tolerance for edge measurements | |
| if (estimate->estimatedPose.X() >= -margin && | |
| estimate->estimatedPose.X() <= fieldLayout.GetFieldLength() + margin && | |
| estimate->estimatedPose.Y() >= -margin && | |
| estimate->estimatedPose.Y() <= fieldLayout.GetFieldWidth() + margin) { | |
| poseEstimator.AddVisionMeasurement( | |
| estimate->estimatedPose.ToPose2d(), | |
| estimate->timestamp | |
| ); | |
| } | |
| } | |
| } |
|
|
||
| Most vision libraries provide this timestamp: | ||
|
|
||
| - PhotonVision: ``result.timestampSeconds`` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| - PhotonVision: ``result.timestampSeconds`` | |
| - PhotonVision: ``estimate.timestampSeconds`` |
Summary
Creates a comprehensive guide for using AprilTags for robot pose estimation, addressing the lack of documentation on obtaining vision measurements for pose estimators.
Changes
New file:
apriltag-pose-estimation.rstin vision processing sectionModified:
state-space-pose-estimators.rstModified:
apriltag/index.rstOrganization
Per review feedback, vision measurement content now lives in vision processing docs (where teams look for vision guidance) rather than pose estimator docs (which focus on the API).
Fixes #1413