This repository was archived by the owner on Feb 5, 2022. It is now read-only.
Repository navigation
Vision
Amar Maksumić edited this page Feb 24, 2020
·
52 revisions
- Stream high fps and resolution video to the driver station
- Detect the loading bay, power port, and balls and send distance values to the RoboRio
- Pipeline gstreamer issues in C++
- 1/6/2020: Pre-build issues continue, spurs change to Python for Jetson Nano for ease of access but mostly everyone is new to python
- Lacked method to power vision lights and use them to test for object detection
- Frame rate problems - SOLVED through setting cap properties-2nd gstreamer link
- "Failed to load 'canberra-gtk-module' " during gstreamer - SOLVED by tinker with the tkinter settings
- Error with decoding h264 encoded video from x264enc - SOLVED by converting to omxh264enc, and switching to avdec_h264 and h264depay for decoding
- OpenCv Ubuntu errors, reinstall through link in resources, will improve vision speed too
- Refined last years vision prototype that was in python to work on power port and loading bay, now also checks for vertices as well (1/5/2020)
- Can open camera in python (1/5/2020)
- Updated last years python vision code to use python 3 (1/6/2020)
- Fixed frame rate by setting cap properties - 2nd gstreamer link(1/7/20)
- network tables and webpage dashboard set up (1/9/20)
- got gstreamer running (1/9/20)
- Decide and get documentation for Flask vs Gstreamer for our current set-up and choose one for driver station streaming
- Create networktables
- Still investigating issues with framerate on Jetson Nano, find cause (GPU?)
- Object detection for power port, ball, and loading bay(focusing on tensorflow but one person can still work on adapting and refining previous methods of vision detection just in case)
- Calculate target distance and needed robot adjustments
- start pseudo coding algorithms for vision detection - 10:00 - DONE
- test algorithms on laptop - 7:00 - DONE
- start working on jetson nano, switch to python - 7:00 - DONE
- get GStreamer OR Flask running - 7:30 - RAN INTO H264 DECODE ERROR
- adapt last year's cargo vision code to work for power cell - 7:30 - NEED TO TEST
- fix area detection on object detection (currently going in the thousand - 7:30 - DONE
- test ball vision code and refine it- 5:30 - DONE
- create a method to convert pixels into inches - 7:30 - DONE
- get something streaming to the driver station (GStreamer/Flask) - 7:30 - RAN INTO H264 DECODE ERROR
- research FRC dashboard - Ongoing
- get object detection to stop displaying width as height and vice versa - 4:00 DONE
- get h264 streaming, encoding, and decoding working with omxh264enc- 5:00 - DONE
- setup picture vision testing - 5:00 - DONE
- create separate color masks for ball and reflective objects - 2:00 - DONE
- fet h265 streaming, encoding, and decoding working with omxh265enc - 5:00 - DONE (9:25 AM)
- integrate gstreamer code with Weijian's network table code - 5:00 - DONE
- try out ball detection on picture of power cell and get it to consistent recognize it - 6:00 - MOVED TO TOMMOROW TO TEST ON VIDEO THROUGH JETSON
- work on custom driver dashboard to toggle settings - 7:00 - DONE
- fix bay input starting port detection - 10:30 - DONE
- fix errors with inputs on vision webpage (currently inversed) - 11:00 - DONE
- get object detections methods on jetson and test - 3:00 - DONE
- refine ball calibration settings - 8:00 - Ongoing
- refine ball calibration settings - 5:00 - NOT DONE
- hit 120fps vision processing - 6:00 - DONE
- push data to roborio over networktables - 7:30 - NOT DONE
- push data to roborio over networktables - 1:00 - ERROR WITH JSON PACKAGES
- develop animation interface as backup to video streaming - 3:00 - DONE, FOR POWER CELL
- alignment method - 4:00 - CODED, NEEDS TO BE TESTED
- fix distance inch pixel conversion for new camera - 5:00 - FINISHING CALCULATIONS
- create method to detect real object dimensions regardless of distance - 7:00 - DONE
- push data to roborio over networktables - 2:00 - DONE
- develop animation interface as backup to video streaming (for everything but power cell) - 3:00 - NOT DONE
- alignment method - 4:00 - NOT DONE
- fix distance inch pixel conversion for new camera - 5:00 - NOT DONE
- finish alignment method for static alignment - 11:00 - DONE
- install and build new OpenCV - 4:30 - CONTINUING
- finish alignment method for c-curve alignment - 2:00 - DONE
- get exposure and contrast working on ELP camera - 3:00 - DONE
- install and build new OpenCV - 4:30 - DONE
- create animations for hex and port - 4:30 - DONE
- refine calibration - 4:00 - DONE
- finish up align code - 4:00 - DONE, need to test
- create and coin vision-pixel units - 10:00 - DONE
- work on static align - 3:00 - DONE, always strays to the left
- work on curve align - 4:30 - WIP, always starts far to the left
- fix static align (dual stage alignment, stop at a certain radius and re-align)- DONE
- test out alignment with loading bay and power port - DONE
- calibrate port detection to extend to 30 ft - DONE
- fix distance inch pixel conversion for new camera - 5:00 - DONE
- calibrate ball detection for more lighting conditions - git stash kinda deleted all our code...
- test alignment (c-curve) more - git stash kinda deleted all our code...
- start vision upon bootup - DONE
- restore hali's code :( - DONE
- recalibrate hsv values and test alignment- NOT DONE
- merge Weijian's code with local vision directory - DONE
- test fix for image filtering - Friday 1:00
- calibrate values for object detection - Friday - 3:00
- find out best camera lens for long range port detection - Friday 5:00
- test alignment - Rest of day
- fix animations-12:00-DONE
- find best lens for long range port detection - 2:00 - DONE
- test alignment-rest of day - Tuesday
- replace network tables with websockets completely done - DONE
- create dual camera support - DONE
- calibrate settings for long range camera -DONE
- test alignment - DONE
- add button to swap camera ports - DONE
- add vision start method upon robot bootup - 1:00 - DONE
- create new conversion formula for 3.6mm lens camera - 2:00 - DONE
- separate alignment for different camera modes - 4:00 - DONE
- turn off and on light according to camera mode - 5:00 - DONE
- work on integrating shooter and vision code - rest of day
- bullfight with the robot! - rest of day
- work on all types of alignment (ball, bay, and port) - DONE (very, very aggressive alignment)
- finish final tunings for chassis - DONE
- tune alignment for when the cameras are not in the center of the robot - DONE
- integrate HawkSight Vizion into PathCorder - NOT DONE (waiting on Auton)
- integrate HawkSight Vizion into PathCorder - NOT DONE (Auton has issues with long paths)
Gstreamer
- https://gstreamer.freedesktop.org/documentation/?gi-language=c
- https://github.com/madams1337/python-opencv-gstreamer-examples
Tensorflow
Network Tables
Websockets Java
Tornado (Jetson side)
OpenCv
FRC Team 2601 | Steel Hawks
steelhawks.net
Make sure to check out the 2020 20/20 Vision Repo and Wiki as well!
FRC Team 2601 | Steel Hawks
steelhawks.net