This project is a part of Master's Computer Vision course at Innopolis University
Our project is a set of methods and algorithms for working with a manipulator, on which an RGBD camera and a grip are attached, which allows you to capture objects. In this project we are focusing on the following tasks:
- Table Position Calibration : Since the table under the camera may have a certain angle, it is necessary to automatically determine the angle of inclination and other parameters.
- Object detection : Using a depth map and a color image allows you to determine the number and size of objects, as well as their position and orientation relative to the table coordinate system.
- Object recognition: Using pre-trained YOLO will allow us to recognize objects on the table, which will make it possible to give human-readable commands to the manipulator.
| Weeks | Task |
|---|---|
| 1 | Installation of the required drivers |
| 1 | Table Calibration |
| 2 | Object Detection |
| 2 | Object Recognition |
- ros noetic
- python:
- ultralytics==8.0.20
- pyrealsense2
- opencv-python
System recieve video stream and detect edges and depth map:
As an output, logs are sent to the console:
{
"Data:": [
{
"Name": "apple",
"Bounding_box": [
267.0,
194.0,
505.0,
423.0
],
"Center_in_pixels": [
386,
308
],
"Height_in_cm": 5.4,
"Real_coords": [
246.1999969482422,
70.09951782226562,
15.287549018859863
]
},
{
"Name": "apple",
"Bounding_box": [
676.0,
260.0,
940.0,
518.0
],
"Center_in_pixels": [
808,
389
],
"Height_in_cm": 6.5,
"Real_coords": [
243.0,
-42.28042221069336,
-6.351202011108398
]
}
]
}