FeatureDetection
class FeatureDetection
A class that contains computer vision methods that are used in the project
Variables
| Name | Description |
|---|---|
| K_int | 3*3 intrinsic matrix of the RGB lens |
| Lbb | Parameters of the designed markers |
| Lbp | Distance between ball2 and pitch (m) |
| fx | Intrinsic parameters |
Functions
| Name | Description |
|---|---|
| FeatureDetection | Construct a new Feature Detection object |
| ~FeatureDetection | Destroy the Feature Detection object |
| ReadIntrinsicMatrix | Read in the intrinsic matrix of the RGB lens |
| ReadPointCloudPCD | Read in pcd file and save it as a black and white image, where white pixels correspond to points that have depth values |
| ReconstructJ4Position | a global function for finding joint 4 position in the camera frame |
| ReadHandEyeTransform | Read 4*4 homogeneous hand-eye transformation matrix |
| ReadAcusenseDepth2RGBMat | Read extrinsic matrix that transforms from the RGB frame to the depth frame |
| cvtDepth2Colour_Acusense | convert a 3D point in the depth frame to the RGB frame |
| drawShaftAxisColourAcusense | Overlay the tool shaft axis on the colour image |
| cvt2cameraFrame | convert a 3D point the robot base frame to the depth frame |
| ContourDetection | Detect contours in the black and white image |
| Fit3DSphere | Find out the 3D position of the centre of a spherical marker |
| FindBallCentres | Find out the 3D positions of the centre of two spherical markers |
| world2pixel_Acusense | project a 3D point in the depth frame to the RGB image plane |
| world2pixel | project a 3D point in the depth frame to the depth image plane |
Variable Details
K_int
Eigen::Matrix3d K_int
3*3 intrinsic matrix of the RGB lens
Lbb
double Lbb
Parameters of the designed markers
- Lbb: Distance between ball1 and ball2 in meters
- Lbp: Distance between ball2 and pitch in meters
Distance between ball1 and ball2 (m)
Lbp
double Lbp
Distance between ball2 and pitch (m)
fx
double fx, fy, cx, cy
Intrinsic parameters
Function Details
ContourDetection
void ContourDetection(const cv::Mat image, int min_area, std::vector<cv::Mat> &contour_img_list)
Detect contours in the black and white image
- Parameter
image - input black and white image
- Parameter
min_area - minimum area of the contour to be detected
- Parameter
contour_img_list - vector of output images of infilled detected contours.
FeatureDetection
FeatureDetection()
Construct a new Feature Detection object
FindBallCentres
void FindBallCentres(cv::Mat img, PointCloudT::Ptr cloud, double radius_ball_1, double radius_ball_2, Eigen::Vector3d ¢re_ball_1, Eigen::Vector3d ¢re_ball_2)
Find out the 3D positions of the centre of two spherical markers
- Parameter
img - input black and white image
- Parameter
cloud - input point cloud
- Parameter
radius_ball_1 - radius of the larger marker, unit (mm)
- Parameter
radius_ball_2 - radius of the smaller marker, unit (mm)
- Parameter
centre_ball_1 - output 3D position of the centre of the larger marker in the depth frame, unit (mm)
- Parameter
centre_ball_2 - output 3D position of the centre of the smaller marker in the depth frame, unit (mm)
Fit3DSphere
void Fit3DSphere(const std::vector<cv::Point3d> &pt_list, double &xc, double &yc, double &zc, double &rc)
Find out the 3D position of the centre of a spherical marker
- Parameter
pt_list - vector of 3d points on the surface of the marker
- Parameter
xc - output x coordinate of the centre of the marker, unit (mm)
- Parameter
yc - output y coordinate of the centre of the marker, unit (mm)
- Parameter
zc - output z coordinate of the centre of the marker, unit (mm)
- Parameter
rc - output radius of the marker, unit (mm)
ReadAcusenseDepth2RGBMat
void ReadAcusenseDepth2RGBMat(const Eigen::Matrix4d &T_d_rgb)
Read extrinsic matrix that transforms from the RGB frame to the depth frame
- Parameter
T_d_rgb - extrinsic matrix
ReadHandEyeTransform
void ReadHandEyeTransform(const Eigen::Matrix4d &T)
Read 4*4 homogeneous hand-eye transformation matrix
- Parameter
T - 4*4 homogeneous transformation matrix
ReadIntrinsicMatrix
void ReadIntrinsicMatrix(const Eigen::MatrixXd &Intrinsic_mat)
Read in the intrinsic matrix of the RGB lens
- Parameter
Intrinsic_mat - 3*3 camera intrinsic matrix
ReadPointCloudPCD
void ReadPointCloudPCD(const std::string &file_path, const int &n_rows, const int &n_cols, cv::Mat &img, PointCloudT::Ptr &cloud)
Read in pcd file and save it as a black and white image, where white pixels correspond to points that have depth values
- Parameter
file_path - path to the pcd file
- Parameter
n_rows - number of rows of the image
- Parameter
n_cols - number of columns of the image
- Parameter
img - output image
- Parameter
cloud - output point cloud pointer
ReconstructJ4Position
Eigen::Vector3d ReconstructJ4Position(const cv::Mat &img_depth, const PointCloudT::Ptr &cloud, const double &ball_1_radius, const double &ball_2_radius)
a global function for finding joint 4 position in the camera frame
- Parameter
img_depth - black and image where white only pixels have depth values (unit (mm))
- Parameter
cloud - input point cloud pointer
- Parameter
ball_1_radius - radius of the larger spherical marker, unit (mm)
- Parameter
ball_2_radius - radius of the smaller spherical marker, unit (mm)
- Return
- Eigen::Vector3d output reconstructed 3d position of joint 4 in the camera frame, unit (mm)
cvt2cameraFrame
void cvt2cameraFrame(const Eigen::Vector3d &point_robot, Eigen::Vector3d &point_camera)
convert a 3D point the robot base frame to the depth frame
- Parameter
point_robot - input position of a 3D point in the robot base frame
- Parameter
point_camera - output position of the 3D point in the depth frame
cvtDepth2Colour_Acusense
void cvtDepth2Colour_Acusense(const Eigen::Vector3d &pt_depth, Eigen::Vector3d &pt_colour)
convert a 3D point in the depth frame to the RGB frame
- Parameter
pt_depth - input position of a 3D point in the depth frame
- Parameter
pt_colour - output position of the 3D point in the RGB frame
drawShaftAxisColourAcusense
cv::Mat drawShaftAxisColourAcusense(cv::Mat img, std::string window_name, const Eigen::Vector3d &origin, const Eigen::Vector3d &end_pt)
Overlay the tool shaft axis on the colour image
- Parameter
img - Colour image
- Parameter
window_name - name of the window
- Parameter
origin - 3D position of the starting point on the tool shaft, expressed in the depth frame
- Parameter
end_pt - 3D position of the ending point on the tool shaft, expressed in the depth frame
world2pixel
void world2pixel(const Eigen::Vector3d &point_3D, Eigen::Vector2d &pixel_2D)
project a 3D point in the depth frame to the depth image plane
- Parameter
point_3D - input 3D position of the point
- Parameter
pixel_2D - output pixel coordinates of the back-projected 3D point in the depth image plane
world2pixel_Acusense
void world2pixel_Acusense(const Eigen::Vector3d &point_3D, Eigen::Vector2d &pixel_2D)
project a 3D point in the depth frame to the RGB image plane
- Parameter
point_3D - input 3D position of the point
- Parameter
pixel_2D - output pixel coordinates of the back-projected 3D point in the RGB image plane
~FeatureDetection
~FeatureDetection()
Destroy the Feature Detection object