One of the big advantages of using ChAruco board is being able to interpolate corners, which I hope would help me calibrate cameras more accurately. However the issue is that the corners are not being interpolated properly near the edge of the image. How can I adjust the parameters so that the corners are found at the edge of the image?
The image is this:
The code used below:
cv::Ptr<cv::aruco::Dictionary> charuco_dictionary = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_4X4_50);
cv::Ptr<cv::aruco::DetectorParameters> charuco_params = cv::aruco::DetectorParameters::create();
cv::Ptr<cv::aruco::CharucoBoard> board = cv::aruco::CharucoBoard::create(7, 5, 35.0 / 1000, 26.0 / 1000, charuco_dictionary);
std::vector<int> image_ids, marker_ids;
vector<vector<cv::Point2f>> image_point, marker_point, rejected;
cv::Mat charuco_chess_corners, charuco_chess_corners_id;
cv::Mat input = cv::imread("image.png", cv::IMREAD_ANYCOLOR);
cv::flip(input, input, 1);
cv::aruco::detectMarkers(input, charuco_dictionary, marker_point, marker_ids, charuco_params, rejected);
cv::aruco::refineDetectedMarkers(input, board, marker_point, marker_ids, rejected);
if (!marker_ids.empty()) {
cv::aruco::interpolateCornersCharuco(marker_point, marker_ids, input, board, charuco_chess_corners, charuco_chess_corners_id,
cv::noArray(), cv::noArray(), 0);
}
cv::Mat out_color;
cv::cvtColor(input, out_color, cv::COLOR_GRAY2RGB);
cv::aruco::drawDetectedCornersCharuco(out_color, charuco_chess_corners, charuco_chess_corners_id);
cv::imshow("test", out_color);
cv::waitKey(0);
return 0;
And the output is this with the corner not interpolated:
One of the big advantages of using ChAruco board is being able to interpolate corners, which I hope would help me calibrate cameras more accurately. However the issue is that the corners are not being interpolated properly near the edge of the image. How can I adjust the parameters so that the corners are found at the edge of the image?
The image is this:
The code used below:
cv::Ptr<cv::aruco::Dictionary> charuco_dictionary = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_4X4_50);
cv::Ptr<cv::aruco::DetectorParameters> charuco_params = cv::aruco::DetectorParameters::create();
cv::Ptr<cv::aruco::CharucoBoard> board = cv::aruco::CharucoBoard::create(7, 5, 35.0 / 1000, 26.0 / 1000, charuco_dictionary);
std::vector<int> image_ids, marker_ids;
vector<vector<cv::Point2f>> image_point, marker_point, rejected;
cv::Mat charuco_chess_corners, charuco_chess_corners_id;
cv::Mat input = cv::imread("image.png", cv::IMREAD_ANYCOLOR);
cv::flip(input, input, 1);
cv::aruco::detectMarkers(input, charuco_dictionary, marker_point, marker_ids, charuco_params, rejected);
cv::aruco::refineDetectedMarkers(input, board, marker_point, marker_ids, rejected);
if (!marker_ids.empty()) {
cv::aruco::interpolateCornersCharuco(marker_point, marker_ids, input, board, charuco_chess_corners, charuco_chess_corners_id,
cv::noArray(), cv::noArray(), 0);
}
cv::Mat out_color;
cv::cvtColor(input, out_color, cv::COLOR_GRAY2RGB);
cv::aruco::drawDetectedCornersCharuco(out_color, charuco_chess_corners, charuco_chess_corners_id);
cv::imshow("test", out_color);
cv::waitKey(0);
return 0;
And the output is this with the corner not interpolated:
Share Improve this question edited Mar 17 at 19:34 Zhu Mengxi asked Mar 17 at 19:18 Zhu MengxiZhu Mengxi 851 gold badge1 silver badge6 bronze badges 01 Answer
Reset to default 1Yeah that's expected. The algorithm will only see (internal) corners if they are properly supported by markers. All the corners on the perimeter of the checkerboard will not be detected in any case. Neither will that corner in the bottom left of your picture because its adjacent markers are partially out of view.
Charuco for calibration really is a kludge. The decoding of markers requires either higher resolution than for a plain checkerboard, or fewer corners within view (more pixels between corners).
A proper solution to the "whole board must be in view" problem of legacy methods, sadly not (yet?) implemented in OpenCV, would simply detect all the saddle points, then take that point set and recover the grid structure required for pose estimation and calibration. Maybe one of these days I'll bother writing a POC for that.