Joint calibration of sports camera and lidar based on LM algorithm

In order to solve the data matching problem between sports camera and lidar, a joint calibration optimization method of sports camera and lidar based on Levenberg-Marquard(LM) algorithm is designed. First, the calibration board is placed in the common field of view of the lidar and the sports camera, and the laser point cloud and image data of the calibration object at different positions are collected by changing the position of the calibration board. Then the fisheye distortion correction function is called through OpenCV to correct the image distortion, and obtain multiple sets of pixel coordinates of the corner points of the calibration plate image. At the same time, point cloud filtering and point cloud registration are performed on the laser point cloud, and the laser point cloud is segmented by a combination of manual and automatic methods, and then the point cloud center iterative algorithm is used to solve the calibration board point cloud center coordinates and The point cloud coordinates of each corner point. Finally, through multiple sets of point cloud coordinates representing the corner points of the calibration board and the corresponding image pixel coordinates, the direct linear transformation method (DLT) is used to calculate the initial value of the joint calibration between the two sensors, and the difference between the point cloud reprojection coordinates and the image pixel coordinates is constructed. The least squares function of, the function is optimized by the LM algorithm that introduces the damping factor, and the optimized joint calibration result is solved. Experiments show that the joint calibration result reduces the reprojection error by 35% compared with the initial value. The joint calibration result is used to achieve laser point cloud and image fusion based on the principle of collinear equations, which verifies the accuracy and effectiveness of the method.