I am working with a zoom camera (10x zoom levels) with a frame size of 1920 × 1080. My goal is to calibrate the camera at each zoom level and apply undistortion to remove lens distortion.

For calibration, I captured images of a 9×6 checkerboard at each zoom level and used OpenCV to compute the camera matrix, distortion coefficients, rotation matrix, and translation matrix. I then stored these values in an .npz file for later use.

However, I noticed that the focal length values in the camera matrix vary significantly across zoom levels. Additionally, when I use cv2.undistort() to undistort frames, the output frame size is inconsistent, and in some cases, the resolution becomes much smaller than the original 1920 × 1080.

Tried:

Capturing checkerboard images at different zoom levels for calibration.
Using OpenCV's cv2.calibrateCamera() to compute the camera matrix and distortion coefficients.
Applying cv2.undistort() using the stored calibration parameters.
Using cv2.getOptimalNewCameraMatrix() to refine the undistortion process.

Expected Outcome:

A properly undistorted frame that maintains the original 1920 × 1080 resolution across all zoom levels.
A consistent frame size after undistortion.
Minimal distortion with straight lines appearing straight in the output frame.

However, the output resolution varies, and at some zoom levels (especially 3x zoom), the undistorted frame size is much smaller than expected.

I am looking for insights on why this happens and how to ensure a properly undistorted frame at each zoom level. Any guidance would be greatly appreciated!