Abstract: Focusing on the random vibration phenomena in infrared video sequences taken from moving vehicles, a BRISK characteristic point matching based high-precision global motion estimation algorithm is proposed. For mismatch in feature point matching and moving objects in the scene, fuzzy clustering method is used to separate global motion and local motion, which improves the robustness of the algorithm. A Kalman particle filter is used to separate the complex scanning movements and random vibration effectively. Meanwhile, we use bilinear interpolation for image compensation. Finally, a panoramic image is acquired by processing the undefined region with fast image alignment. A series of experiments have been carried out using video sequences taken on vehicle-borne thermal camera and the experimental results illustrate that the video sequences are well stabilized, which meet the requirements for practical use.