With the rapid development of the nano-science, an atomic force microscopy (AFM) has been playing an increasingly important role in many fields. Nevertheless, the data utilization rate of the traditional AFM tremendously limits the performance of the system. In this paper, Kalman filtering is applied to replace the traditional data processing mode to improve the system efficiency and image quality. To be specific, with the purpose of utilizing all the data of each scanning point, firstly we use the least square method to match the sample height information in forward an backward scanning processes. Then, for each scanning point, Kalman filtering is applied to deal with all the data recursively to acquire the optimal outcome. Afterwards, we use the optimal outcome to calculate the more accurate height value of samples rather than only utilize a single data of forward scanning to get image. In this way, the system efficiency and the imaging precision of the AFM system are greatly improved. Experiment results demonstrate the effectiveness of the proposed method.