留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于深度学习的无波前探测自适应光学系统研究进展

张之光 杨慧珍 刘金龙 李松恒 苏杭 罗宇湘 魏谢文

张之光, 杨慧珍, 刘金龙, 等. 基于深度学习的无波前探测自适应光学系统研究进展[J]. 强激光与粒子束, 2021, 33: 081004. doi: 10.11884/HPLPB202133.210295
引用本文: 张之光, 杨慧珍, 刘金龙, 等. 基于深度学习的无波前探测自适应光学系统研究进展[J]. 强激光与粒子束, 2021, 33: 081004. doi: 10.11884/HPLPB202133.210295
Zhang Zhiguang, Yang Huizhen, Liu Jinlong, et al. Research progress in deep learning based WFSless adaptive optics system[J]. High Power Laser and Particle Beams, 2021, 33: 081004. doi: 10.11884/HPLPB202133.210295
Citation: Zhang Zhiguang, Yang Huizhen, Liu Jinlong, et al. Research progress in deep learning based WFSless adaptive optics system[J]. High Power Laser and Particle Beams, 2021, 33: 081004. doi: 10.11884/HPLPB202133.210295

基于深度学习的无波前探测自适应光学系统研究进展

doi: 10.11884/HPLPB202133.210295
基金项目: 国家自然科学基金项目(11573011);江苏省“六大人才高峰”高层次人才项目(KTHY-058);江苏省“333”高层次人才培养项目(BRA2019244)
详细信息
    作者简介:

    张之光(1988—),男,讲师,博士,主要从事自适应光学与激光雷达研究

    通讯作者:

    杨慧珍(1973—),女,教授,博士,主要从事自适应光学技术及其应用方面的研究

  • 中图分类号: TN929.12

Research progress in deep learning based WFSless adaptive optics system

  • 摘要:

    近年来自适应光学(AO)系统向着小型化和低成本化趋势发展,无波前探测自适应光学(WFSless AO)系统由于结构简单、应用范围广,成为目前相关领域的研究热点。硬件环境确定后,系统控制算法决定了WFSless AO系统的校正效果和系统收敛速度。新兴的深度学习及人工神经网络为WFSless AO系统控制算法注入了新的活力,进一步推动了WFSless AO系统的理论发展与应用发展。在回顾前期WFSless AO系统控制算法的基础上,全面介绍了近年来卷积神经网络(CNN)、长短期记忆神经网络(LSTM)、深度强化学习在WFSless AO系统控制中的应用,并对WFSless AO系统中各种深度学习模型的特点进行了总结。概述了WFSless AO技术在天文观测、显微成像、眼底成像、激光通信等领域的应用。

  • 图  1  用于相位复原的感知机式人工神经网络[18]

    Figure  1.  Perceptron artificial neural network for phase retrieval[18]

    图  2  用于推测Zernike系数的修改版Inception v3CNN模型[20]

    Figure  2.  Modified Inception v3CNN model for predicting Zernike coefficients[20]

    图  3  聚焦点目标的Zernike系数估计结果[23]

    Figure  3.  Zernike coefficients predicting results of focused target[23]

    图  4  经过曝预处理点目标的Zernike系数估计结果[23]

    Figure  4.  Zernike coefficients predicting results of overexposed target[23]

    图  5  WFSless AO系统结构[26]

    Figure  5.  WFSless system architecture[26]

    图  6  CNN结构[26]

    Figure  6.  Architecture of CNN[26]

    图  7  训练与推测过程的数据流[29]

    Figure  7.  Data flow of training and predictions[29]

    图  8  不同湍流程度下有无补偿时的残留波前方差[29]

    Figure  8.  Residual wavefront RMS with and without compensation under different turbulence levels[29]

    图  9  以不同湍流强度数据训练得到的模型推测Zernike系数[31]

    Figure  9.  Zernike coefficients prediction results by models trained with dataset of different turbulence levels[31]

    图  10  训练后的神经网络经TensorRT优化生成用于部署的推测引擎[35]

    Figure  10.  Trained neural network is optimized by TensorRT to build the inference engine for implementation[35]

    图  11  波前网络(WFNet)[36]

    Figure  11.  Wavefront Net (WFNet)[36]

    图  12  CNN结构[38]

    Figure  12.  CNN architecture[38]

    图  13  相位畸变校正前后的相位标准差[38]

    Figure  13.  Standard deviation of phase before and after phase aberration revision[38]

    图  14  采用LSTM神经网络的与目标无关的波前感知方法流程图[41]

    Figure  14.  An object irrelevant wavefront sensing scheme using LSTM neural network[41]

    图  15  根据LSTM推测的波前像差进行图像复原仿真结果[41]

    Figure  15.  Image restoration results based on wavefront error inferred by LSTM[41]

    图  16  LSTM对将来5帧波前的预测结果[44]

    Figure  16.  Prediction results of the next 5 frames wavefront made by LSTM[44]

    图  17  WFSless AO的强化学习[47]

    Figure  17.  Reinforcement Learning(RL) of WFSless AO[47]

    图  18  存在波前像差时点目标光强分布与经深度强化学习校正波前像差后的点目标光强分布[47]

    Figure  18.  Intensity distribution of point target with wavefront error and that after restoration by deep RL[47]

    图  19  不同振动频率的功率谱密度[51]

    Figure  19.  PSD of different vibration frequency[51]

    图  20  残留相位[51]

    Figure  20.  Residual phase[51]

    图  21  高分辨率光学显微镜像差校正原理[59]

    Figure  21.  Principle of aberration correction in high resolution optical microscopes[59]

    图  22  多致动器WFSless AO扫描源光学相干断层扫描系统原理图[63]

    Figure  22.  Schematic diagram of MAL-WSAO-SS-OCT system[63]

    表  1  采用过曝、离焦、散射预处理后估计出的Zernike系数准确度(均方差)[23]

    Table  1.   Accuracy of Zernike coefficients (RMS) with overexposure, defocus and scattering preprocessing[23]

    Zernike coefficients
    in-focusoverexposuredefocusscatter
    point source0.142±0.0320.036±0.0130.040±0.0160.057±0.018
    extended source0.288±0.0240.214±0.0510.099±0.0640.195±0.064
    下载: 导出CSV

    表  2  三种不同湍流强度数据集[31]

    Table  2.   Dataset of three different turbulence levels[31]

    dataset No.$ D/{r}_{0} $$ D/{r}_{0} $
    interval
    data volume/
    interval
    total data
    volume
    $D/{r}_{0}$
    interval
    data volume/intervaltotal data
    volume
    training datasettest dataset
    1 5 100 15000 10 1500
    2 15 100 15000 10 1500
    3 1-15 1 100 15000 1 10 1500
    下载: 导出CSV

    表  3  仿真不同湍流条件时波前复原误差(NPMS:归一化像素均方 RMS:均方根)[32]

    Table  3.   Simulation results of wavefront restoration error under different turbulence levels (NPMS: Normalized Pixel Mean Square; RMS: Root Mean Square)[32]

    $ D/{r}_{0} $NPMSRMS/λ
    $ 20 $0.00670.1307
    $ 15 $0.00410.0909
    $ 10 $0.00290.0718
    $ 6 $0.00250.0703
    下载: 导出CSV

    表  4  实验得到波前复原误差与运算时间(NPMS:归一化像素均方 RMS:均方根)[32]

    Table  4.   Wavefront restoration error and time consumption of experiments (NPMS: Normalized Pixel Mean Square; RMS: Root Mean Square)[32]

    $ D/{r}_{0} $NPMSRMS/λrunning time/ms
    $ 20 $0.00660.1304~12
    下载: 导出CSV

    表  5  PD-CNN和Xception模型推测时间对比[34]

    Table  5.   Comparison of inference time of PD-CNN with that of Xception[34]

    networkfocal model/msdefocused model/msPD model/ms
    PD-CNN2.24952.29892.5591
    Xception10.46910.110810.469
    下载: 导出CSV

    表  6  经TensorRT优化前后的推测时间对比[34]

    Table  6.   Comparison of inference time with and without optimization by TensorRT[34]

    modelbefore acceleration/msafter acceleration/msacceleration ratio
    focal model2.24950.46784.8091
    defocused model2.29890.44065.2178
    PD model2.55910.49095.2135
    下载: 导出CSV

    表  7  WFSless AO仿真软件

    Table  7.   WFSless AO simulation software

    AO simulation tooldeep learning framework
    Soapy[21]PyTorch (www.pytorch.org)
    HCIPy[52]Keras (www.keras.io)
    OOMAO[54]TensorFlow (www.tensorflow.org)
    YAO[55]MATLAB + Deep Learning Toolbox
    DASP[56]Caffe (https://caffe.berkeleyvision.org/)
    Soapy:Simulation ‘OptiqueAdaptative’ with Python    HCIPy:High Contrast Imaging for Python
    OOMAO:Object-Oriented MATLAB Adaptive Optics Toolbox    YAO:Yorick Adaptive Optics
    DASP: Durham Adaptive Optics Simulation Platform
    下载: 导出CSV
  • [1] Shorter R S. Principles of adaptive optics, 3rd edn., by Robert K. Tyson[J]. Contemporary Physics, 2011, 52(5): 501-502.
    [2] 冯麓, 张玉佩, 宋菲君, 等. 夜天文中的自适应光学[J]. 物理, 2018, 47(6):355-366. (Feng Lu, Zhang Yupei, Song Feijun, et al. Adaptive optics for night astronomy[J]. Physics, 2018, 47(6): 355-366 doi: 10.7693/wl20180602
    [3] Yazdani R, Hajimahmoodzadeh M, Fallah H R. Adaptive phase aberration correction based on imperialist competitive algorithm[J]. Applied Optics, 2014, 53(1): 132-140. doi: 10.1364/AO.53.000132
    [4] 杨慧珍, 陈波, 李新阳, 等. 自适应光学系统随机并行梯度下降控制算法实验研究[J]. 光学学报, 2008, 28(2):205-210. (Yang Huizhen, Chen Bo, Li Xinyang, et al. Experimental demonstration of stochastic parallel gradient descent control algorithm for adaptive optics system[J]. Acta Optica Sinica, 2008, 28(2): 205-210 doi: 10.3321/j.issn:0253-2239.2008.02.001
    [5] Facomprez A, Beaurepaire E, Débarre D. Accuracy of correction in modal sensorless adaptive optics[J]. Optics Express, 2012, 20(3): 2598-2612. doi: 10.1364/OE.20.002598
    [6] Huang Linhai, Rao Changhui. Wavefront sensorless adaptive optics: a general model-based approach[J]. Optics Express, 2011, 19(1): 371-379. doi: 10.1364/OE.19.000371
    [7] 方舟, 徐项项, 李鑫, 等. 自适应增益的SPGD算法[J]. 红外与激光工程, 2020, 49:20200274. (Fang Zhou, Xu Xiangxiang, Li Xin, et al. SPGD algorithm with adaptive gain[J]. Infrared and Laser Engineering, 2020, 49: 20200274
    [8] Zommer S, Ribak E N, Lipson S G, et al. Simulated annealing in ocular adaptive optics[J]. Optics Letters, 2006, 31(7): 939-941. doi: 10.1364/OL.31.000939
    [9] 杨平, 许冰, 姜文汉, 等. 遗传算法在自适应光学系统中的应用[J]. 光学学报, 2007, 27(9):1628-1632. (Yang Ping, Xu Bing, Jiang Wenhan, et al. Study of a genetic algorithm used in an adaptive optical system[J]. Acta Optica Sinica, 2007, 27(9): 1628-1632 doi: 10.3321/j.issn:0253-2239.2007.09.017
    [10] Yang Huizhen, Li Xinyang. Comparison of several stochastic parallel optimization algorithms for adaptive optics system without a wavefront sensor[J]. Optics & Laser Technology, 2011, 43(3): 630-635.
    [11] Booth M J. Wavefront sensorless adaptive optics for large aberrations[J]. Optics Letters, 2007, 32(1): 5-7. doi: 10.1364/OL.32.000005
    [12] 杨慧珍, 吴健, 龚成龙. 基于模型的无波前探测自适应光学系统[J]. 光学学报, 2014, 34:0801002. (Yang Huizhen, Wu Jian, Gong Chenglong. Model-based sensorless adaptive optics system[J]. Acta Optica Sinica, 2014, 34: 0801002 doi: 10.3788/AOS201434.0801002
    [13] Wen Lianghua, Yang Ping, Wang Shuai, et al. A high speed model-based approach for wavefront sensorless adaptive optics systems[J]. Optics & Laser Technology, 2018, 99: 124-132.
    [14] Huang Linhai. Coherent beam combination using a general model-based method[J]. Chinese Physics Letters, 2014, 31: 094205. doi: 10.1088/0256-307X/31/9/094205
    [15] Wen Lianghua, Yang Ping, Yang Kangjian, et al. Synchronous model-based approach for wavefront sensorless adaptive optics system[J]. Optics Express, 2017, 25(17): 20584-20597. doi: 10.1364/OE.25.020584
    [16] Song Hong, Fraanje R, Schitter G, et al. Model-based aberration correction in a closed-loop wavefront-sensor-less adaptive optics system[J]. Optics Express, 2010, 18(23): 24070-24084. doi: 10.1364/OE.18.024070
    [17] Angel J R P, Wizinowich P, Lloyd-Hart M, et al. Adaptive optics for array telescopes using neural-network techniques[J]. Nature, 1990, 348(6298): 221-224. doi: 10.1038/348221a0
    [18] Sandler D G, Barrett T K, Palmer D A, et al. Use of a neural network to control an adaptive optics system for an astronomical telescope[J]. Nature, 1991, 351(6324): 300-302. doi: 10.1038/351300a0
    [19] Barrett T K, Sandler D G. Artificial neural network for the determination of Hubble Space Telescope aberration from stellar images[J]. Applied Optics, 1993, 32(10): 1720-1727. doi: 10.1364/AO.32.001720
    [20] Paine S W, Fienup J R. Machine learning for improved image-based wavefront sensing[J]. Optics Letters, 2018, 43(6): 1235-1238. doi: 10.1364/OL.43.001235
    [21] Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 2818-2826.
    [22] Byrd R H, Lu Peihuang, Nocedal J, et al. A limited memory algorithm for bound constrained optimization[J]. SIAM Journal on Scientific Computing, 1995, 16(5): 1190-1208. doi: 10.1137/0916069
    [23] Nishizaki Y, Valdivia M, Horisaki R, et al. Deep learning wavefront sensing[J]. Optics Express, 2019, 27(1): 240-251. doi: 10.1364/OE.27.000240
    [24] Chollet F. Xception: Deep learning with depthwise separable convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 1251-1258.
    [25] Kingma D, Ba J. Adam: A method for stochastic optimization[DB/OL]. arXiv preprint, arXiv: 1412.6980, 2014.
    [26] Tian Qinghua, Lu Chenda, Liu Bo, et al. DNN-based aberration correction in a wavefront sensorless adaptive optics system[J]. Optics Express, 2019, 27(8): 10765-10776. doi: 10.1364/OE.27.010765
    [27] Polo A, Haber A, Pereira S F, et al. An innovative and efficient method to control the shape of push-pull membrane deformable mirror[J]. Optics Express, 2012, 20(25): 27922-27932. doi: 10.1364/OE.20.027922
    [28] Gonsalves R A. Phase retrieval and diversity in adaptive optics[J]. Optical Engineering, 1982, 21: 215829.
    [29] Ma Huimin, Liu Haiqiu, Qiao Yan, et al. Numerical study of adaptive optics compensation based on convolutional neural networks[J]. Optics Communications, 2019, 433: 283-289. doi: 10.1016/j.optcom.2018.10.036
    [30] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. 2012: 1097-1105.
    [31] 马慧敏, 焦俊, 乔焰, 等. 一种基于光强图像深度学习的波前复原方法[J]. 激光与光电子学进展, 2020, 57:081103. (Ma Huimin, Jiao Jun, Qiao Yan, et al. Wavefront restoration method based on light intensity image deep learning[J]. Laser & Optoelectronics Progress, 2020, 57: 081103
    [32] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[DB/OL]. arXiv preprint, arXiv: 1409.1556, 2014.
    [33] Guo Hongyang, Xu Yangjie, Li Qing, et al. Improved machine learning approach for wavefront sensing[J]. Sensors, 2019, 19(16): 3533. doi: 10.3390/s19163533
    [34] Wu Yu, Guo Youming, Bao Hua, et al. Sub-millisecond phase retrieval for phase-diversity wavefront sensor[J]. Sensors, 2020, 20(17): 4877. doi: 10.3390/s20174877
    [35] Nvidia Tensor RT[EB/OL]. (2021-07-09). https://developer.nvidia.com/tensorrt.
    [36] Vera E, Guzmán F, Weinberger C. Boosting the deep learning wavefront sensor for real-time applications [Invited][J]. Applied Optics, 2021, 60(10): B119-B124. doi: 10.1364/AO.417574
    [37] Weinberger C, Guzmán F, Vera E. Improved training for the deep learning wavefront sensor[C]//Proceedings Volume 11448, Adaptive Optics Systems VII. 2020, 11448: 114484G.
    [38] Wang Kaiqiang, Zhang Mengmeng, Tang Ju, et al. Deep learning wavefront sensing and aberration correction in atmospheric turbulence[J]. PhotoniX, 2021, 2: 8. doi: 10.1186/s43074-021-00030-4
    [39] He Kaiming, Zhang Xiangyu, Ren Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 770-778.
    [40] Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780. doi: 10.1162/neco.1997.9.8.1735
    [41] Xin Qi, Ju Guohao, Zhang Chunyue, et al. Object-independent image-based wavefront sensing approach using phase diversity images and deep learning[J]. Optics Express, 2019, 27(18): 26102-26119. doi: 10.1364/OE.27.026102
    [42] Liu Xuewen, Morris T, Saunter C. Using long short-term memory for wavefront prediction in adaptive optics[C]//Proceeding of the 28th International Conference on Artificial Neural Networks. Munich: Springer, 2019: 537-542.
    [43] Chen Ying. LSTM recurrent neural network prediction algorithm based on Zernike modal coefficients[J]. Optik, 2020, 203: 163796. doi: 10.1016/j.ijleo.2019.163796
    [44] Swanson R, Lamb M, Correia C, et al. Wavefront reconstruction and prediction with convolutional neural networks[C]//Proceedings Volume 10703, Adaptive Optics Systems VI. 2018, 10703: 107031F.
    [45] Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529-533. doi: 10.1038/nature14236
    [46] Hu Ke, Xu Bing, Xu Zhenxing, et al. Self-learning control for wavefront sensorless adaptive optics system through deep reinforcement learning[J]. Optik, 2019, 178: 785-793. doi: 10.1016/j.ijleo.2018.09.160
    [47] Hu Ke, Xu Zhenxing, Yang Wei, et al. Build the structure of WFSless AO system through deep reinforcement learning[J]. IEEE Photonics Technology Letters, 2018, 30(23): 2033-2036. doi: 10.1109/LPT.2018.2874998
    [48] Konda V R, Tsitsiklis J N. Actor-critic algorithms[C]//Proceedings of Advances in Neural Information Processing Systems. 2000: 1008-1014.
    [49] Nousiainen J, Rajani C, Kasper M, et al. Adaptive optics control using model-based reinforcement learning[J]. Optics Express, 2021, 29(10): 15327-15344. doi: 10.1364/OE.420270
    [50] Cantalloube F, Farley O, Milli J, et al. Wind-driven halo in high-contrast images. I. Analysis of the focal-plane images of SPHERE[J]. Astronomy and Astrophysics, 2020, 638: A98. doi: 10.1051/0004-6361/201937397
    [51] Landman R, Haffert S Y, Radhakrishnan V M, et al. Self-optimizing adaptive optics control with reinforcement learning[C]//Proceedings Volume 11448, Adaptive Optics Systems VII. 2020, 11448: 1144849.
    [52] Por E H, Haffert S Y, Radhakrishnan V M, et al. High Contrast Imaging for Python (HCIPy): an open-source adaptive optics and coronagraph simulator[C]//Proceedings Volume 10703, Adaptive Optics Systems VI. 2018, 10703: 1070342.
    [53] Gendron E, Léna P. Astronomical adaptive optics. II. Experimental results of an optimized modal control[J]. Astronomy and Astrophysics Supplement Series, 1995, 111: 153-167.
    [54] Conan R, Correia C. Object-oriented Matlab adaptive optics toolbox[C]//Proceedings Volume 9148, Adaptive Optics Systems IV. 2014, 9148: 91486C.
    [55] Rigaut F, Van Dam M. Simulating astronomical adaptive optics systems using YAO[C]//Third AO4ELT Conference - Adaptive Optics for Extremely Large Telescopes. 2013.
    [56] Basden A G, Bharmal N A, Jenkins D, et al. The Durham Adaptive Optics Simulation Platform (DASP): Current status[J]. SoftwareX, 2018, 7: 63-69. doi: 10.1016/j.softx.2018.02.005
    [57] Zhu Licheng, Wen Lianghua, Yang Ping, et al. Aberration correction based on wavefront sensorless adaptive optics in membrane diffractive optical telescope[J]. Optics Communications, 2019, 451: 220-225. doi: 10.1016/j.optcom.2019.06.063
    [58] Marx V. Microscopy: hello, adaptive optics[J]. Nature Methods, 2017, 14(12): 1133-1136. doi: 10.1038/nmeth.4508
    [59] Booth M J. Adaptive optical microscopy: the ongoing quest for a perfect image[J]. Light: Science & Applications, 2014, 3: e165.
    [60] Hussain S A, Kubo T, Hall N, et al. Wavefront-sensorless adaptive optics with a laser-free spinning disk confocal microscope[J]. Journal of Microscopy, 2020.
    [61] 刘立新, 张美玲, 吴兆青, 等. 自适应光学在荧光显微镜中的应用[J]. 激光与光电子学进展, 2020, 57:120001. (Liu Lixin, Zhang Meiling, Wu Zhaoqing, et al. Application of adaptive optics in fluorescence microscope[J]. Laser & Optoelectronics Progress, 2020, 57: 120001
    [62] Qin Zhongya, He Sicong, Yang Chao, et al. Adaptive optics two-photon microscopy enables near-diffraction-limited and functional retinal imaging in vivo[J]. Light: Science & Applications, 2020, 9: 79.
    [63] Jian Yifan, Lee Sujin, Ju M J, et al. Lens-based wavefront sensorless adaptive optics swept source OCT[J]. Scientific Reports, 2016, 6: 27620. doi: 10.1038/srep27620
    [64] 牛超君, 于诗杰, 韩香娥. 无波前探测自适应光学对光通信性能影响分析[J]. 激光与光电子学进展, 2015, 52:080102. (Niu Chaojun, Yu Shijie, Han Xiang’e. Analysis about effect of wavefront sensorless adaptive optics on optical communication[J]. Laser & Optoelectronics Progress, 2015, 52: 080102
    [65] 徐晓帆, 陆洲. 星地激光通信可靠性保障技术研究现状[J]. 中国电子科学研究院学报, 2018, 13(6):650-657. (Xu Xiaofan, Lu Zhou. Research status of mitigation techniques to assure the reliability of satellite-to-ground laser communications[J]. Journal of CAEIT, 2018, 13(6): 650-657 doi: 10.3969/j.issn.1673-5692.2018.06.005
  • 加载中
图(22) / 表(7)
计量
  • 文章访问数:  1757
  • HTML全文浏览量:  475
  • PDF下载量:  258
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-07-19
  • 修回日期:  2021-08-11
  • 网络出版日期:  2021-08-20
  • 刊出日期:  2021-08-15

目录

    /

    返回文章
    返回