磁共振(Magnetic Resonance, MR)成像是一种先进的成像方式,其特殊的成像方法导致采集数据的时间较长,容易受噪声的干扰;通过忽略部分数据(欠采样)可以提高采集速度,但会使图像分辨率丧失或产生图像伪影,导致图像质量下降。近年来,即插即用图像复原可以灵活地处理不同的图像复原任务,被广泛应用在自然图像中。受此启发,本文提出了一种基于深度学习的MR图像重建框架,通过训练卷积神经网络模型来建立深度去噪先验,并将其嵌入到交替迭代优化算法中,进而隐式地对图像先验进行建模。以解决欠采样MR图像中的噪声和模糊问题,从而重建出高质量图像。和其他先进算法相比,我们的方法在MR图像重建上具有更好的竞争力。
Magnetic Resonance (MR) imaging is an advanced imaging method, its special imaging method leads to a long time to collect data, easy to be disturbed by noise; by ignoring part of the data (undersampling), the acquisition speed can be improved, but the image resolution will be lost or image artifacts will be generated, resulting in image quality degradation. In recent years, Plug-and-Play image restoration has been widely used in natural images because it can flexibly handle different image restoration tasks. Inspired by this, this paper proposes a deep learning-based MR image reconstruction framework, which establishes a deep denoising prior by training a convolutional neural network model and embedding it into an alternating iterative optimization algorithm to implicitly model the image prior. To solve the problem of noise and blur in undersampled MR image and reconstruct high-quality images. Compared with other advanced algorithms, our method has better competitiveness in MR image reconstruction.
磁共振成像,深度学习,卷积神经网络,优化算法, Magnetic Resonance Imaging Deep Learning Convolutional Neural Network Optimization
Algorithm摘要
Magnetic Resonance (MR) imaging is an advanced imaging method, its special imaging method leads to a long time to collect data, easy to be disturbed by noise; by ignoring part of the data (undersampling), the acquisition speed can be improved, but the image resolution will be lost or image artifacts will be generated, resulting in image quality degradation. In recent years, Plug-and-Play image restoration has been widely used in natural images because it can flexibly handle different image restoration tasks. Inspired by this, this paper proposes a deep learning-based MR image reconstruction framework, which establishes a deep denoising prior by training a convolutional neural network model and embedding it into an alternating iterative optimization algorithm to implicitly model the image prior. To solve the problem of noise and blur in undersampled MR image and reconstruct high-quality images. Compared with other advanced algorithms, our method has better competitiveness in MR image reconstruction.
Keywords:Magnetic Resonance Imaging, Deep Learning, Convolutional Neural Network, Optimization Algorithm
其中,x为未知的原始图像,y为观测后的图像,H是一个退化矩阵,n为加性高斯白噪声(AWGN), τ ( x ) 通过指定不同的退化操作可以相应的得到不同的图像重建任务。在这里,我们的操作是欠采样导致图像模糊,故,H是一个稀疏矩阵。几乎任何恢复x的方法都涉及代价函数(由保真度和惩罚项组成),它由期望得到最小化解。其代价函数表示为:
f ( x ˜ ) = 1 2 σ e 2 ‖ y − H x ˜ ‖ 2 2 + s ( x ˜ ) (2)
其中, x ˜ 为优化变量, ‖ . ‖ 2 为欧氏范数。然而,利用变量分裂,P&P方法将最小化问题描述为:
min x ˜ , v ˜ l ( x ˜ ) + β s ( v ˜ ) s .t . x ˜ = v ˜ (3)
其中, l ( x ˜ ) ≜ 1 2 σ e 2 ‖ y − H x ˜ ‖ 2 2 是公式(2)中的保真度项,而β是一个正参数,使得代价函数变得复杂。通过ADMM [11] 构造一个增广拉格朗日可以解决该问题,即:
L λ = l ( x ˜ ) + β s ( v ˜ ) + u T ( x ˜ − v ˜ ) + λ 2 ‖ x ˜ − v ˜ ‖ 2 2 = l ( x ˜ ) + β s ( v ˜ ) + λ 2 ‖ x ˜ − v ˜ + u ˜ ‖ 2 2 − λ 2 ‖ u ˜ ‖ 2 2 (4)
其中,u是对偶变量, u ˜ ≜ 1 λ u 是缩放对偶变量, λ 是ADMM的惩罚参数,ADMM算法在以下三个步骤中进行迭代直到收敛。
x k ∨ = arg min x ˜ L λ ( x ˜ , v k − 1 ∨ , u k − 1 ∨ ) v k ∨ = arg min v ˜ L λ ( x k ∨ , v ˜ , u k − 1 ∨ ) u k ∨ = u k − 1 ∨ + ( x k ∨ − v k ∨ ) (5)
结合公式(4)和(5),则有:
x k ∨ = arg min x ˜ l ( x ˜ ) + λ 2 ‖ x ˜ − ( v k − 1 ∨ − u k − 1 ∨ ) ‖ 2 2 v k ∨ = arg min v ˜ λ 2 β ‖ ( x k ∨ + u k − 1 ∨ ) − v ˜ ‖ 2 2 + s ( v ˜ ) u k ∨ = u k − 1 ∨ + ( x k ∨ − v k ∨ ) (6)
输入: H , y , σ e 去噪运算符 D ( . ; ) ,中断准则。 y = H x + n , e ~ N ( 0 , σ e 2 I m ) 和x是一个未知信号,通过由先验模型的 D ( . ; ) 指定。 输出: x ^ 是对x的预测。 初始化: u 0 ∨ 为初始化, u 0 ∨ = 0 , k = 0 是对 β 和 λ 的初始化 While stopping criterion not met do k = k + 1 x k ∨ = ( H T H + λ σ e 2 I n ) − 1 × ( H T y + λ σ e 2 ( v k − 1 ∨ − u k − 1 ∨ ) ) v k ∨ = D ( x k ∨ + u k − 1 ∨ ; β / λ ) u k ∨ = u k − 1 ∨ + ( x k ∨ − v k ∨ ) End x ^ = x k ∨
表1. 即插即用算法
3.3. 我们的方法
众所周知,图像重建是一个逆问题,在此之前,还需要采用正则化来约束空间解 [38] 。从贝叶斯角度来看,通过求解最大后验估计(Maximum APosteriori, MAP)可以得到 x ^ 。
x ^ = arg max x log p ( y | x ) + log p ( x ) (7)
其中, log p ( y | x ) 表示观测值y的对数似然值, log p ( x ) 表示干净图像x的先验,并且与退化图像y无关。公式(7)可以重新表述为:
x ^ = arg min x 1 2 σ 2 ‖ y − τ ( x ) ‖ 2 + λ R ( x ) (8)
该公式最小化了由一个数据项 1 2 σ 2 ‖ y − τ ( x ) ‖ 2 和一个正则化参数为 λ 的正则化项 λ R ( x ) 组成的能量函数。为了解耦前面所述的数据项和先验项,我们引入了一个辅助变量z,得到一个约束优化公式:
x ^ = arg min x 1 2 σ 2 ‖ y − τ ( x ) ‖ 2 + λ R ( x ) s .t . z = x (9)
通过最小化处理可以得到:
L u ( x , y ) = 1 2 σ 2 ‖ y − τ ( x ) ‖ 2 + λ R ( z ) + u 2 ‖ z − x ‖ 2 (10)
其中,µ是一个惩罚因子。可以通过迭代解决x和z的子问题,同时保持其余的变量固定。
{ x k = arg min x ‖ y − τ ( x ) ‖ 2 + μ σ 2 ‖ x − z k − 1 ‖ 2 ( a ) z k = arg min z a 1 2 ( λ / μ ) 2 ‖ z − x k ‖ 2 + R ( z ) ( b ) (11)
公式(11)将数据项和先验项解耦为两个独立的子问题。具体地说,公式(11a)的目的是找到 z k − 1 的近端点,通常有一个快速的闭型解,取决于 τ ;而从贝叶斯角度来看,(11b)的子问题对应于噪声水平为 λ / μ 的高斯噪声 x k 。因此,任何高斯去噪器都可以插入到交替迭代中来求解公式(8)。为了解决这个问题,我们将公式(11b)整理为:
其中, x ⊗ k 表示干净图像与模糊核之间的二维卷积。假设在圆形边界条件下进行卷积,则得到公式11(a)的快速解:
x k = F − 1 ( F ( k ) ¯ F ( y ) + α k F ( z k − 1 ) F ( k ) ¯ F ( k ) + α k ) (14)
其中, F ( . ) 和 F − 1 ( . ) 表示快速傅里叶变化(Fast Fourier Transform, FFT)和逆傅里叶变化, F ( . ) ¯ 表示 F ( . ) 的复共轭。可以注意到,模糊内核k只涉及到公式(14),明确地处理了模糊的失真。具体详细算法描述见表2所示。
CNN-based algorith
算法2:CNN-based algorithm
输入:深度去噪先验模型,退化图像y,退化操作T,去噪水平 σ ,去噪先验模型在第k次迭代时的 σ k ,共K次迭代,权衡参数 λ 。 输出:重建图像zk。 初始化:从y初始化z0,预先计算 α k ≜ λ σ 2 / σ k 2 for k = 1 , 2 , ⋯ , K do x k = arg min x ‖ y − τ ( x ) ‖ 2 + α k ‖ x − z k − 1 ‖ 2 z k = D e n o i s e r ( x k , σ k ) end
蔡宇佳,张 利,吴康宁,陈 旋. 基于深度学习的MR医学图像重建方法研究Study on Deep Learning-Based MR Medical Image Reconstruction Method[J]. 软件工程与应用, 2022, 11(05): 1047-1057. https://doi.org/10.12677/SEA.2022.115107
参考文献ReferencesYang, J., Fan, J., Ai, D., Zhou, S., Tang, S. and Wang, Y. (2015) Brain MR Image Denoising for Rician Noise Using Pre-Smooth Non-Local Means Filter. Biomedical Engineering Online, 14, 1-20.Zhu, B., Liu, J.Z., Cauley, S.F., Rosen, B.R. and Rosen, M.S. (2018) Image Reconstruction by Domain-Transform Manifold Learning. Nature, 555, 487-492. <br>https://doi.org/10.1038/nature25988Yaqub, M., Jinchao, F., Arshid, K., Ahmed, S., Zhang, W., Nawaz, M.Z. and Mahmood, T. (2022) Deep Learning-Based Image Reconstruction for Different Medical Imaging Modalities. Computational and Mathematical Methods in Medicine, 2022, Article ID: 8750648. <br>https://doi.org/10.1155/2022/8750648Ahishakiye, E., Bastiaan Van Gijzen, M., Tumwiine, J., Wario, R. and Obungoloch, J. (2021) A Survey on Deep Learning in Medical Image Reconstruction. Intelligent Medicine, 1, 118-127.
<br>https://doi.org/10.1016/j.imed.2021.03.003Farnia, P., Mohammadi, M., Najafzadeh, E., Alimohamadi, M., MakkiAbadi, B. and Ahmadian, A. (2020) High-Quality Photoacoustic Image Reconstruction Based on Deep Convolutional Neural Network: Towards Intra-Operative Photoacoustic Imaging. Biomedical Physics & Engineering Express, 6, Article ID: 045019.
<br>https://doi.org/10.23919/ICIF.2017.8009769Liu, Y., Chen, X., Cheng, J. and Peng, H. (2017) A Medical Image Fusion Method Based on Convolutional Neural Networks. 2017 20th International Conference on Information Fusion (Fusion), Xi’an, 10-13 July 2017, 1-7.
<br>https://doi.org/10.23919/ICIF.2017.8009769Yi, X., Walia, E. and Babyn, P. (2019) Generative Adversarial Network in Medical Imaging: A Review. Medical Image Analysis, 58, Article ID: 101552. <br>https://doi.org/10.1016/j.media.2019.101552Kaldate, A., Patre, B.M., Harsh, R. and Verma, D. (2016) MR Image Reconstruction Based on Compressed Sensing Using Poisson Sampling Pattern. 2016 Second International Conference on Cognitive Computing and Information Processing (CCIP), Mysuru, 12-13 August 2016, 1-4. <br>https://doi.org/10.1109/CCIP.2016.7802884Roy, S., Carass, A. and Prince, J. (2011) A Compressed Sensing Approach for MR Tissue Contrast Synthesis. Biennial International Conference on Information Processing in Medical Imaging, Kloster Irsee, 3-8 July 2011, 371-383.
<br>https://doi.org/10.1007/978-3-642-22092-0_31Dutta, P., Upadhyay, P., De, M. and Khalkar, R.G. (2020) Medical Image Analysis Using Deep Convolutional Neural Networks: CNN Architectures and Transfer Learning. 2020 International Conference on Inventive Computation Technologies (ICICT), Coimbatore, 26-28 February 2020, 175-180. <br>https://doi.org/10.1109/ICICT48043.2020.9112469Boyd, S., Parikh, N., Chu, E., Peleato, B. and Eckstein, J. (2011) Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Foundations and Trends® in Machine learning, 3, 1-122.
<br>https://doi.org/10.1561/2200000016Geman, D. and Yang, C. (1995) Nonlinear Image Recovery with Half-Quadratic Regularization. IEEE Transactions on Image Processing, 4, 932-946. <br>https://doi.org/10.1109/83.392335Parikh, N. and Boyd, S. (2014) Proximal Algorithms. Foundations and Trends® in Optimization, 1, 127-239.
<br>https://doi.org/10.1561/2400000003Xu, L., Ren, J.S., Liu, C. and Jia, J. (2014) Deep Convolutional Neural Network for Image Deconvolution. Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, Montreal, 8-13 2014 December, 1790-1798.Venkatakrishnan, S.V., Bouman, C.A. and Wohlberg, B. (2013) Plug-and-Play Priors for Model Based Reconstruction. 2013 IEEE Global Conference on Signal and Information Processing, Austin, 3-5 December 2013, 945-948.
<br>https://doi.org/10.1109/GlobalSIP.2013.6737048Zhang, K., Li, Y., Zuo, W., Zhang, L., Van Gool, L. and Timofte, R. (2021) Plug-and-Play Image Restoration with Deep Denoiser Prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44, 6360-6376.
<br>https://doi.org/10.1109/TPAMI.2021.3088914Elad, M. and Aharon, M. (2006) Image Denoising via Sparse and Redundant Representations over Learned Dictionaries. IEEE Transactions on Image Processing, 15, 3736-3745. <br>https://doi.org/10.1109/TIP.2006.881969Dabov, K., Foi, A., Katkovnik, V. and Egiazarian, K. (2007) Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering. IEEE Transactions on Image Processing, 16, 2080-2095. <br>https://doi.org/10.1109/TIP.2007.901238Zoran, D. and Weiss, Y. (2011) From Learning Models of Natural Image Patches to Whole Image Restoration. 2011 International Conference on Computer Vision, Barcelona, 6-13 November 2011, 479-486.
<br>https://doi.org/10.1109/ICCV.2011.6126278Danielyan, A., Katkovnik, V. and Egiazarian, K. (2010) Image Deblurring by Augmented Lagrangian with BM3D Frame Prior. Proceedings of the Third Workshop on Information Theoretic Methods in Science and Engineering, Tampere, 16-18 August 2010, 16-18.Danielyan, A., Katkovnik, V. and Egiazarian, K. (2011) BM3D Frames and Variational Image Deblurring. IEEE Transactions on Image Processing, 21, 1715-1728. <br>https://doi.org/10.1109/TIP.2011.2176954Chambolle, A. and Pock, T. (2011) A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging. Journal of Mathematical Imaging and Vision, 40, 120-145.Teodoro, A.M., Bioucas-Dias, J.M. and Figueiredo, M.A. (2016) Image Restoration and Reconstruction Using Variable Splitting and Class-Adapted Image Priors. 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, 25-28 September 2016, 3518-3522. <br>https://doi.org/10.1109/ICIP.2016.7533014Metzler, C.A., Maleki, A. and Baraniuk, R.G. (2016) From Denoising to Compressed Sensing. IEEE Transactions on Information Theory, 62, 5117-5144. <br>https://doi.org/10.1109/TIT.2016.2556683Portilla, J., Strela, V., Wainwright, M.J. and Simoncelli, E.P. (2003) Image Denoising Using Scale Mixtures of Gaussians in the Wavelet Domain. IEEE Transactions on Image Processing, 12, 1338-1351.
<br>https://doi.org/10.1109/TIP.2003.818640Chan, S.H., Wang, X. and Elgendy, O.A. (2016) Plug-and-Play ADMM for Image Restoration: Fixed-Point Convergence and Applications. IEEE Transactions on Computational Imaging, 3, 84-98.
<br>https://doi.org/10.1109/TCI.2016.2629286Romano, Y., Elad, M. and Milanfar, P. (2017) The Little Engine That Could: Regularization by Denoising (RED). SIAM Journal on Imaging Sciences, 10, 1804-1844. <br>https://doi.org/10.1137/16M1102884Tirer, T. and Giryes, R. (2018) Image Restoration by Iterative Denoising and Backward Projections. IEEE Transactions on Image Processing, 28, 1220-1234. <br>https://doi.org/10.1109/TIP.2018.2875569Gu, S., Timofte, R. and Van Gool, L. (2018) Integrating Local and Non-Local Denoiser Priors for Image Restoration. 2018 24th International Conference on Pattern Recognition (ICPR), Beijing, 20-24 August 2018, 2923-2928.
<br>https://doi.org/10.1109/ICPR.2018.8545043Tirer, T. and Giryes, R. (2019) Super-Resolution via Image-Adapted Denoising CNNs: Incorporating External and Internal Learning. IEEE Signal Processing Letters, 26, 1080-1084. <br>https://doi.org/10.1109/LSP.2019.2920250Ryu, E., Liu, J., Wang, S., Chen, X., Wang, Z. and Yin, W. (2019, May) Plug-and-Play Methods Provably Converge with Properly Trained Denoisers. Proceedings of the 36th International Conference on Machine Learning, Long Beac, 9-15 June 2019, 5546-5557.Sun, Y., Liu, J. and Kamilov, U. (2019) Block Coordinate Regularization by Denoising. IEEE Transactions on Computational Imaging, 6, 908-921. <br>https://doi.org/10.1109/TCI.2020.2996385Zhang, K., Zuo, W., Chen, Y., Meng, D. and Zhang, L. (2017) Beyond a Gaussiandenoiser: Residual Learning of Deep CNN for Image Denoising. IEEE Transactions on Image Processing, 26, 3142-3155.
<br>https://doi.org/10.1109/TIP.2017.2662206Ronneberger, O., Fischer, P. and Brox, T. (2015) U-Net: Convolutional Networks for Biomedical Image Segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, 5-9 October 2015, 234-241. <br>https://doi.org/10.1007/978-3-319-24574-4_28He, K., Zhang, X., Ren, S. and Sun, J. (2016) Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 770-778.
<br>https://doi.org/10.1109/CVPR.2016.90Zhang, K., Zuo, W. and Zhang, L. (2018) FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising. IEEE Transactions on Image Processing, 27, 4608-4622. <br>https://doi.org/10.1109/TIP.2018.2839891Venkatesh, G.M., Naresh, Y.G., Little, S. and O’Connor, N.E. (2018) A Deep Residual Architecture for Skin Lesion Segmentation. 1st International Workshop, OR 2.0 2018, 5th International Workshop, CARE 2018, 7th International Workshop, CLIP 2018, Third International Workshop, ISIC 2018, Granada, 16, 20 September 2018, 277-284.
<br>https://doi.org/10.1007/978-3-030-01201-4_30Buades, A., Coll, B. and Morel, J.M. (2005) A Review of Image Denoising Algorithms, with a New One. Multiscale Modeling & Simulation, 4, 490-530. <br>https://doi.org/10.1137/040616024Fan, Q., Witzel, T., Nummenmaa, A., Van Dijk, K.R., Van Horn, J.D., Drews, M.K., et al. (2016) MGH-USC Human Connectome Project Datasets with Ultra-High B-Value Diffusion MRI. Neuroimage, 124, 1108-1114.
<br>https://doi.org/10.1016/j.neuroimage.2015.08.075