Regularization,Bayesian Inference,and Machine Learning Methods for Inverse Problems

Classical methods for inverse problems are mainly based on regularization theory, in particular those, that are based on optimization of a criterion with two parts: a data-model matching and a regularization term. Different choices for these two terms and a great number of optimization algorithms have been proposed. When these two terms are distance or divergence measures, they can have a Bayesian Maximum A Posteriori (MAP) interpretation where these two terms correspond to the likelihood and prior-probability models, respectively. The Bayesian approach gives more flexibility in choosing these terms and, in particular, the prior term via hierarchical models and hidden variables. However, the Bayesian computations can become very heavy computationally. The machine learning (ML) methods such as classification, clustering, segmentation, and regression, based on neural networks (NN) and particularly convolutional NN, deep NN, physics-informed neural networks, etc. can become helpful to obtain approximate practical solutions to inverse problems. In this tutorial article, particular examples of image denoising, image restoration, and computed-tomography (CT) image reconstruction will illustrate this cooperation between ML and inversion.     

Estimation of the L-Curve via Lanczos Bidiagonalization

The L-curve criterion is often applied to determine a suitable value of the regularization parameter when solving ill-conditioned linear systems of equations with a right-hand side contaminated by errors of unknown norm. However, the computation of the L-curve is quite costly for large problems; the determination of a point on the L-curve requires that both the norm of the regularized approximate solution and the norm of the corresponding residual vector be available. Therefore, usually only a few points on the L-curve are computed and these values, rather than the L-curve, are used to determine a value of the regularization parameter. We propose a new approach to determine a value of the regularization parameter based on computing an L-ribbon that contains the L-curve in its interior. An L-ribbon can be computed fairly inexpensively by partial Lanczos bidiagonalization of the matrix of the given linear system of equations. A suitable value of the regularization parameter is then determined from the L-ribbon, and we show that an associated approximate solution of the linear system can be computed with little additional work.     

Tensor invariants of natural mechanical systems on compact surfaces and the corresponding integrals

The classification problem for tensor invariants of geodesic flows on compact surfaces is considered. Such invariants include first integrals, multivalued integrals, and symmetry fields. The problem is solved for tensor invariants of arbitrary degrees. It is shown that the invariants under consideration can vanish at some point only in the presence of two-valued symmetry fields and multivalued integrals. Translated fromMatematicheskie Zametki, Vol. 66, No. 3, pp. 417–430, September, 1999.     

Asymptotic L-based comparison of Tikhonov regularization and Landweber iteration

The distance between the Tikhonov and Landweber regularized solutions of a linear inverse problem is partly controlled by the L^∞-norm of the difference in their corresponding singular value filters. For large Landweber iteration number, the regularization parameter of the closest Tikhonov filter to a given Landweber filter is determined. This asymptotically computed parameter compares well with the numerically computed value even for moderate sized iteration number. A consequence of the analysis is to determine the range of singular values to which the difference in regularized solutions is most sensitive.     

三维波纹型可延展结构振动特性的辛分析

基于三维组装技术的可延展结构具备优异的延展性和可调控性,使其成功应用于各类可延展电子器件的制备中。为了评估该类电子器件的稳定性,本文研究三维波纹型可延展结构的振动行为。首先,基于非线性的Euler-Bernoulli梁理论、Kelvin-Voigt粘弹性理论和考虑压电材料的表面压电效应,建立三维波纹结构的理论分析模型;其次,基于能量原理和扩展拉格朗日运动原理,推导出该结构的动力学控制方程;然后采用二级四阶辛Runge-Kutta求解该动力学方程。通过数值仿真实验验证了辛算法的优越性,同时,还发现随着三维波纹型可延展结构外界激励及其结构参数的变化,该结构的振动特性会从倍周期向分岔和混沌转化;本文结果为三维波纹型可延展结构的优化设计及应用提供理论基础。     

动力学平衡方程的辛两步求解算法

基于线性多步方法的构造格式和辛变换,给出了动力学方程的两种辛两步法求解格式,它们分别具有四阶精度和二阶精度,但都只有二阶格式的计算量,因此四阶辛两步法具有较大的应用价值。对两种辛两步法和解析解进行了数值比较,证明了二阶精度辛两步格式在一定条件下就是欧拉中点保辛算法,或δ=0.5和α=0.25的Newmark辛格式。     

Real Sample Consistency Regularization for GANs

Mode collapse has always been a fundamental problem in generative adversarial networks. The recently proposed Zero Gradient Penalty (0GP) regularization can alleviate the mode collapse, but it will exacerbate a discriminator’s misjudgment problem, that is the discriminator judges that some generated samples are more real than real samples. In actual training, the discriminator will direct the generated samples to point to samples with higher discriminator outputs. The serious misjudgment problem of the discriminator will cause the generator to generate unnatural images and reduce the quality of the generation. This paper proposes Real Sample Consistency (RSC) regularization. In the training process, we randomly divided the samples into two parts and minimized the loss of the discriminator’s outputs corresponding to these two parts, forcing the discriminator to output the same value for all real samples. We analyzed the effectiveness of our method. The experimental results showed that our method can alleviate the discriminator’s misjudgment and perform better with a more stable training process than 0GP regularization. Our real sample consistency regularization improved the FID score for the conditional generation of Fake-As-Real GAN (FARGAN) from 14.28 to 9.8 on CIFAR-10. Our RSC regularization improved the FID score from 23.42 to 17.14 on CIFAR-100 and from 53.79 to 46.92 on ImageNet2012. Our RSC regularization improved the average distance between the generated and real samples from 0.028 to 0.025 on synthetic data. The loss of the generator and discriminator in standard GAN with our regularization was close to the theoretical loss and kept stable during the training process.     

金属凝固过程界面换热系数的Tikhonov正则化辨识

金属凝固过程中,铸件与铸型间的界面换热系数是数值模拟所必需的边界条件之一.充分考虑非线性瞬态热传导过程测温数据的误差特点,根据Tikhonov正则化理论,构造合适的正则化泛函,利用Arcangeli准则和Morozov偏差原理确定正则参数,采用灵敏度系数和Newton-Raphson迭代法求解该泛函.具体的案例分析表明,该方法克服了热传导反问题的不适定性,能够保证辨识结果的稳定性和精度.与其他方法相比,随着测温误差的增大,该方法具有较高的辨识精度.     

构造哈密尔顿系统jet辛差分格式的生成函数法

考虑哈密尔顿系统的保结构算法,在经典哈密尔顿系统的jet辛算法的基础上,给出了一般哈密尔顿系统的jet辛差分格式的定义.并利用带有变系数辛矩阵的一般哈密尔顿系统中的构造辛差分格式的生成函数法的思想,来建立由一般的反对称矩阵所确定的微分二形式与生成函数的关系,再利用哈密尔顿-雅可比方程来构造jet辛的差分格式.     

基于Hamilton体系的辛半解析法在各向异性电磁波导中的应用

力学中的Hamilton体系使用对偶变量来描述问题,而电磁场正好有电场和磁场这一对对偶变量。本文将力学中的Hamilton体系应用到电磁波导问题。根据电磁波导的Hamilton体系理论,辛几何可用于任意各向异性材料。将横向的电场和磁场构成对偶向量,基于Hamilton变分原理做半解析横向离散,并保持结构辛体系。本文以各向异性材料电磁波导为例,求解了问题的辛本征值,得到了镜像线的色散曲线。