Multilevel-in-width training for deep neural network regression

A common challenge in regression is that for many problems, the degrees of freedom required for a high-quality solution also allows for overfitting. Regularization is a class of strategies that seek to restrict the range of possible solutions so as to discourage overfitting while still enabling good solutions, and different regularization strategies impose different types of restrictions. In this paper, we present a multilevel regularization strategy that constructs and trains a hierarchy of neural networks, each of which has layers that are wider versions of the previous network's layers. We draw intuition and techniques from the field of Algebraic Multigrid (AMG), traditionally used for solving linear and nonlinear systems of equations, and specifically adapt the Full Approximation Scheme (FAS) for nonlinear systems of equations to the problem of deep learning. Training through V-cycles then encourage the neural networks to build a hierarchical understanding of the problem. We refer to this approach as multilevel-in-width to distinguish from prior multilevel works which hierarchically alter the depth of neural networks. The resulting approach is a highly flexible framework that can be applied to a variety of layer types, which we demonstrate with both fully connected and convolutional layers. We experimentally show with PDE regression problems that our multilevel training approach is an effective regularizer, improving the generalize performance of the neural networks studied.     

Robust multilevel methods for quadratic finite element anisotropic elliptic problems

This paper discusses a class of multilevel preconditioners based on approximate block factorization for conforming finite element methods employing quadratic trial and test functions. The main focus is on diffusion problems governed by a scalar elliptic partial differential equation with a strongly anisotropic coefficient tensor. The proposed method provides a high robustness with respect to non‐grid‐aligned anisotropy, which is achieved by the interaction of the following components: (i) an additive Schur complement approximation to construct the coarse‐grid operator; (ii) a global block (Jacobi or Gauss–Seidel) smoother complementing the coarse‐grid correction based on (i); and (iii) utilization of an augmented coarse grid, which enhances the efficiency of the interplay between (i) and (ii). The performed analysis indicates the high robustness of the resulting two‐level method. Moreover, numerical tests with a nonlinear algebraic multilevel iteration method demonstrate that the presented two‐level method can be applied successfully in the recursive construction of uniform multilevel preconditioners of optimal or nearly optimal order of computational complexity. Copyright © 2013 John Wiley & Sons, Ltd.     

Multilevel preconditioning and low‐rank tensor iteration for space–time simultaneous discretizations of parabolic PDEs

This paper addresses the solution of parabolic evolution equations simultaneously in space and time as may be of interest in, for example, optimal control problems constrained by such equations. As a model problem, we consider the heat equation posed on the unit cube in Euclidean space of moderately high dimension. An a priori stable minimal residual Petrov–Galerkin variational formulation of the heat equation in space–time results in a generalized least squares problem. This formulation admits a unique, quasi‐optimal solution in the natural space–time Hilbert space and serves as a basis for the development of space–time compressive solution algorithms. The solution of the heat equation is obtained by applying the conjugate gradient method to the normal equations of the generalized least squares problem. Starting from stable subspace splittings in space and in time, multilevel space–time preconditioners for the normal equations are derived. In order to reduce the complexity of the full space–time problem, all computations are performed in a compressed or sparse format called the hierarchical Tucker format, supposing that the input data are available in this format. In order to maintain sparsity, compression of the iterates within the hierarchical Tucker format is performed in each conjugate gradient iteration. Its application to vectors in the hierarchical Tucker format is detailed. Finally, numerical results in up to five spatial dimensions based on the recently developed htucker toolbox for MATLAB are presented. Copyright © 2014 John Wiley & Sons, Ltd.     

里德伯钠原子经绝热快速通道的布居数跃迁相干控制

运用含时多态展开方法和B-样条函数研究了啁啾频率微波场中里德伯钠原子的量子态之间的布居数迁移.计算了里德伯钠原子n=70-77的开普勒频率.计算了在不同的微波场中六个态的布居数从n=70到n=77随时间的迁移,布居数跃迁到最终态n=77达到了98%,这可以通过连续的单光子跃迁来实现.结果表明,通过优化微波脉冲参数可以实现从低态到较高态的布居跃迁的相干控制.     

Population coherent control of Rydberg potassium atom via adiabatic passage

The time-dependent multilevel approach(TDMA) and B-spline expansion technique are used to study the coherent population transfer between the quantum states of a potassium atom by a single frequency-chirped microwave pulse.The Rydberg potassium atom energy levels of n=6-15,l=0-5 states in zero field are calculated and the results are in good agreement with other theoretical values.The time evolutions of the population transfer of the six states from n=70 to n=75 in different microwave fields are obtained.The results show that the coherent control of the population transfer from the lower states to the higher ones can be accomplished by optimizing the microwave pulse parameters.     

Phase shift of the population oscillation and population trapping

Using our recently developed time-dependent multilevel approach, we have calculated the phase shift of the population oscillation of the excited potassium atom in frequency-modulated fields when the modulation amplitude is suddenly changed. The numerical results are in good agreement with the measurements. A new way of realizing population trapping is suggested.