The optimal solution of multi-kernel regularization learning

In regularized kernel methods, the solution of a learning problem is found by minimizing a functional consisting of a empirical risk and a regularization term. In this paper, we study the existence of optimal solution of multi-kernel regularization learning. First, we ameliorate a previous conclusion about this problem given by Micchelli and Pontil, and prove that the optimal solution exists whenever the kernel set is a compact set. Second, we consider this problem for Gaussian kernels with variance σ∈(0,∞), and give some conditions under which the optimal solution exists.     

A METHOD TO APPROACH OPTIMAL RESTORATION IN IMAGE RESTORATION PROBLEMS WITHOUT NOISE ENERGY INFORMATION

This paper proposes a new image restoration technique, in which the resulting regularized image approximates the optimal solution steadily. The affect of the regular-ization operator and parameter on the lower band and upper band energy of the residue of the regularized image is theoretically analyzed by employing wavelet transform. This paper shows that regularization operator should generally be lowstop and highpass. So this paper chooses a lowstop and highpass operator as regularization operator, and construct an optimization model which minimizes the mean squares residue of regularized solution to determine regularization parameter. Although the model is random, on the condition of this paper, it can be solved and yields regularization parameter and regularized solu-tion. Otherwise, the technique has a mechanism to predict noise energy. So, without noisei nformation, it can also work and yield good restoration results.     

Lagrange multipliers, (exact) regularization and error bounds for monotone variational inequalities

We examine two central regularization strategies for monotone variational inequalities, the first a direct regularization of the operative monotone mapping, and the second via regularization of the associated dual gap function. A key link in the relationship between the solution sets to these various regularized problems is the idea of exact regularization, which, in turn, is fundamentally associated with the existence of Lagrange multipliers for the regularized variational inequality. A regularization is said to be exact if a solution to the regularized problem is a solution to the unregularized problem for all parameters beyond a certain value. The Lagrange multipliers corresponding to a particular regularization of a variational inequality, on the other hand, are defined via the dual gap function. Our analysis suggests various conceptual, iteratively regularized numerical schemes, for which we provide error bounds, and hence stopping criteria, under the additional assumption that the solution set to the unregularized problem is what we call weakly sharp of order greater than one.     

Optimal control for systems described by hyperbolic equations with strong nonlinearity

An optimization control problem for a hyperbolic equation is considered. The system is nonlinear with respect to the state derivative. The regularization technique for the state equation is applied. The necessary conditions of optimality for the regularized control problem are proved. It uses the extended differentiability of the control-state mapping for the regularized equation. The convergence of the regularization method is proved. Thus the optimal control for the regularized problem with a small enough regularization parameter can be chosen as an approximate solution of the initial optimization problem.     

A smoothing-regularization approach to mathematical programs with vanishing constraints

We consider a numerical approach for the solution of a difficult class of optimization problems called mathematical programs with vanishing constraints. The basic idea is to reformulate the characteristic constraints of the program via a nonsmooth function and to eventually smooth it and regularize the feasible set with the aid of a certain smoothing- and regularization parameter t>0 such that the resulting problem is more tractable and coincides with the original program for t=0. We investigate the convergence behavior of a sequence of stationary points of the smooth and regularized problems by letting t tend to zero. Numerical results illustrating the performance of the approach are given. In particular, a large-scale example from topology optimization of mechanical structures with local stress constraints is investigated.     

Multigrid Solution of a Lavrentiev-Regularized State-Constrained Parabolic Control Problem

A mesh-independent, robust, and accurate multigrid scheme to solve a linear state-constrained parabolic optimal control problem is presented. We first consider a Lavrentiev regularization of the state-constrained optimization problem. Then, a multigrid scheme is designed for the numerical solution of the regularized optimality system. Central to this scheme is the construction of an iterative pointwise smoother which can be formulated as a local semismooth Newton iteration. Results of numerical experiments and theoretical two-grid local Fourier analysis estimates demonstrate that the proposed scheme is able to solve parabolic state-constrained optimality systems with textbook multigrid efficiency.     

On the approximability of adjustable robust convex optimization under uncertainty

In this paper, we consider adjustable robust versions of convex optimization problems with uncertain constraints and objectives and show that under fairly general assumptions, a static robust solution provides a good approximation for these adjustable robust problems. An adjustable robust optimization problem is usually intractable since it requires to compute a solution for all possible realizations of uncertain parameters, while an optimal static solution can be computed efficiently in most cases if the corresponding deterministic problem is tractable. The performance of the optimal static robust solution is related to a fundamental geometric property, namely, the symmetry of the uncertainty set. Our work allows for the constraint and objective function coefficients to be uncertain and for the constraints and objective functions to be convex, thereby providing significant extensions of the results in Bertsimas and Goyal (Math Oper Res 35:284–305, 2010) and Bertsimas et al. (Math Oper Res 36: 24–54, 2011b) where only linear objective and linear constraints were considered. The models in this paper encompass a wide variety of problems in revenue management, resource allocation under uncertainty, scheduling problems with uncertain processing times, semidefinite optimization among many others. To the best of our knowledge, these are the first approximation bounds for adjustable robust convex optimization problems in such generality.     

应用正则化子建立求解不适定问题的正则化方法的探讨

根据紧算子的奇异系统理论,提出一种新的正则化子进而建立了一类新的求解不适定问题的正则化方法。分别通过正则参数的先验选取和后验确定方法,证明了正则解的收敛性并得到了其最优的渐近收敛阶;验证了应用Newton迭代法计算最佳参数的可行性。最后建立了当算子与右端均有扰动时相应的正则化求解策略。文中所述方法完善了一般优化正则化策略的构造理论。     

Regularized state-constrained boundary optimal control of the Navier-Stokes equations

The numerical solution of the Dirichlet boundary optimal control problem of the Navier-Stokes equations in presence of pointwise state constraints is investigated. Two different regularization techniques are considered. First, a Moreau-Yosida regularization of the problem is studied. Optimality conditions are derived and the convergence of the regularized solutions towards the original one is proved. A source representation of the control combined with a Lavrentiev type regularization strategy is also presented. The analysis concerning optimality conditions and convergence of the regularized solutions is carried out. In the last part of the paper numerical experiments are presented. For the numerical solution of each regularized problem a semi-smooth Newton method is applied.     

A semismooth Newton method for a class of semilinear optimal control problems with box and volume constraints

In this paper we consider optimal control problems subject to a semilinear elliptic state equation together with the control constraints 0≤u≤1 and ∫u=m. Optimality conditions for this problem are derived and reformulated as a nonlinear, nonsmooth equation which is solved using a semismooth Newton method. A regularization of the nonsmooth equation is necessary to obtain the superlinear convergence of the semismooth Newton method. We prove that the solutions of the regularized problems converge to a solution of the original problem and a path-following technique is used to ensure a constant decrease rate of the residual. We show that, in certain situations, the optimal controls take 0–1 values, which amounts to solving a topology optimization problem with volume constraint.     

Adaptive and robust radiation therapy optimization for lung cancer

A previous approach to robust intensity-modulated radiation therapy (IMRT) treatment planning for moving tumors in the lung involves solving a single planning problem before the start of treatment and using the resulting solution in all of the subsequent treatment sessions. In this paper, we develop an adaptive robust optimization approach to IMRT treatment planning for lung cancer, where information gathered in prior treatment sessions is used to update the uncertainty set and guide the reoptimization of the treatment for the next session. Such an approach allows for the estimate of the uncertain effect to improve as the treatment goes on and represents a generalization of existing robust optimization and adaptive radiation therapy methodologies. Our method is computationally tractable, as it involves solving a sequence of linear optimization problems. We present computational results for a lung cancer patient case and show that using our adaptive robust method, it is possible to attain an improvement over the traditional robust approach in both tumor coverage and organ sparing simultaneously. We also prove that under certain conditions our adaptive robust method is asymptotically optimal, which provides insight into the performance observed in our computational study. The essence of our method – solving a sequence of single-stage robust optimization problems, with the uncertainty set updated each time – can potentially be applied to other problems that involve multi-stage decisions to be made under uncertainty.     

A new distributionally robust p-hub median problem with uncertain carbon emissions and its tractable approximation method

The p-hub median problem is to determine the optimal location for p hubs and assign the remaining nodes to hubs so as to minimize the total transportation costs. Under the carbon cap-and-trade policy, we study this problem by addressing the uncertain carbon emissions from the transportation, where the probability distributions of the uncertain carbon emissions are only partially available. A novel distributionally robust optimization model with the ambiguous chance constraint is developed for the uncapacitated single allocation p-hub median problem. The proposed distributionally robust optimization problem is a semi-infinite chance-constrained optimization model, which is computationally intractable for general ambiguity sets. To solve this hard optimization model, we discuss the safe approximation to the ambiguous chance constraint in the following two types of ambiguity sets. The first ambiguity set includes the probability distributions with the bounded perturbations with zero means. In this case, we can turn the ambiguous chance constraint into its computable form based on tractable approximation method. The second ambiguity set is the family of Gaussian perturbations with partial knowledge of expectations and variances. Under this situation, we obtain the deterministic equivalent form of the ambiguous chance constraint. Finally, we validate the proposed optimization model via a case study from Southeast Asia and CAB data set. The numerical experiments indicate that the optimal solutions depend heavily on the distribution information of carbon emissions. In addition, the comparison with the classical robust optimization method shows that the proposed distributionally robust optimization method can avoid over-conservative solutions by incorporating partial probability distribution information. Compared with the stochastic optimization method, the proposed method pays a small price to depict the uncertainty of probability distribution. Compared with the deterministic model, the proposed method generates the new robust optimal solution under uncertain carbon emissions.     

Robust Solution to Fuzzy Identification Problem with Uncertain Data by Regularization

This study considers the robust identification of the parameters describing a Sugeno type fuzzy inference system with uncertain data. The objective is to minimize the worst-case residual error using a numerically efficient algorithm. The Sugeno type fuzzy systems are linear in consequent parameters but nonlinear in antecedent parameters. The robust consequent parameters identification problem can be formulated as second-order cone programming problem. The optimal solution of this second-order cone problem can be interpreted as solution of a Tikhonov regularization problem with a special choice of regularization parameter which is optimal for robustness (Ghaoui and Lebret (1997). SAIM Journal of Matrix Analysis and Applications 18, 1035–1064). The final regularized nonlinear optimization problem allowing simultaneous identification of antecedent and consequent parameters is solved iteratively using a generalized Gauss–Newton like method. To illustrate the approach, several simulation studies on numerical examples including the modelling of a spectral data function (one-dimensional benchmark example) is provided. The proposed robust fuzzy identification scheme has been applied to approximate the physical fitness of patients with a fuzzy expert system. The identified fuzzy expert system is shown to be capable of capturing the decisions (experiences) of a medical expert.     

SOS-Convex Semialgebraic Programs and its Applications to Robust Optimization: A Tractable Class of Nonsmooth Convex Optimization

In this paper, we introduce a new class of nonsmooth convex functions called SOS-convex semialgebraic functions extending the recently proposed notion of SOS-convex polynomials. This class of nonsmooth convex functions covers many common nonsmooth functions arising in the applications such as the Euclidean norm, the maximum eigenvalue function and the least squares functions with ? ₁-regularization or elastic net regularization used in statistics and compressed sensing. We show that, under commonly used strict feasibility conditions, the optimal value and an optimal solution of SOS-convex semialgebraic programs can be found by solving a single semidefinite programming problem (SDP). We achieve the results by using tools from semialgebraic geometry, convex-concave minimax theorem and a recently established Jensen inequality type result for SOS-convex polynomials. As an application, we show that robust SOS-convex optimization proble ms under restricted spectrahedron data uncertainty enjoy exact SDP relaxations. This extends the existing exact SDP relaxation result for restricted ellipsoidal data uncertainty and answers an open question in the literature on how to recover a robust solution of uncertain SOS-convex polynomial programs from its semidefinite programming relaxation in this broader setting.     

Balancing principle in supervised learning for a general regularization scheme

We discuss the problem of parameter choice in learning algorithms generated by a general regularization scheme. Such a scheme covers well-known algorithms as regularized least squares and gradient descent learning. It is known that in contrast to classical deterministic regularization methods, the performance of regularized learning algorithms is influenced not only by the smoothness of a target function, but also by the capacity of a space, where regularization is performed. In the infinite dimensional case the latter one is usually measured in terms of the effective dimension. In the context of supervised learning both the smoothness and effective dimension are intrinsically unknown a priori. Therefore we are interested in a posteriori regularization parameter choice, and we propose a new form of the balancing principle. An advantage of this strategy over the known rules such as cross-validation based adaptation is that it does not require any data splitting and allows the use of all available labeled data in the construction of regularized approximants. We provide the analysis of the proposed rule and demonstrate its advantage in simulations.     

一类偏积分-微分方程中的反问题

对一类偏积分-微分方程中参数校准的反问题进行研究.在弱解的框架下,原问题可转化为含具体正则化项的最优化问题.文中证明了该最优化问题的解的存在性和稳定性,并考察了最优解存在的一阶必要条件.另外,证明了当正则化参数足够大时,该最优化问题关于参数a的凸性性质.基于偏积分-微分方程反问题的研究对于金融市场中的模型校准问题具有重要的意义.     

Variable-sized uncertainty and inverse problems in robust optimization

In robust optimization, the general aim is to find a solution that performs well over a set of possible parameter outcomes, the so-called uncertainty set. In this paper, we assume that the uncertainty size is not fixed, and instead aim at finding a set of robust solutions that covers all possible uncertainty set outcomes. We refer to these problems as robust optimization with variable-sized uncertainty. We discuss how to construct smallest possible sets of min–max robust solutions and give bounds on their size.A special case of this perspective is to analyze for which uncertainty sets a nominal solution ceases to be a robust solution, which amounts to an inverse robust optimization problem. We consider this problem with a min–max regret objective and present mixed-integer linear programming formulations that can be applied to construct suitable uncertainty sets.Results on both variable-sized uncertainty and inverse problems are further supported with experimental data.     

Robust optimization approximation for joint chance constrained optimization problem

Chance constraint is widely used for modeling solution reliability in optimization problems with uncertainty. Due to the difficulties in checking the feasibility of the probabilistic constraint and the non-convexity of the feasible region, chance constrained problems are generally solved through approximations. Joint chance constrained problem enforces that several constraints are satisfied simultaneously and it is more complicated than individual chance constrained problem. This work investigates the tractable robust optimization approximation framework for solving the joint chance constrained problem. Various robust counterpart optimization formulations are derived based on different types of uncertainty set. To improve the quality of robust optimization approximation, a two-layer algorithm is proposed. The inner layer optimizes over the size of the uncertainty set, and the outer layer optimizes over the parameter t which is used for the indicator function upper bounding. Numerical studies demonstrate that the proposed method can lead to solutions close to the true solution of a joint chance constrained problem.     

Robust optimization with applications to game theory

In this article, we investigate robust optimization equilibria with two players, in which each player can neither evaluate his opponent's strategy nor his own cost matrix accurately while may estimate a bounded set of the strategy or cost matrix. We obtain a result that solving this equilibria can be formulated as solving a second-order cone complementarity problem under an ellipsoid uncertainty set or a mixed complementarity problem under a box uncertainty set. We present some numerical results to illustrate the behaviour of robust optimization equilibria.