Adaptive greedy approximations

The problem of optimally approximating a function with a linear expansion over a redundant dictionary of waveforms is NP-hard. The greedy matching pursuit algorithm and its orthogonalized variant produce suboptimal function expansions by iteratively choosing dictionary waveforms that best match the function’s structures. A matching pursuit provides a means of quickly computing compact, adaptive function approximations. Numerical experiments show that the approximation errors from matching pursuits initially decrease rapidly, but the asymptotic decay rate of the errors is slow. We explain this behavior by showing that matching pursuits are chaotic, ergodic maps. The statistical properties of the approximation errors of a pursuit can be obtained from the invariant measure of the pursuit. We characterize these measures using group symmetries of dictionaries and by constructing a stochastic differential equation model. We derive a notion of the coherence of a signal with respect to a dictionary from our characterization of the approximation errors of a pursuit. The dictionary elements slected during the initial iterations of a pursuit correspond to a function’s coherent structures. The tail of the expansion, on the other hand, corresponds to a noise which is characterized by the invariant measure of the pursuit map. When using a suitable dictionary, the expansion of a function into its coherent structures yields a compact approximation. We demonstrate a denoising algorithm based on coherent function expansions.     

Sampling and multilevel coarsening algorithms for fast matrix approximations

This paper addresses matrix approximation problems for matrices that are large, sparse, and/or representations of large graphs. To tackle these problems, we consider algorithms that are based primarily on coarsening techniques, possibly combined with random sampling. A multilevel coarsening technique is proposed, which utilizes a hypergraph associated with the data matrix and a graph coarsening strategy based on column matching. We consider a number of standard applications of this technique as well as a few new ones. Among standard applications, we first consider the problem of computing partial singular value decomposition, for which a combination of sampling and coarsening yields significantly improved singular value decomposition results relative to sampling alone. We also consider the column subset selection problem, a popular low‐rank approximation method used in data‐related applications, and show how multilevel coarsening can be adapted for this problem. Similarly, we consider the problem of graph sparsification and show how coarsening techniques can be employed to solve it. We also establish theoretical results that characterize the approximation error obtained and the quality of the dimension reduction achieved by a coarsening step, when a proper column matching strategy is employed. Numerical experiments illustrate the performances of the methods in a few applications.     

A fast genetic method for inducting descriptive fuzzy models

Under certain inference mechanisms, fuzzy rule bases can be regarded as extended additive models. This relationship can be applied to extend some statistical techniques to learn fuzzy models from data. The interest in this parallelism is twofold: theoretical and practical. First, extended additive models can be estimated by means of the matching pursuit algorithm, which has been related to Support Vector Machines, Boosting and Radial Basis neural networks learning; this connection can be exploited to better understand the learning of fuzzy models. In particular, the technique we propose here can be regarded as the counterpart to boosting fuzzy classifiers in the field of fuzzy modeling. Second, since matching pursuit is very efficient in time, we can expect to obtain faster algorithms to learn fuzzy rules from data. We show that the combination of a genetic algorithm and the backfitting process learns faster than ad hoc methods in certain datasets.     

Model recovery for Hammerstein systems using the auxiliary model based orthogonal matching pursuit method

This article investigates parameter and order identification of a block-oriented Hammerstein system by using the orthogonal matching pursuit method in the compressive sensing theory which deals with how to recover a sparse signal in a known basis with a linear measurement model and a small set of linear measurements. The idea is to parameterize the Hammerstein system into the linear measurement model containing a measurement matrix with some unknown variables and a sparse parameter vector by using the key variable separation principle, then an auxiliary model based orthogonal matching pursuit algorithm is presented to recover the sparse vector.The standard orthogonal matching pursuit algorithm with a known measurement matrix is a popular recovery strategy by picking the supporting basis and the corresponding non-zero element of a sparse signal in a greedy fashion. In contrast to this, the auxiliary model based orthogonal matching pursuit algorithm has unknown variables in the measurement matrix. For a K-sparse signal, the standard orthogonal matching pursuit algorithm takes a fixed number of K stages to pick K columns (atoms) in the measurement matrix, while the auxiliary model based orthogonal matching pursuit algorithm takes steps larger than K to pick K atoms in the measurement matrix with the process of picking and deleting atoms, due to the gradually accurate estimates of the unknown variables step by step.The auxiliary model based orthogonal matching pursuit algorithm can simultaneously identify parameters and orders of the Hammerstein system, and has a high efficient identification performance.     

One-Bit Compressed Sensing by Greedy Algorithms

Sign truncated matching pursuit (STrMP) algorithm is presented in this paper. STrMP is a new greedy algorithm for the recovery of sparse signals from the sign measurement, which combines the principle of consistent reconstruction with orthogonal matching pursuit (OMP). The main part of STrMP is as concise as OMP and hence STrMP is simple to implement. In contrast to previous greedy algorithms for one-bit compressed sensing, STrMP only need to solve a convex and unconstrained subproblem at each iteration. Numerical experiments show that STrMP is fast and accurate for one-bit compressed sensing compared with other algorithms.     

A memetic algorithm for the virtual network mapping problem

The Internet has ossified. It has lost its capability to adapt as requirements change. A promising technique to solve this problem is the introduction of network virtualization. Instead of directly using a single physical network, working just well enough for a limited range of applications, multiple virtual networks are embedded on demand into the physical network, each of them perfectly adapted to a specific application class. The challenge lies in mapping the different virtual networks with all the resources they require into the available physical network, which is the core of the virtual network mapping problem. In this work, we introduce a memetic algorithm that significantly outperforms the previously best algorithms for this problem. We also offer an analysis of the influence of different problem representations and in particular the implementation of a uniform crossover for the grouping genetic algorithm that may also be interesting outside of the virtual network mapping domain. Furthermore, we study the influence of different hybridization techniques and the behaviour of the developed algorithm in an online setting.     

Integer programming formulations for the minimum weighted maximal matching problem

Given an undirected graph, the problem of finding a maximal matching that has minimum total weight is NP-hard. This problem has been studied extensively from a graph theoretical point of view. Most of the existing literature considers the problem in some restricted classes of graphs and give polynomial time exact or approximation algorithms. On the contrary, we consider the problem on general graphs and approach it from an optimization point of view. In this paper, we develop integer programming formulations for the minimum weighted maximal matching problem and analyze their efficacy on randomly generated graphs. We also compare solutions found by a greedy approximation algorithm, which is based on the literature, against optimal solutions. Our results show that our integer programming formulations are able to solve medium size instances to optimality and suggest further research for improvement.     

Convergence and restart in branch-and-bound algorithms for global optimization. Application to concave minimization and D.C. Optimization problems

A general branch-and-bound conceptual scheme for global optimization is presented that includes along with previous branch-and-bound approaches also grid-search techniques. The corresponding convergence theory, as well as the question of restart capability for branch-and-bound algorithms used in decomposition or outer approximation schemes are discussed. As an illustration of this conceptual scheme, a finite branch-and-bound algorithm for concave minimization is described and a convergent branch-and-bound algorithm, based on the previous one, is developed for the minimization of a difference of two convex functions.     

Algorithm for Optimal Triangulations in Scattered Data Representation and Implementation

Scattered data collected at sample points may be used to determine simple functions to best fit the data. An ideal choice for these simple functions is bivariate splines. Triangulation of the sample points creates partitions over which the bivariate splines may be defined. But the optimality of the approximation is dependent on the choice of triangulation. An algorithm, referred to as an Edge Swapping Algorithm, has been developed to transform an arbitrary triangulation of the sample points into an optimal triangulation for representation of the scattered data. A Matlab package has been completed that implements this algorithm for any triangulation on a given set of sample points.     

Sparse-grid finite-volume multigrid for 3D-problems

We introduce a multigrid algorithm for the solution of a second order elliptic equation in three dimensions. For the approximation of the solution we use a partially ordered hierarchy of finite-volume discretisations. We show that there is a relation with semicoarsening and approximation by more-dimensional Haar wavelets. By taking a proper subset of all possible meshes in the hierarchy, a sparse grid finite-volume discretisation can be constructed.The multigrid algorithm consists of a simple damped point-Jacobi relaxation as the smoothing procedure, while the coarse grid correction is made by interpolation from several coarser grid levels.The combination of sparse grids and multigrid with semi-coarsening leads to a relatively small number of degrees of freedom,N, to obtain an accurate approximation, together with anO(N) method for the solution. The algorithm is symmetric with respect to the three coordinate directions and it is fit for combination with adaptive techniques.To analyse the convergence of the multigrid algorithm we develop the necessary Fourier analysis tools. All techniques, designed for 3D-problems, can also be applied for the 2D case, and — for simplicity — we apply the tools to study the convergence behaviour for the anisotropic Poisson equation for this 2D case.     

An algorithm for fitting data over a circle using tensor product splines

An algorithm is described for surface fitting over a circle by using tensor product splines which satisfy certain boundary conditions. This algorithm is an extension of an existing one for fitting data over a rectangle. The knots of the splines are chosen automatically but a single parameter must be specified to control the tradeoff between closeness of fit and smoothness of fit. The algorithm can easily be generalized for fitting data over any domain that can be described in polar coordinates. Constraints at the boundaries of this approximation domain can be imposed.     

最短时限缺省指派问题的一种解法

将周良泽在 1998年提出的最短时限缺省指派问题转化成赋权二分图的最小权 K-匹配问题。研究了其解的最优性充分及必要条件 ,并给出了适合在图上求解的生长树法及适合在表上直接求解的标号法 ,最后给出一个实例。该解法是一种较简便的算法。     

Branch-and-Bound Outer Approximation Algorithm for Sum-of-Ratios Fractional Programs

In this article, a branch and-bound outer approximation algorithm is presented for globally solving a sum-of-ratios fractional programming problem. To solve this problem, the algorithm instead solves an equivalent problem that involves minimizing an indefinite quadratic function over a nonempty, compact convex set. This problem is globally solved by a branch-and-bound outer approximation approach that can create several closed-form linear inequality cuts per iteration. In contrast to pure outer approximation techniques, the algorithm does not require computing the new vertices that are created as these cuts are added. Computationally, the main work of the algorithm involves solving a sequence of convex programming problems whose feasible regions are identical to one another except for certain linear constraints. As a result, to solve these problems, an optimal solution to one problem can potentially be used to good effect as a starting solution for the next problem.     

Approximation Algorithms for Pick-and-Place Robots

In this paper we study the problem of finding placement tours for pick-and-place robots, also known as the printed circuit board assembly problem with m positions on a board, n bins containing m components and n locations for the bins. In the standard model where the working time of the robot is proportional to the distances travelled, the general problem appears as a combination of the travelling salesman problem and the matching problem, and for m=n we have an Euclidean, bipartite travelling salesman problem. We give a polynomial-time algorithm which achieves an approximation guarantee of 3+. An important special instance of the problem is the case of a fixed assignment of bins to bin-locations. This appears as a special case of a bipartite TSP satisfying the quadrangle inequality and given some fixed matching arcs. We obtain a 1.8 factor approximation with the stacker crane algorithm of Frederikson, Hecht and Kim. For the general bipartite case we also show a 2.0 factor approximation algorithm which is based on a new insertion technique for bipartite TSPs with quadrangle inequality. Implementations and experiments on real-world as well as random point configurations conclude this paper.     

A multivariate QD-like algorithm

The problem of constructing a univariate rational interpolant or Padé approximant for given data can be solved in various equivalent ways: one can compute the explicit solution of the system of interpolation or approximation conditions, or one can start a recursive algorithm, or one can obtain the rational function as the convergent of an interpolating or corresponding continued fraction.In case of multivariate functions general order systems of interpolation conditions for a multivariate rational interpolant and general order systems of approximation conditions for a multivariate Padé approximant were respectively solved in [6] and [9]. Equivalent recursive computation schemes were given in [3] for the rational interpolation case and in [5] for the Padé approximation case. At that moment we stated that the next step was to write the general order rational interpolants and Padé approximants as the convergent of a multivariate continued fraction so that the univariate equivalence of the three main defining techniques was also established for the multivariate case: algebraic relations, recurrence relations, continued fractions. In this paper a multivariate qd-like algorithm is developed that serves this purpose.     

Nonlinear function approximation: Computing smooth solutions with an adaptive greedy algorithm

In contrast to linear schemes, nonlinear approximation techniques allow for dimension independent rates of convergence. Unfortunately, typical algorithms (such as, e.g., backpropagation) are not only computationally demanding, but also unstable in the presence of data noise. While we can show stability for a weak relaxed greedy algorithm, the resulting method has the drawback that it requires in practise unavailable smoothness information about the data.In this work we propose an adaptive greedy algorithm which does not need this information but rather recovers it iteratively from the available data. We show that the generated approximations are always at least as smooth as the original function and that the algorithm also remains stable, when it is applied to noisy data. Finally, the applicability of this algorithm is demonstrated by numerical experiments.     

Efficient Algorithms for Variants of Weighted Matching and Assignment Problems

Obtaining a matching in a graph satisfying a certain objective is an important class of graph problems. Matching algorithms have received attention for several decades. However, while there are efficient algorithms to obtain a maximum weight matching, not much is known about the maximum weight maximum cardinality, and maximum cardinality maximum weight matching problems for general graphs. Our contribution in this work is to show that for bounded weight input graphs one can obtain an algorithm for both maximum weight maximum cardinality (for real weights), and maximum cardinality maximum weight matching (for integer weights) by modifying the input and running the existing maximum weight matching algorithm. Also, given the current state of the art in maximum weight matching algorithms, we show that, for bounded weight input graphs, both maximum weight maximum cardinality, and maximum cardinality maximum weight matching have algorithms of similar complexities to that of maximum weight matching. Subsequently, we also obtain approximation algorithms for maximum weight maximum cardinality, and maximum cardinality maximum weight matching.      

An algorithm for surface reconstruction from planar contours using smoothing splines

This paper presents a fast algorithm for constructing a smooth three-dimensional surface over a set of cross-sectional contours. We assume that these sections are perpendicular to the z-axis and first consider the case that the surface can be represented in cylindrical coordinates. An approximation is then determined for r(θ, z) by using tensor product splines which satisfy certain boundary constraints. The algorithm is an extension of an existing semi-automatic surface fitting algorithm. The knots of the spline are chosen automatically but a single parameter is expected to control the tradeoff between closeness of fit and smoothness of fit.Both open and closed surfaces can be represented. In particular we demonstrate the use of a non-linear transformation for obtaining smooth closed surfaces.The algorithm can easily be extended to the reconstruction of surfaces which cannot be represented in cylindrical coordinates. A number of applications are also briefly discussed such as the calculation of volumes and the intersection with other surfaces. We have applied the method in practice to obtain a 3-D integrated image of the cerebral blood vessels and CT view of tumor for stereotactic neurosurgery.     

On the use of suboptimal matchings for scaling and ordering sparse symmetric matrices

The use of matchings is a powerful technique for scaling and ordering sparse matrices prior to the solution of a linear system Ax = b. Traditional methods such as implemented by the HSL software package MC64 use the Hungarian algorithm to solve the maximum weight maximum cardinality matching problem. However, with advances in the algorithms and hardware used by direct methods for the parallelization of the factorization and solve phases, the serial Hungarian algorithm can represent an unacceptably large proportion of the total solution time for such solvers. Recently, auction algorithms and approximation algorithms have been suggested as alternatives for achieving near‐optimal solutions for the maximum weight maximum cardinality matching problem. In this paper, the efficacy of auction and approximation algorithms as replacements for the Hungarian algorithm is assessed in the context of sparse symmetric direct solvers when used in problems arising from a range of practical applications. High‐cardinality suboptimal matchings are shown to be as effective as optimal matchings for the purposes of scaling. However, matching‐based ordering techniques require that matchings are much closer to optimality before they become effective. The auction algorithm is demonstrated to be capable of finding such matchings significantly faster than the Hungarian algorithm, but our ‐approximation matching approach fails to consistently achieve a sufficient cardinality. Copyright © 2015 The Authors Numerical Linear Algebra with Applications Published by John Wiley & Sons Ltd.     

A fully polynomial bicriteria approximation scheme for the constrained spanning tree problem

We propose a fully polynomial bicriteria approximation scheme for the constrained spanning tree problem. First, an exact pseudo-polynomial algorithm is developed based on a two-variable extension of the well-known matrix-tree theorem. The scaling and approximate binary search techniques are then utilized to yield a fully polynomial approximation scheme.