A computational study of local search algorithms for Italian high-school timetabling

In this paper we report on a computational experience with a local search algorithm for High-school Timetabling Problems. The timetable has to satisfy “hard” requirements, that are mandatory, and should minimize the violation of “soft” constraints. In our approach, we combine Simulated Annealing with a Very Large-Scale Neighborhood search where the neighborhood is explored by solving an Integer Programming problem. We report on a computational experience validating the usefulness of the proposed approach. Support for I. Vasil’ev was provided by NATO grant CBP.NR.RIG.911258.     

Convex programming with single separable constraint and bounded variables

In this paper a minimization problem with convex objective function subject to a separable convex inequality constraint “≤” and bounded variables (box constraints) is considered. We propose an iterative algorithm for solving this problem based on line search and convergence of this algorithm is proved. At each iteration, a separable convex programming problem with the same constraint set is solved using Karush-Kuhn-Tucker conditions. Convex minimization problems subject to linear equality/ linear inequality “≥” constraint and bounds on the variables are also considered. Numerical illustration is included in support of theory.     

A Discrete Lagrangian-Based Global-Search Method for Solving Satisfiability Problems

Satisfiability is a class of NP-complete problems that model a wide range of real-world applications. These problems are difficult to solve because they have many local minima in their search space, often trapping greedy search methods that utilize some form of descent. In this paper, we propose a new discrete Lagrange-multiplier-based global-search method (DLM) for solving satisfiability problems. We derive new approaches for applying Lagrangian methods in discrete space, we show that an equilibrium is reached when a feasible assignment to the original problem is found and present heuristic algorithms to look for equilibrium points. Our method and analysis provides a theoretical foundation and generalization of local search schemes that optimize the objective alone and penalty-based schemes that optimize the constraints alone. In contrast to local search methods that restart from a new starting point when a search reaches a local trap, the Lagrange multipliers in DLM provide a force to lead the search out of a local minimum and move it in the direction provided by the Lagrange multipliers. In contrast to penalty-based schemes that rely only on the weights of violated constraints to escape from local minima, DLM also uses the value of an objective function (in this case the number of violated constraints) to provide further guidance. The dynamic shift in emphasis between the objective and the constraints, depending on their relative values, is the key of Lagrangian methods. One of the major advantages of DLM is that it has very few algorithmic parameters to be tuned by users. Besides the search procedure can be made deterministic and the results reproducible. We demonstrate our method by applying it to solve an extensive set of benchmark problems archived in DIMACS of Rutgers University. DLM often performs better than the best existing methods and can achieve an order-of-magnitude speed-up for some problems.     

Variable space search for graph coloring

Let G=(V,E) be a graph with vertex set V and edge set E. The k-coloring problem is to assign a color (a number chosen in {1,…,k}) to each vertex of G so that no edge has both endpoints with the same color. We propose a new local search methodology, called Variable Space Search, which we apply to the k-coloring problem. The main idea is to consider several search spaces, with various neighborhoods and objective functions, and to move from one to another when the search is blocked at a local optimum in a given search space. The k-coloring problem is thus solved by combining different formulations of the problem which are not equivalent, in the sense that some constraints are possibly relaxed in one search space and always satisfied in another. We show that the proposed algorithm improves on every local search used independently (i.e., with a unique search space), and is competitive with the currently best coloring methods, which are complex hybrid evolutionary algorithms.     

A column generation algorithm for the vehicle routing problem with soft time windows

The Vehicle Routing Problem with Time Windows consists of computing a minimum cost set of routes for a fleet of vehicles of limited capacity visiting a given set of customers with known demand, with the additional constraint that each customer must be visited in a specified time window. We consider the case in which time window constraints are relaxed into “soft” constraints, that is penalty terms are added to the solution cost whenever a vehicle serves a customer outside of his time window. We present a branch-and-price algorithm which is the first exact optimization algorithm for this problem.     

A proximal method for identifying active manifolds

The minimization of an objective function over a constraint set can often be simplified if the “active manifold” of the constraints set can be correctly identified. In this work we present a simple subproblem, which can be used inside of any (convergent) optimization algorithm, that will identify the active manifold of a “prox-regular partly smooth” constraint set in a finite number of iterations.     

Attraction probabilities in variable neighborhood search

Empirical evidence demonstrates that when the same local search operator is used, variable neighborhood search consistently outperforms random multistart local search on all types of combinatorial and global optimization problems tested. In this paper we suggest that this superiority in performance may be explained by the distribution of the attraction basins around a current solution as a function of the distance from the solution. We illustrate with a well-known instance of the multisource Weber problem that the “attraction probabilities” for finding better solutions can be orders of magnitude larger in neighborhoods that are close to the current solution. The paper also discusses the global convergence properties of both general methods in the context of attraction probabilities.     

A continuous approach to combinatorial optimization: application of water system pump operations

In this paper, we have suggested a penalty method to modify the combinatorial optimization problem with the linear constraints to a global optimization problem with linear constraints. It also deals with a topic of vital significance of pump operation optimization in a water system. In this connection we have done a lot of work to formulate a model based on a simplified flow volume balance to resolve the problem of optimal pump operation settings of switching “ON” and “OFF” with the reduced gradient method. This global solution approach incorporates some benefits for practical application to a real system as is shown in the case study.     

Conditional Viability for Impulse Differential Games

We introduce in this paper the concept of “impulse evolutionary game”. Examples of evolutionary games are usual differential games, differentiable games with history (path-dependent differential games), mutational differential games, etc. Impulse evolutionary systems and games cover in particular “hybrid systems” as well as “qualitative systems”. The conditional viability kernel of a constrained set (with a target) is the set of initial states such that for all strategies (regarded as continuous feedbacks) played by the second player, there exists a strategy of the first player such that the associated run starting from this initial state satisfies the constraints until it hits the target. This paper characterizes the concept of conditional viability kernel for “qualitative games” and of conditional valuation function for “qualitative games” maximinimizing an intertemporal criterion. The theorems obtained so far about viability/capturability issues for evolutionary systems, conditional viability for differential games and about impulse and hybrid systems are used to provide characterizations of conditional viability under impulse evolutionary games.     

The return of the linear search problem

The linear search problem concerns a search on the real line for a point selected at random according to a given probability distribution. The search begins at zero and is made by a continuous motion with constant speed, first in one direction and then the other. The problem is to determine when it is possible to devise a “best” search plan. In former papers the best plan has been selected according to the criterion of minimum expected path length. In this paper we consider a more general, nonlinear criterion for a “best” plan and show that the substantive requirements of the earlier results are not affected by these changes.     

The sweet mystery of compatibilism

Any satisfactory account of freedom must capture, or at least permit, the mysteriousness of freedom—a “sweet” mystery involving a certain kind of ignorance rather than a “sour” mystery of unintelligibility, incoherence, or unjustifiedness. I argue that compatibilism can capture the sweet mystery of freedom. I argue first that an action is free if and only if a certain “rationality constraint” is satisfied, and that nothing in standard libertarian accounts of freedom entails its satisfaction. Satisfaction of this constraint is consistent with the universal causal predetermination of action (UCP). If UCP is true and the rationality constraint satisfied, there’s a sense in which our actions are explanatorily (though not necessarily causally) overdetermined. While it seems plausible (given UCP) that our actions are so overdetermined, it seems utterly mysterious why they should be so overdetermined. Compatibilism’s capacity to accommodate this mystery is a mark in its favor.     

Asymptotics of the optimal time in a singular perturbed linear time-optimal problem

We consider an abstract attainability problem under constraints of asymptotic character and describe a general approach to constructing “nonsequential” attraction sets in the space of generalized elements formalized as finitely additive measures. We also study the existence and structure of an asymptotic formula universal in the range of “asymptotic constraints” and topologies of the space of generalized elements in the case when the space of ordinary solutions is not necessarily compactifiable.     

Analysis of space–time discontinuous Galerkin method for nonlinear convection–diffusion problems

The paper presents the theory of the discontinuous Galerkin finite element method for the space–time discretization of a nonstationary convection–diffusion initial-boundary value problem with nonlinear convection and linear diffusion. The problem is not singularly perturbed with dominating convection. The discontinuous Galerkin method is applied separately in space and time using, in general, different space grids on different time levels and different polynomial degrees p and q in space and time dicretization. In the space discretization the nonsymmetric, symmetric and incomplete interior and boundary penalty (NIPG, SIPG, IIPG) approximation of diffusion terms is used. The paper is concerned with the proof of error estimates in “L ²(L ²)”- and “DG”-norm formed by the “L ²(H ¹)”-seminorm and penalty terms. A special technique based on the use of the Gauss–Radau interpolation and numerical integration has been used for the derivation of an abstract error estimate. In the “DG”-norm the error estimates are optimal with respect to the size of the space grid. They are optimal with respect to the time step, if the Dirichlet boundary condition has behaviour in time as a polynomial of degree ≤ q.     

An iterated local search algorithm for the vehicle routing problem with convex time penalty functions

We propose an iterated local search algorithm for the vehicle routing problem with time window constraints. We treat the time window constraint for each customer as a penalty function, and assume that it is convex and piecewise linear. Given an order of customers each vehicle to visit, dynamic programming (DP) is used to determine the optimal start time to serve the customers so that the total time penalty is minimized. This DP algorithm is then incorporated in the iterated local search algorithm to efficiently evaluate solutions in various neighborhoods. The amortized time complexity of evaluating a solution in the neighborhoods is a logarithmic order of the input size (i.e., the total number of linear pieces that define the penalty functions). Computational comparisons on benchmark instances with up to 1000 customers show that the proposed method is quite effective, especially for large instances.     

On numerical optimization theory of infinite kernel learning

In Machine Learning algorithms, one of the crucial issues is the representation of the data. As the given data source become heterogeneous and the data are large-scale, multiple kernel methods help to classify “nonlinear data”. Nevertheless, the finite combinations of kernels are limited up to a finite choice. In order to overcome this discrepancy, a novel method of  “infinite”  kernel combinations is proposed with the help of infinite and semi-infinite programming regarding all elements in kernel space. Looking at all infinitesimally fine convex combinations of the kernels from the infinite kernel set, the margin is maximized subject to an infinite number of constraints with a compact index set and an additional (Riemann–Stieltjes) integral constraint due to the combinations. After a parametrization in the space of probability measures, it becomes semi-infinite. We adapt well-known numerical methods to our infinite kernel learning model and analyze the existence of solutions and convergence for the given algorithms. We implement our new algorithm called “infinite” kernel learning (IKL) on heterogenous data sets by using exchange method and conceptual reduction method, which are well known numerical techniques from solve semi-infinite programming. The results show that our IKL approach improves the classifaction accuracy efficiently on heterogeneous data compared to classical one-kernel approaches.     

Local Search Proximal Algorithms as Decision Dynamics with Costs to Move

Acceptable moves for the “worthwhile-to-move” incremental principle are such that “advantages-to-move” are higher than some fraction of “costs-to-move”. When combined with optimization, this principle gives raise to adaptive local search proximal algorithms. Convergence results are given in two distinctive cases, namely low local costs-to-move and high local costs-to-move. In this last case, one obtains a dynamic cognitive approach to Ekeland’s ϵ-variational principle. Introduction of costs-to-move in the algorithms yields robustness and stability properties.     

The spherical constraint in Boolean quadratic programs

We propose a new approach to bound Boolean quadratic optimization problems. The idea is to re-express the Boolean constraints as one “spherical” constraint, whose dualization amounts to semidefinite least-squares problems. Studying this dualization provides an alternative interpretation of the sdp relaxation. It also reveals a new class of non-convex problems with no duality gap.     

Univariate Global Optimization with Multiextremal Non-Differentiable Constraints Without Penalty Functions

This paper proposes a new algorithm for solving constrained global optimization problems where both the objective function and constraints are one-dimensional non-differentiable multiextremal Lipschitz functions. Multiextremal constraints can lead to complex feasible regions being collections of isolated points and intervals having positive lengths. The case is considered where the order the constraints are evaluated is fixed by the nature of the problem and a constraint i is defined only over the set where the constraint i−1 is satisfied. The objective function is defined only over the set where all the constraints are satisfied. In contrast to traditional approaches, the new algorithm does not use any additional parameter or variable. All the constraints are not evaluated during every iteration of the algorithm providing a significant acceleration of the search. The new algorithm either finds lower and upper bounds for the global optimum or establishes that the problem is infeasible. Convergence properties and numerical experiments showing a nice performance of the new method in comparison with the penalty approach are given. This research was supported by the following grants: FIRB RBNE01WBBB, FIRB RBAU01JYPN, and RFBR 04-01-00455-a. The author thanks Prof. D. Grimaldi for proposing the application discussed in the paper. An erratum to this article is available at .     

The Economic Lot Scheduling Problem without Capacity Constraints

This study presents a comprehensive analysis on the Economic Lot Scheduling Problem (ELSP) without capacity constraints. We explore the optimality structure of the ELSP without capacity constraints and discover that the curve for the optimal objective values is piecewise convex with repsect to B, i.e., the values of basic period. The theoretical properties of the junction points on the piecewise convex curve not only provides us the information on “which product i” to modify, but also on “where on the B-axis” to change the set of optimal multpliers in the search process. By making use of the junction points, we propose an effective search algorithm to secure a global optimal solution for the ELSP without capacity constraints. Also, we use random experiments to verify that the proposed algorithm is efficient. The results in this paper lay important foundation for deriving an efficient heuristic to solve the conventional ELSP with capacity constraints.