Statistical methods in multi-speaker automatic speech recognition

Automatic speech recognition and understanding (ASR) plays an important role in the framework of man-machine communication. Substantial industrial developments are at present in progress in this area. However, after 40 years or so of efforts several fundamental questions remain open. This paper is concerned with a comparative study of four different methods for multi-speaker word recognition: (i) clustering of acoustic templates, (ii) comparison with a finite state automaton, (iii) dynamic programming and vector quantization, (iv) stochastic Markov sources. In order to make things comparable, the four methods were tested with the same material made up of the ten digits (0 to 9) pronounced four times by 60 different speakers (30 males and 30 females). We will distinguish in our experiments between multi-speaker systems (capable of recognizing words pronounced by speakers that have been used during the training phase of the system) and speaker-independent systems (capable of recognizing words pronounced by speakers totally unknown to the system). Half of the corpus (15 male and 15 female) were used for training, and the remaining part for test.     

A primal-dual augmented Lagrangian

Nonlinearly constrained optimization problems can be solved by minimizing a sequence of simpler unconstrained or linearly constrained subproblems. In this paper, we consider the formulation of subproblems in which the objective function is a generalization of the Hestenes-Powell augmented Lagrangian function. The main feature of the generalized function is that it is minimized with respect to both the primal and the dual variables simultaneously. The benefits of this approach include: (i) the ability to control the quality of the dual variables during the solution of the subproblem; (ii) the availability of improved dual estimates on early termination of the subproblem; and (iii) the ability to regularize the subproblem by imposing explicit bounds on the dual variables. We propose two primal-dual variants of conventional primal methods: a primal-dual bound constrained Lagrangian (pdBCL) method and a primal-dual ℓ ₁ linearly constrained Lagrangian (pdℓ ₁LCL) method. Finally, a new sequential quadratic programming (pdSQP) method is proposed that uses the primal-dual augmented Lagrangian as a merit function.     

Optimal control applied on an HIV‐1 within‐host model

The treatment of human immunodeficiency virus (HIV) remains a major challenge, even if significant progress has been made in infection treatment by ‘drug cocktails’. Nowadays, research trend is to minimize the number of pills taken when treating infection. In this paper, an HIV‐1 within host model where healthy cells follow a simple logistic growth is considered. Basic reproduction number of the model is calculated using next generation matrix method, steady states are derived; their local, as well as global stability, is discussed using the Routh–Hurwitz criteria, Lyapunov functions and the Lozinskii measure approach. The optimal control policy is formulated and solved as an optimal control problem. Numerical simulations are performed to compare several cases, representing a treatment by Interleukin2 alone, classical treatment by multitherapy drugs alone, then both treatments at the same time. Objective functionals aim to (i) minimize infected cells quantity; (ii) minimize free virus particles number; and (iii) maximize healthy cells density in blood. Copyright © 2015 John Wiley & Sons, Ltd.     

Locally Optimal Risk-Sensitive Controllers for Strict-Feedback Nonlinear Systems1,2

For a class of risk-sensitive nonlinear stochastic control problems with dynamics in strict-feedback form, we obtain through a constructive derivation state-feedback controllers which (i) are locally optimal, (ii) are globally inverse optimal, and (iii) lead to closed-loop system trajectories that are bounded in probability. The first feature implies that a linearized version of these controllers solve a linear exponential-quadratic Gaussian (LEQG) problem, and the second feature says that there exists an appropriate cost function according to which these controllers are optimal.     

Inverse problem for structural acoustic interaction

We consider an inverse problem of determining a source term for a structural acoustic partial differential equation (PDE) model that is comprised of a two- or a three-dimensional interior acoustic wave equation coupled to an elastic plate equation. The coupling takes place across a boundary interface. For this PDE system, we obtain uniqueness and stability estimates for the source term from a single measurement of boundary values of the “structure” (acceleration of the elastic plate). The proof of uniqueness is based on a Carleman estimate (first version) of the wave problem within the chamber. The proof of stability relies on three main points: (i) a more refined Carleman estimate (second version) and its resulting implication, a continuous observability-type estimate; (ii) a compactness/uniqueness argument; (iii) an operator theoretic approach for obtaining the needed regularity in terms of the initial conditions.     

Performance comparison of the NWF and DC methods for implementing High-Resolution schemes in a fully coupled incompressible flow solver

This paper reports on the use of the Normalized Weighting Factor (NWF) method and the Deferred Correction (DC) approach for the implementation of High Resolution (HR) convective schemes in an implicit, fully coupled, pressure-based flow solver. Four HR schemes are realized within the framework of the NWF and DC methods and employed to solve the following three laminar flow problems: (i) lid-driven flow in a square cavity, (ii) sudden expansion in a square cavity, and (iii) flow in a planar T-junction, over three grid systems with sizes of 10⁴, 5 × 10⁴, and 3 × 10⁵ control volumes. The merit of both approaches is demonstrated by comparing the computational costs required to solve these problems using the various HR schemes on the different grid systems. Whereas previous attempts to use the NWF method in a segregated flow solver failed to produce converged solutions, current results clearly demonstrate that both methods are suitable for utilization in a coupled flow solver. In terms of CPU efficiency, there is no global and consistent superiority of any method over another even though the DC method outperformed the NWF method in two of the three test problems solved.     

On Potts Model Clustering,Kernel K-Means and Density Estimation

Many clustering methods, such as K -means, kernel K -means, and MNcut clustering, follow the same recipe: (i) choose a measure of similarity between observations; (ii) define a figure of merit assigning a large value to partitions of the data that put similar observations in the same cluster; and (iii) optimize this figure of merit over partitions. Potts model clustering represents an interesting variation on this recipe. Blatt, Wiseman, and Domany defined a new figure of merit for partitions that is formally similar to the Hamiltonian of the Potts model for ferromagnetism, extensively studied in statistical physics. For each temperature T, the Hamiltonian defines a distribution assigning a probability to each possible configuration of the physical system or, in the language of clustering, to each partition. Instead of searching for a single partition optimizing the Hamiltonian, they sampled a large number of partitions from this distribution for a range of temperatures. They proposed a heuristic for choosing an appropriate temperature and from the sample of partitions associated with this chosen temperature, they then derived what we call a consensus clustering: two observations are put in the same consensus cluster if they belong to the same cluster in the majority of the random partitions. In a sense, the consensus clustering is an “average” of plausible configurations, and we would expect it to be more stable (over different samples)than the configuration optimizing the Hamiltonian.

The goal of this article is to contribute to the understanding of Potts model clustering and to propose extensions and improvements: (1) We show that the Hamiltonian used in Potts model clustering is closely related to the kernel K -means and MNCutcriteria. (2) We propose a modification of the Hamiltonian penalizing unequal clustersizes and show that it can be interpreted as a weighted version of the kernel K -meanscriterion. (3) We introduce a new version of the Wolff algorithm to simulate configurations from the distribution defined by the penalized Hamiltonian, leading to penalized Potts model clustering. (4) We note a link between kernel based clustering methods and nonparametric density estimation and exploit it to automatically determine locally adaptive kernel bandwidths. (5) We propose a new simple rule for selecting a good temperature T.

As an illustration we apply Potts model clustering to gene expression data and compare our results to those obtained by model based clustering and a nonparametric dendrogram sharpening method.     

Robust discrimination under a hierarchy on the scatter matrices

Under normality, Flury and Schmid [Quadratic discriminant functions with constraints on the covariances matrices: some asymptotic results, J. Multivariate Anal. 40 (1992) 244-261] investigated the asymptotic properties of the quadratic discrimination procedure under hierarchical models for the scatter matrices, that is: (i) arbitrary scatter matrices, (ii) common principal components, (iii) proportional scatter matrices and (iv) identical matrices. In this paper, we study the properties of robust quadratic discrimination rules based on robust estimates of the involved parameters. Our analysis is based on the partial influence functions of the functionals related to these parameters and allows to derive the asymptotic variances of the estimated coefficients under models (i)-(iv). From them, we conclude that the asymptotic variances verify the same order relations as those obtained by Flury and Schmid [Quadratic discriminant functions with constraints on the covariances matrices: some asymptotic results, J. Multivariate Anal. 40 (1992) 244-261] for the classical estimators. We also perform a Monte Carlo study for different sample sizes and different hierarchies which shows the advantage of using robust procedures over classical ones, when anomalous data are present. It also confirms that better rates of misclassification can be achieved if a more parsimonious model among all the correct ones is used instead of the standard quadratic discrimination.     

Torsion Free Commutator Subgroups of Generalized Coxeter Groups

In this note we consider generalized Coxeter groups and we study the problem of when their commutator subgroup is torsion free. As a consequence we describe all (i) Coxeter groups, (ii) triangle groups and (iii) index two orientation preserving subgroups of the finite co-volume hyperbolic Coxeter tetrahedra, for which the commutator subgroup is torsion free.     

Uniform boundary stabilization of a wave equation with nonlinear acoustic boundary conditions and nonlinear boundary damping

We consider a wave equation with nonlinear acoustic boundary conditions. This is a nonlinearly coupled system of hyperbolic equations modeling an acoustic/structure interaction, with an additional boundary damping term to induce both existence of solutions as well as stability. Using the methods of Lasiecka and Tataru for a wave equation with nonlinear boundary damping, we demonstrate well-posedness and uniform decay rates for solutions in the finite energy space, with the results depending on the relationship between (i) the mass of the structure, (ii) the nonlinear coupling term, and (iii) the size of the nonlinear damping. We also show that solutions (in the linear case) depend continuously on the mass of the structure as it tends to zero, which provides rigorous justification for studying the case where the mass is equal to zero.     

Parallel collocation solution of index-1 BVP-DAEs arising from constrained optimal control problems

The indirect solution of constrained optimal control problems gives rise to two-point boundary value problems (BVPs) that involve index-1 differential-algebraic equations (DAEs) and inequality constraints. This paper presents a parallel collocation algorithm for the solution of these inequality constrained index-1 BVP-DAEs. The numerical algorithm is based on approximating the DAEs using piecewise polynomials on a nonuniform mesh. The collocation method is realized by requiring that the BVP-DAE be satisfied at Lobatto points within each interval of the mesh. A Newton interior-point method is used to solve the collocation equations, and maintain feasibility of the inequality constraints. The implementation of the algorithm involves: (i) parallel evaluation of the collocation equations; (ii) parallel evaluation of the system Jacobian; and (iii) parallel solution of a boarded almost block diagonal (BABD) system to obtain the Newton search direction. Numerical examples show that the parallel implementation provides significant speedup when compared to a sequential version of the algorithm.     

Strong stability and uniform decay of solutions to a wave equation with semilinear porous acoustic boundary conditions

We consider a wave equation with semilinear porous acoustic boundary conditions. This is a coupled system of second and first order in time partial differential equations, with possibly semilinear boundary conditions on the interface. The results obtained are (i) strong stability for the linear model, (ii) exponential decay rates for the energy of the linear model, and (iii) local exponential decay rates for the energy of the semilinear model. This work builds on a previous result showing generation of a well-posed dynamical system. The main tools used in the proofs are (i) the Stability Theorem of Arendt-Batty, (ii) energy methods used in the study of a wave equation with boundary damping, and (iii) an abstract result of I. Lasiecka applicable to hyperbolic-like systems with nonlinearly perturbed boundary conditions.     

Strategies for time-dependent PDE control with inequality constraints using an integrated modeling and simulation environment

In Neitzel et al. (Strategies for time-dependent PDE control using an integrated modeling and simulation environment. Part one: problems without inequality constraints. Technical Report 408, Matheon, Berlin, 2007) we have shown how time-dependent optimal control for partial differential equations can be realized in a modern high-level modeling and simulation package. In this article we extend our approach to (state) constrained problems. “Pure” state constraints in a function space setting lead to non-regular Lagrange multipliers (if they exist), i.e. the Lagrange multipliers are in general Borel measures. This will be overcome by different regularization techniques. To implement inequality constraints, active set methods and barrier methods are widely in use. We show how these techniques can be realized in a modeling and simulation package. We implement a projection method based on active sets as well as a barrier method and a Moreau Yosida regularization, and compare these methods by a program that optimizes the discrete version of the given problem. Ira Neitzel’s research was supported by the DFG Schwerpunktprogramm SPP 1253. Uwe Prüfert’s research was supported by the DFG Research Center Matheon. Thomas Slawig’s research was supported by the DFG Cluster of Excellence The Future Ocean and the DFG Schwerpunktprogramm SPP 1253. Website     

Lyapunov type functions for classes of autonomous parabolic feedback control problems and applications

In this note we provide sufficient conditions for the existence of a Lyapunov function for a class of parabolic feedback control problems. The results are applied to the long-time behavior of state functions for the following problems: (i) a model of combustion in porous media; (ii) a model of conduction of electrical impulses in nerve axons; and (iii) a climate energy balance model.     

Exploration of a hybrid feature selection algorithm

In the Knowledge Discovery Process, classification algorithms are often used to help create models with training data that can be used to predict the classes of untested data instances. While there are several factors involved with classification algorithms that can influence classification results, such as the node splitting measures used in making decision trees, feature selection is often used as a pre-classification step when using large data sets to help eliminate irrelevant or redundant attributes in order to increase computational efficiency and possibly to increase classification accuracy. One important factor common to both feature selection as well as to classification using decision trees is attribute discretization, which is the process of dividing attribute values into a smaller number of discrete values. In this paper, we will present and explore a new hybrid approach, ChiBlur, which involves the use of concepts from both the blurring and χ²-based approaches to feature selection, as well as concepts from multi-objective optimization. We will compare this new algorithm with algorithms based on the blurring and χ²-based approaches.     

Some new two-weight codes and strongly regular graphs

It is well known (and due to Delsarte [3]) that the three concepts (i) two-weight projective code, (ii) strongly regular graph defined by a difference set in a vector space, and (iii) subset X of a projective space such that |X∩H| takes only two values when H runs over all hyperplanes, are equivalent. Here we construct some new examples (formulated as in (iii)) by taking a quadric defined over a small field and cutting out a quadratic defined over a larger field.     

Discrete Approximations for Singularly Perturbed Boundary Value Problems with Parabolic Layers,I

In this series of three papers we study singularly perturbed (SP) boundary value problems for equations of elliptic and parabolic type. For small values of the perturbation parameter parabolic boundary and interior layers appear in these problems. If classical discretisation methods are used, the solution of the finite difference scheme and the approximation of the diffusive flux do not converge uniformly with respect to this parameter. Using the method of special, adapted grids, we can construct difference schemes that allow approximation of the solution and the normalised diffusive flux uniformly with respect to the small parameter. We also consider singularly perturbed boundary value problems for convection-diffusion equations. Also for these problems we construct special finite difference schemes, the solution of which converges $ε$-uniformly. We study what problems appear, when classical schemes are used for the approximation of the spatial derivatives. We compare the results with those obtained by the adapted approach. Results of numerical experiments are discussed. In the three papers we first give an introduction on the general problem, and then we consider respectively (i) Problems for SP parabolic equations, for which the solution and the normalised diffusive fluxes are required; (ii) Problems for SP elliptic equations with boundary conditions of Dirichlet, Neumann and Robin type; (iii) Problems for SP parabolic equation with discontinuous boundary conditions.     

Interior-Point Methods for Nonconvex Nonlinear Programming: Filter Methods and Merit Functions

Recently, Fletcher and Leyffer proposed using filter methods instead of a merit function to control steplengths in a sequential quadratic programming algorithm. In this paper, we analyze possible ways to implement a filter-based approach in an interior-point algorithm. Extensive numerical testing shows that such an approach is more efficient than using a merit function alone.     

Properties of a family of generalized NCP-functions and a derivative free algorithm for complementarity problems

In this paper, we propose a new family of NCP-functions and the corresponding merit functions, which are the generalization of some popular NCP-functions and the related merit functions. We show that the new NCP-functions and the corresponding merit functions possess a system of favorite properties. Specially, we show that the new NCP-functions are strongly semismooth, Lipschitz continuous, and continuously differentiable; and that the corresponding merit functions have SC¹$S{C}^1$ property (i.e., they are continuously differentiable and their gradients are semismooth) and LC¹$L{C}^1$ property (i.e., they are continuously differentiable and their gradients are Lipschitz continuous) under suitable assumptions. Based on the new NCP-functions and the corresponding merit functions, we investigate a derivative free algorithm for the nonlinear complementarity problem and discuss its global convergence. Some preliminary numerical results are reported.