State and parameter estimation for nonlinear systems

State and parameter estimators are obtained for systems described by nonlinear evolution equations. Linear infinite dimensional observability theory together with a variety of fixed point theorems can be employed to obtain a finite time observer. Moreover, a nonlinear asymptotic observer is produced using stability results. The problem of joint state and parameter estimation is converted to the state estimation case, via an augmented state, so that these observer results can be utilised. Examples and remarks on the generality of the results are given.     

Two fundamental convergence theorems for nonlinear conjugate gradient methods and their applications

1. IntroductionWe consider the global convergence of conjugate gradient methods for the unconstrainednonlinear optimization problemadn f(x),where f: Re - RI is continuously dtherelltiable and its gradiellt is denoted by g. Weconsider only the cajse where the methods are implemented without regular restarts. Theiterative formula is given byXk 1 = Xk Akdk, (1'1).and the seaxch direction da is defined bywhere gb is a scalar, ^k is a stenlength, and gb denotes g(xk).The best-known formulas fo…     

Optimal control of systems governed by a class of nonlinear evolution equations in a reflexive Banach space

The paper presents a closure theorem for the attainable trajectories of a class of control systems governed by a large class of nonlinear evolution equations in reflexive Banach spaces. Several existence theorems for optimal controls are proven that include a terminal control problem, a time-optimal control problem, and a special Bolza problem. Some results of independent interest are also presented.This work was supported in part by the National Research Council of Canada under Grant No. 7109.The authors would like to thank Professor L. Cesari for pointing out that joint continuity off is required for the setsG andR to satisfy the upper semicontinuity property (Theorems 5.1 and 5.2).     

Maximum-likelihood recursive nonlinear filtering

The basic model for the general nonlinear filtering problem consists of a nonlinear plant driven by noise followed by nonlinear observation with additive noise. The object is to estimate, at each instant, the current state of the plant, given thea priori information and the history of the observations up to the current time. The estimation procedure studied here is that of computing, at each instant, the most probable trajectory given the data at that time, and taking its final value. The purpose of the present paper is to clarify some earlier studies of this procedure.This research was partially supported by the National Science Foundation, Grant No. GK-806.     

Bemerkungen zum Gradientenverfahren

Summary Certain iterative methods are applied to unconstrained optimization problems in a general Hilbert space. Convergence and monotonicity theorems are proved for gradient, steepest descent, Newton and some related methods that converge for arbitrary initial values and converge quadratically in a neighbourhood of the optimum.     

Equidistribution of negative statistics and quotients of Coxeter groups of type B and D

We generalize some identities and q-identities previously known for the symmetric group to Coxeter groups of type B and D. The extended results include theorems of Foata and Schützenberger, Gessel, and Roselle on various distributions of statistics, like inversion number, major index, and descent number. In order to show our results we provide explicit characterizations of the systems of minimal coset representatives of Coxeter groups of type B and D.     

A three-parameter family of nonlinear conjugate gradient methods

In this paper, we propose a three-parameter family of conjugate gradient methods for unconstrained optimization. The three-parameter family of methods not only includes the already existing six practical nonlinear conjugate gradient methods, but subsumes some other families of nonlinear conjugate gradient methods as its subfamilies. With Powell's restart criterion, the three-parameter family of methods with the strong Wolfe line search is shown to ensure the descent property of each search direction. Some general convergence results are also established for the three-parameter family of methods. This paper can also be regarded as a brief review on nonlinear conjugate gradient methods.

     

Shadowing Homoclinic Chains to a Symplectic Critical Manifold

We prove the existence of trajectories shadowing chains of heteroclinic orbits to a symplectic normally hyperbolic critical manifold of a Hamiltonian system.The results are quite different for real and complex eigenvalues. General results are applied to Hamiltonian systems depending on a parameter which slowly changes with rate ε. If the frozen autonomous system has a hyperbolic equilibrium possessing transverse homoclinic orbits, we construct trajectories shadowing homoclinic chains with energy having quasirandom jumps of order ε and changing with average rate of orderε| ln ε|. This provides a partial multidimensional extension of the results of A. Neishtadt on the destruction of adiabatic invariants for systems with one degree of freedom and a figure 8 separatrix.     

An Efficient Hybrid Conjugate Gradient Method for Unconstrained Optimization

Recently, we propose a nonlinear conjugate gradient method, which produces a descent search direction at every iteration and converges globally provided that the line search satisfies the weak Wolfe conditions. In this paper, we will study methods related to the new nonlinear conjugate gradient method. Specifically, if the size of the scalar _k with respect to the one in the new method belongs to some interval, then the corresponding methods are proved to be globally convergent; otherwise, we are able to construct a convex quadratic example showing that the methods need not converge. Numerical experiments are made for two combinations of the new method and the Hestenes–Stiefel conjugate gradient method. The initial results show that, one of the hybrid methods is especially efficient for the given test problems.     

Theory of pseudo-orbit shadowing in dynamical systems

This is a survey of the main results obtained in the theory of pseudo-orbit shadowing in dynamical systems in the first decade of the 21st century. The main directions are shadowing and structural stability, C ¹-interiors of sets of systems with the shadowing property, shadowing properties equivalent to the structural stability, and the denseness problem.     

Nonlinear Leray-Schauder alternatives and application to nonlinear problem arising in the theory of growing cell population

Motivated by a mathematical model of an age structured proliferating cell population, we state some new variants of Leray-Schauder type fixed point theorems for (ws)-compact operators. Further, we apply our results to establish some new existence and locality principles for nonlinear boundary value problem arising in the theory of growing cell population in L ₁-setting. Besides, a topological structure of the set of solutions is provided.     

Continuous Generalized Gradient Descent

This article derives characterizations and computational algorithms for continuous general gradient descent trajectories in high-dimensional parameter spaces for statistical model selection, prediction, and classification. Examples include proportional gradient shrinkage as an extension of LASSO and LARS, threshold gradient descent with right-continuous variable selectors, threshold ridge regression, and many more with proper combinations of variable selectors and functional forms of a kernel. In all these problems, general gradient descent trajectories are continuous piecewise analytic vector-valued curves as solutions to matrix differential equations. We show the monotonicity and convergence of the proposed algorithms in the loss or negative likelihood functions. We prove that approximations of continuous solutions via infinite series expansions are computationally more efficient and accurate compared with discretization methods. We demonstrate the applicability of our algorithms through numerical experiments with real and simulated datasets.     

Superlinear convergence theorems for Newton-type methods for nonlinear systems of equations

In this paper, someQ-order convergence theorems are given for the problem of solving nonlinear systems of equations when using very general finitely terminating methods for the solution of the associated linear systems. The theorems differ from those of Dembo, Eisenstat, and Steihaug in the different stopping condition and in their applicability to the nonlinear ABS algorithm.Lecture presented at the University of Bergamo, Bergamo, Italy, October 1989.     

Slow Passage Through the Nonhyperbolic Homoclinic Orbit of the Saddle-Center Hamiltonian Bifurcation

Slowly varying Hamiltonian systems, for which action is a well-known adiabatic invariant, are considered in the case where the system undergoes a saddle center bifurcation. We analyze the situation in which the solution slowly passes through the nonhyperbolic homoclinic orbit created at the saddle-center bifurcation. The solution near this homoclinic orbit consists of a large sequence of homoclinic orbits surrounded by near approaches to the autonomous nonlinear nonhyperbolic saddle point. By matching this solution to the strongly nonlinear oscillations obtained by averaging before and after crossing the homoclinic orbit, we determine the change in the action. If one orbit comes sufficiently close to the nonlinear saddle point, then that one saddle approach instead satisfies the nonautonomous first Painlevé equation, whose stable manifold of the unstable saddle (created in the saddle-center bifurcation) separates solutions approaching the stable center from those involving sequences of nearly homoclinic orbits.     

Convergence Rates of Multivariate Regression Estimators with Errors-In-Variables

This paper studies the regression estimation with errors-in-variables. We first extend Meister’s theorems (Meister, 2009. Deconvolution Problems in Nonparametric Statistics. Springer, Berlin) from one to multi-dimensional setting, when a noise density has no zeros in the Fourier domain. Then motivated by the work of Delaigle and Meister (Delaigle, Meister, 2011. Nonparametric function estimation under Fourier-oscillating noise. Statistica Sinica 21, 1065–1092), we show a desired convergence rate of a kernel estimator for Fourier-oscillating noises. Finally, two technical conditions are removed, when a wavelet estimator is used.     

The relationship between theorems of the alternative,least norm problems,steepest descent directions,and degeneracy: A review

Recently, it has been observed that several nondifferentiable minimization problems share the property that the question of whether a given point is optimal can be answered by solving a certain bounded least squares problem. If the resulting residual vector,r, vanishes then the current point is optimal. Otherwise,r is a descent direction. In fact, as we shall see,r points at the steepest descent direction. On the other hand, it is customary to characterize the optimality conditions (and the steepest descent vector) of a convex nondifferentiable function via its subdifferential. Also, it is well known that optimality conditions are usually related to theorems of the alternative. One aim of our survey is to clarify the relations between these subjects. Another aim is to introduce a new type of theorems of the alternative. The new theorems characterize the optimality conditions of discretel ₁ approximation problems and multifacility location problems, and provide a simple way to obtain the subdifferential and the steepest descent direction in such problems. A further objective of our review is to demonstrate that the ability to compute the steepest descent direction at degenerate dead points opens a new way for handling degeneracy in active set methods.     

The controllability problem for nonlinear Volterra systems

We discuss the controllability of systems whose dynamics are governed by a large class of nonlinear Volterra integral equations. The property of controllability is shown to be equivalent to the existence of a fixed point of a certain set-valued map. We show that convexity and seminormality conditions intimately related to those assumed in proofs of existence theorems for optimal controls are sufficient to guarantee controllability. Approximate controllability results are obtained by first introducing generalized solutions and then showing, under only mild additional restrictions, that ordinary solutions are dense in this broader class of trajectories.Dedicated to L. CesariTu se' lo mio maestro e il mio autore: Tu se' solo colui, da cui io tolsi Lo bello stile che m' ha fatto onore. Dante, Canto I: 85–87This work was written while the author was on leave to the Institut für Numerische und Angewandte Mathematik, Universität Göttingen, Göttingen, West Germany.     

Functional Optimal Estimation Problems and Their Solution by Nonlinear Approximation Schemes

The design of state estimators for nonlinear dynamic systems affected by disturbances is addressed in a functional optimization framework. The estimator contains an innovation function that has to be chosen within a suitably defined class of functions in such a way to minimize a cost functional given by the worst-case ratio of the ℒ _p norms of the estimation error and the disturbances. Since this entails an infinite-dimensional optimization problem that under general hypotheses cannot be solved analytically, an approximate solution is sought by minimizing the cost functional over linear combinations of simple “basis functions,” represented by computational units with adjustable parameters. The selection of the parameters is made by solving a constrained nonlinear programming problem, where the constraints are given by pointwise conditions that ensure the well-definiteness of the functional and the existence of a solution. Penalty terms are introduced in the cost function to account for constraints imposed on points that result from sampling the sets to which the trajectories of the state and of the estimation error belong. To ensure an efficient covering of the sets, low-discrepancy sampling techniques are exploited that generate samples deterministically spread in a uniform way, without leaving regions of the space undersampled. Work supported by a PRIN grant from the Italian Ministry of University and Research (Project “New Techniques for the Identification and Adaptive Control of Industrial Systems”) and by the EU and the Regione Liguria trough the Regional Programs of Innovative Action of the European Regional Development Fund.     

Degenerate billiards in celestial mechanics

In an ordinary billiard trajectories of a Hamiltonian system are elastically reflected after a collision with a hypersurface (scatterer). If the scatterer is a submanifold of codimension more than one, we say that the billiard is degenerate. Degenerate billiards appear as limits of systems with singularities in celestial mechanics. We prove the existence of trajectories of such systems shadowing trajectories of the corresponding degenerate billiards. This research is motivated by the problem of second species solutions of Poincaré.     

A family of derivative-free conjugate gradient methods for large-scale nonlinear systems of equations

In this paper, we propose a family of derivative-free conjugate gradient methods for large-scale nonlinear systems of equations. They come from two modified conjugate gradient methods [W.Y. Cheng, A two term PRP based descent Method, Numer. Funct. Anal. Optim. 28 (2007) 1217–1230; L. Zhang, W.J. Zhou, D.H. Li, A descent modified Polak–Ribiére–Polyak conjugate gradient method and its global convergence, IMA J. Numer. Anal. 26 (2006) 629–640] recently proposed for unconstrained optimization problems. Under appropriate conditions, the global convergence of the proposed method is established. Preliminary numerical results show that the proposed method is promising.