A hybrid optimization approach for automated parameter estimation problems

We present a hybrid optimization approach for solving automated parameter estimation models. The hybrid approach is based on the coupling of the Simultaneous Perturbation Stochastic Approximation (SPSA) [1] and a Newton-Krylov Interior-Point method (NKIP) [2] via a surrogate model. The global method SPSA performs a stochastic search to find target regions with low function values. Next, we generate a surrogate model based on the points of regions on which the local method NKIP algorithm is applied for finding an optimal solution. We illustrate the behavior of the hybrid optimization algorithm on one testcase. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Stability of solutions for a class of nonlinear cone constrained optimization problems,part 2: Application to parameter estimation

In this paper we study a problem of parameter estimation in two point boundary value problems. Using a stability theorem for nonlinear cone constrained optimization problems derived in Part 1 of this paper we investigate stability properties of the solutions of the parameter estimation problem in the output-least-squares formulation.     

Convergence of algorithms for perturbed optimization problems

Infinite-dimensional optimization problems occur in various applications such as optimal control problems and parameter identification problems. If these problems are solved numerically the methods require a discretization which can be viewed as a perturbation of the data of the optimization problem. In this case the expected convergence behavior of the numerical method used to solve the problem does not only depend on the discretized problem but also on the original one. Algorithms which are analyzed include the gradient projection method, conditional gradient method, Newton's method and quasi-Newton methods for unconstrained and constrained problems with simple constraints.     

Analyticity with respect to a parameter of regular solutions of weakly nonlinear singularly perturbed problems

We consider a weakly nonlinear singular perturbed problem with right side independent of time. We show that the regularized asymptotic series obtained by Lomov's method are analytic with respect to the parameter and regularizing variables.Translated from Ukrainskii Matematicheskii Zhurnal, Vol. 42, No. 3, pp. 393—398, March, 1990.     

Improved parameter estimation of the log-logistic distribution with applications

In this paper, we deal with parameter estimation of the log-logistic distribution. It is widely known that the maximum likelihood estimators (MLEs) are usually biased in the case of the finite sample size. This motivates a study of obtaining unbiased or nearly unbiased estimators for this distribution. Specifically, we consider a certain ‘corrective’ approach and Efron’s bootstrap resampling method, which both can reduce the biases of the MLEs to the second order of magnitude. As a comparison, the commonly used generalized moments method is also considered for estimating parameters. Monte Carlo simulation studies are conducted to compare the performances of the various estimators under consideration. Finally, two real-data examples are analyzed to illustrate the potential usefulness of the proposed estimators, especially when the sample size is small or moderate.     

Porosity of perturbed optimization problems in Banach spaces

Let X be a Banach space and Z a nonempty closed subset of X. Let be a lower semicontinuous function bounded from below. This paper is concerned with the perturbed optimization problem infz∈Z{J(z)+‖x−z‖}, denoted by (x,J)-inf for x∈X. In the case when X is compactly fully 2-convex, it is proved in the present paper that the set of all points x in X for which there does not exist z₀∈Z such that J(z₀)+‖x−z₀‖=infz∈Z{J(z)+‖x−z‖} is a σ-porous set in X. Furthermore, if X is assumed additionally to be compactly locally uniformly convex, we verify that the set of all points x∈X?Z⁰ such that the problem (x,J)-inf fails to be approximately compact, is a σ-porous set in X?Z⁰, where Z⁰ denotes the set of all z∈Z such that z∈PZ(z). Moreover, a counterexample to which some results of Ni [R.X. Ni, Generic solutions for some perturbed optimization problem in nonreflexive Banach space, J. Math. Anal. Appl. 302 (2005) 417-424] fail is provided.     

On Schwarz-type smoothers for saddle point problems with applications to PDE-constrained optimization problems

In this paper we consider a (one-shot) multigrid strategy for solving the discretized optimality system (KKT system) of a PDE-constrained optimization problem. In particular, we discuss the construction of an additive Schwarz-type smoother for a certain class of optimal control problems. A rigorous multigrid convergence analysis is presented. Numerical experiments are shown which confirm the theoretical results. The work was supported by the Austrian Science Fund (FWF) under grant SFB 013/F1309.     

Chaotic invasive weed optimization algorithm with application to parameter estimation of chaotic systems

This paper introduces a novel hybrid optimization algorithm by taking advantage of the stochastic properties of chaotic search and the invasive weed optimization (IWO) method. In order to deal with the weaknesses associated with the conventional method, the proposed chaotic invasive weed optimization (CIWO) algorithm is presented which incorporates the capabilities of chaotic search methods. The functionality of the proposed optimization algorithm is investigated through several benchmark multi-dimensional functions. Furthermore, an identification technique for chaotic systems based on the CIWO algorithm is outlined and validated by several examples. The results established upon the proposed scheme are also supplemented which demonstrate superior performance with respect to other conventional methods.     

Systems of generalized quasivariational inclusions problems with applications to variational analysis and optimization problems

In this paper, we study an existence theorem of systems of generalized quasivariational inclusions problem. By this result, we establish the existence theorems of solutions of systems of generalized equations, systems of generalized vector quasiequilibrium problem, collective variational fixed point, systems of generalized quasiloose saddle point, systems of minimax theorem, mathematical program with systems of variational inclusions constraints, mathematical program with systems of equilibrium constraints and systems of bilevel problem and semi-infinite problem with systems of equilibrium problem constraints. This research was supported by the National Science Council of the Republic of China.     

A Kalman particle filter for online parameter estimation with applications to affine models

Statistical Inference for Stochastic Processes - In this paper we address the problem of estimating the posterior distribution of the static parameters of a continuous-time state space model with...     

Recursive state and parameter estimation with applications in water resources

Hydrologic models, as well as measurements of hydrologic processes, are corrupted by noise. The Kalman filter is a convenient tool to estimate the true but unknown state of a hydrologic system. It is, however, difficult to specify the necessary error covariances. A procedure is proposed to estimate the error covariances recursively in a combined state and parameter filter. Applications of the procedure yield meaningful results for two hydrologic data series of very different character. A major benefit of the proposed algorithm seems to be its robustness against instability.     

Some optimization problems with applications to canonical correlations and sphericity tests

Optimization problems are connected with maximization of three functions, namely, geometric mean, arithmetic mean and harmonic mean of the eigenvalues of (X′ΣX)^?1ΣY(Y′ΣY)^?1Y′ΣX, where Σ is positive definite, X and Y are p × r and p × s matrices of ranks r and s (≥r), respectively, and X′Y = 0. Some interpretations of these functions are given. It is shown that the maximum values of these functions are obtained at the same point given by X = (h₁ + ?₁h_p, …, h_r + ?_rh_p?r+1) and $\text{Y = (h}{}_1\text{? ?}{}_1\text{h}{}_{\mathrm{p}}\text{, …, h}{}_{\mathrm{r}}\text{? ?}{}_{\mathrm{r}}\text{h}{}_{\mathrm{p?r+1}}\text{,}\text{Y}{}_{\mathrm{r+1}}\text{, …,}\text{Y}{}_{\mathrm{s}}\text{)}$, where h₁, …, h_p are the eigenvectors of Σ corresponding to the eigenvalues λ₁ ≥ λ₂ ≥ … ≥ λ_p > 0, ?_j = +1 or ?1 for j = 1,2,…, r and $\text{Y}{}_{\mathrm{r+1}}\text{, …,}\text{Y}{}_{\mathrm{s}}$, are linear functions of h_r+1,…, h_p?r. These results are extended to intermediate stationary values. They are utilized in obtaining the inequalities for canonical correlations θ₁,…,θ_r and they are given by expressions (3.8)–(3.10). Further, some new union-intersection test procedures for testing the sphericity hypothesis are given through test statistics (3.11)–(3.13).     

Drift parameter estimation in fractional diffusions driven by perturbed random walks

We estimate the drift parameter in a simple linear model driven by fractional Brownian motion. We propose maximum likelihood estimators (MLE) for the drift parameter construct by using a random walk approximation of the fractional Brownian motion.     

Generalized vector quasi-equilibrium problems with applications to common fixed point theorems and optimization problems

In this paper, we first establish the existence theorems of generalized vector quasi-equilibrium problems. From these results, we establish the existence theorems of common fixed point theorems for two multivalued maps and mathematical programs with an equilibrium constraint as applications.     

A convexification method for a class of global optimization problems with applications to reliability optimization

A convexification method is proposed for solving a class of global optimization problems with certain monotone properties. It is shown that this class of problems can be transformed into equivalent concave minimization problems using the proposed convexification schemes. An outer approximation method can then be used to find the global solution of the transformed problem. Applications to mixed-integer nonlinear programming problems arising in reliability optimization of complex systems are discussed and satisfactory numerical results are presented.     

Mixing times with applications to perturbed Markov chains

A measure of the “mixing time” or “time to stationarity” in a finite irreducible discrete time Markov chain is considered. The statistic , where {π_j} is the stationary distribution and m_ij is the mean first passage time from state i to state j of the Markov chain, is shown to be independent of the initial state i (so that η_i = η for all i), is minimal in the case of a periodic chain, yet can be arbitrarily large in a variety of situations. An application considering the effects perturbations of the transition probabilities have on the stationary distributions of Markov chains leads to a new bound, involving η, for the 1-norm of the difference between the stationary probability vectors of the original and the perturbed chain. When η is large the stationary distribution of the Markov chain is very sensitive to perturbations of the transition probabilities.     

Stability of minimal solutions to parametric set optimization problems

In this paper we study the stability of the minimal solutions of set optimization problems. We provide sufficient conditions for the upper and lower semicontinuity and compactness of the minimal solutions of parametric set optimization problems whose objective values are not necessarily compact.     

Stability of solutions for variational relation problems with applications

In this paper, we prove that most of problems in variational relations (in the sense of Baire category) are essential and that, for any problem in variational relations, there exists at least one essential component of its solution set. As applications, we deduce the existence of essential components of the set of Ky Fan’s points based on Ky Fan’s minimax inequality theorem, the existence of essential components of the set of Nash equilibrium points for general n-person non-cooperative games, the existence of essential component of the set of solutions for vector Ky Fan’s minimax inequality, the existence of essential components of the set of KKM points and the existence of essential components of the set of solutions for Ky Fan’s section theorem.     

Approximate solutions in set-valued optimization problems with applications to maximal monotone operators

Positivity - This paper is devoted to the study of efficient elements for set-valued maps. We propose two new notions of relative weak $$\epsilon $$ -efficient element and strict relative weak...