A self‐guided search for good local minima of the sum‐of‐squared‐error in nonlinear least squares regression

Hard modeling of nonlinear chemical or biological systems is highly relevant as a model function together with values for model parameters provides insights in the systems' functionalities. Deriving values for said model parameters via nonlinear regression, however, is challenging as usually one of the numerous local minima of the sum‐of‐squared errors (SSEs) is determined; furthermore, for different starting points, different minima may be found. Thus, nonlinear regression is prone to low accuracy and low reproducibility. Therefore, there is a need for a generally applicable, automated initialization of nonlinear least squares algorithms, which reaches a good, reproducible solution after spending a reasonable computation time probing the SSE‐hypersurface. For this purpose, a three‐step methodology is presented in this study. First, the SSE‐hypersurface is randomly probed in order to estimate probability density functions of initial model parameter that generally lead to an accurate fit solution. Second, these probability density functions then guide a high‐resolution sampling of the SSE‐hypersurface. This second probing focuses on those model parameter ranges that are likely to produce a low SSE. As the probing continues, the most appropriate initial guess is retained and eventually utilized in a subsequent nonlinear regression. It is shown that this “guided random search” derives considerably better regression solutions than linearization of model functions, which has so far been considered the best‐case scenario. Examples from infrared spectroscopy, cell culture monitoring, reaction kinetics, and image analyses demonstrate the broad and successful applicability of this novel method. Copyright © 2014 John Wiley & Sons, Ltd.     

Kinetic analysis of nonisothermal solid-state reactions: determination of the kinetic parameters by means of a nonlinear regression method

A nonlinear regression method has been proposed for a simultaneous calculation of the activation energy, frequency factor, and reaction order from a single TG curve. This method was based on the new temperature integral approximation proposed in this paper and the Levenberg–Marquardt method. The newly proposed nonlinear regression method was applied for determining the kinetic parameters from two simulated TG curves. The results of the calculations were compared with values obtained by the traditional method. It can be concluded from this comparison that the new nonlinear regression method is more accurate than the traditional method for the determination of the kinetic parameters of solid-state heterogeneous reactions.     

Adsorption of methylene blue dye from aqueous solution by agricultural waste: Equilibrium,thermodynamics, kinetics,mechanism and process design

The adsorption of methylene blue (MB) dye from aqueous solution onto a cashew nut shell (CNS) was investigated as a function of parameters such as solution pH, CNS dose, contact time, initial MB dye concentration and temperature. The CNS was shown to be effective for the quantitative removal of MB dye, and the equilibrium was reached in 60 min. The experimental data were analysed by two-parameter isotherms (Langmuir, Freundlich, Temkin and Dubinin-Radushkevich models) using nonlinear regression analysis. The characteristic parameters for each isotherm and the related correlation coefficients were determined. Thermodynamic parameters such as ΔG°, ΔH° and ΔS° were also evaluated, the sorption process was found to be spontaneous and exothermic. Pseudo-first-order, pseudo-second-order and Elovich kinetic models were used to analyze the adsorption process. The results of the kinetic study suggest that the adsorption of MB dye matches the pseudo-second-order equation, suggesting that the adsorption process is presumably chemisorption. The adsorption process was found to be controlled by both surface and pore diffusion. Analysis of adsorption data using a Boyd kinetic plot confirmed that the external mass transfer is a rate determining step in the sorption process. A single-stage batch adsorber was designed for different CNS doses to effluent volume ratios using the Freundlich equation. The results indicated that the CNS could be used effectively to adsorb MB dye from aqueous solutions.     

Massively parallel high-energy time-dependent wave-packet calculations

Summary The exact solution of the time-dependent Schrödinger equation is obtained using a parallel implementation of the standard grid techniques. Most of the operations involved in this calculation may be executed concurrently for each of the grid points. For the few operations which may not be executed concurrently, we have implemented parallel algorithms. In our two-dimensional implementation on the Connection Machine, we have obtained optimal speed-up — that is, by usingN processors we achieve a speed-up which is proportional toN. In addition to the discussion of our 2-dimensional implementation, we shall discuss our proposed 3-dimensional implementation of these grid techniques.     

The ellipsoid algorithm as a method for the determination of polypeptide conformations from experimental distance constraints and energy minimization

A new method for constrained nonlinear optimization known as the ellipsoid algorithm is evaluated as a means of determining and refining the conformations of peptides. Advantages of the ellipsoid algorithm over conventional optimization methods include that it avoids many local minima that other methods would be trapped by, and that it is sometimes able to find optimum solutions in which the constraints are satisfied exactly. The dihedral angles about single bonds were used as variables to keep the dimensionality low (the rate of convergence decreases rapidly with increasing dimensionality of the problem). The method is evaluated on problems involving distance constraints, and for minimization of conformational energy functions. In an initial application, conformations consistent with an experimental set of NMR distance constraints were obtained in a problem involving 48 variable dihedral angles.     

Investigating the performance of parallel eigensolvers for large processor counts

Summary Eigensolving (diagonalizing) small dense matrices threatens to become a bottleneck in the application of massively parallel computers to electronic structure methods. Because the computational cost of electronic structure methods typically scales asO(N ³) or worse, even teraflop computer systems with thousands of processors will often confront problems withN 10,000. At present, diagonalizing anN×N matrix onP processors is not efficient whenP is large compared toN. The loss of efficiency can make diagonalization a bottleneck on a massively parallel computer, even though it is typically a minor operation on conventional serial machines. This situation motivates a search for both improved methods and identification of the computer characteristics that would be most productive to improve.In this paper, we compare the performance of several parallel and serial methods for solving dense real symmetric eigensystems on a distributed memory message passing parallel computer. We focus on matrices of sizeN=200 and processor countsP=1 toP=512, with execution on the Intel Touchstone DELTA computer. The best eigensolver method is found to depend on the number of available processors. Of the methods tested, a recently developed Blocked Factored Jacobi (BFJ) method is the slowest for smallP, but the fastest for largeP. Its speed is a complicated non-monotonic function of the number of processors used. A detailed performance analysis of the BFJ method shows that: (1) the factor most responsible for limited speedup is communication startup cost; (2) with current communication costs, the maximum achievable parallel speedup is modest (one order of magnitude) compared to the best serial method; and (3) the fastest solution is often achieved by using less than the maximum number of available processors.Pacific Northwest Laboratory is operated for the U.S. Department of Energy (DOE) by Battelle Memorial Institute under contract DE-AC06-76RLO 1830     

An efficient parallelization of phosphorylated peptide and protein identification

Protein sequence database search based on tandem mass spectrometry is an essential method for protein identification. As the computational demand increases, parallel computing has become an important technique for accelerating proteomics data analysis. In this paper, we discuss several factors which could affect the runtime of the pFind search engine and build an estimation model. Based on this model, effective on‐line and off‐line scheduling methods were developed. An experiment on the public dataset from PhosphoPep consisting of 100 RAW files of phosphopeptides shows that the speedup on 100 processors is 83.7. The parallel version can complete the identification task within 9 min, while a stand‐alone process on a single PC takes more than 10 h. On another larger dataset consisting of 1 366 471 spectra, the speedup on 320 processors is 258.9 and the efficiency is 80.9%. Our approach can be applied to other similar search engines. Copyright © 2010 John Wiley & Sons, Ltd.     

Capillary electrophoresis study of human serum albumin binding to basic drugs

Summary The applicability of capillary electrophoresis/frontal analysis (CE/FA) for determining the binding constants of the drugs propranolol (PRO) and verapamil (VER) to human serum albumin (HSA) was investigated. After direct hydrodynamic injection of a drug-HAS mixture solution into a coated capillary (32 cm × 50 μm i.d.), the basic drug was eluted as a zonal peak with a plateau region under condition of phosphate buffer (pH 7.4; ionic strength 0.17) at 12 kV positive running voltage. The unbound drug concentrations measured from the plateau peak heights had good correlation coefficients,r>0.999. Employing the Scatchard plot, the Klotz plot and nonlinear regression, the drug protein binding parameters, the binding constant and the number of binding sites on one protein molecule, were obtained. The binding constant obtained was compared to a reported equilibrium dialysis result and they are basically in good agreement.     

The dependence of the variances of the parameters in non-linear regression analysis on the number of data points

In designing an experimentin which non-linear regression analysis is used to evaluate the physicochemical parameters that chracterize the behavior of the system being studied, it would be useful to be able to predict how the number of data points affects th reliabilities of the values obtained. The relation between these depends on hos the points are distributed as well as on the nature of the equation to which thd data are fitted. Weighted non-linear regression analysis has been applied to six common equations involving a total of fourteen parameters, using a number of n of points that was varied from 4 to     

Software news and updates

A parallel version of the valence bond program TURTLE has been developed. In this version the calculation of matrix elements is distributed over the processors. The implementation has been done using the message‐passing interface (MPI), and is, therefore, portable. The parallel version of the program is shown to be quite efficient with a speed‐up of 55 at 64 processors. © 2001 John Wiley & Sons, Inc. J Comput Chem 22: 665–672, 2001     

A comparison between two massively parallel algorithms for Monte Carlo computer simulation: An investigation in the grand canonical ensemble

We present a comparison between two different approaches to parallelizing the grand canonical Monte Carlo simulation technique (GCMC) for classical fluids: a spatial decomposition and a time decomposition. The spatial decomposition relies on the fact that for short-ranged fluids, such as the cut and shifted Lennard-Jones potential used in this work, atoms separated by a greater distance than the reach of the potential act independently, and thus different processors can work concurrently in regions of the same system which are sufficiently far apart. The time decomposition is an exactly parallel approach which employs simultaneous (GCMC) simulations, one per processor, identical in every respect except the initial random number seed, with the thermodynamic output variables averaged across all processors. While scaling characteristics for the spatial decomposition are presented for 8–1024 processor systems, the comparison between the two decompositions is limited to the 8–128 processor range due to the warm-up time and memory imitations of the time decomposition. Using a combination of speed and statistical efficiency, the two algorithms are compared at two different state points. While the time decomposition reaches a given value of standard error in the system's potential energy more quickly than the spatial decomposition for both densities, the warm-up time demands of the time decomposition quickly become insurmountable as the system size increases. © 1996 by John Wiley & Sons, Inc.     

Constrained kernelized partial least squares

Nonlinear kernel methods have been widely used to deal with nonlinear problems in latent variable methods. However, in the presence of structured noise, these methods have reduced efficacy. We have previously introduced constrained latent variable methods that make use of any available additional knowledge about the structured noise. These methods improve performance by introducing additional constraints into the algorithm. In this paper, we build upon our previous work and introduce hard‐constrained and soft‐constrained nonlinear partial least squares methods using nonlinear kernels. The addition of nonlinear kernels reduces the effects of structured noise in nonlinear spaces and improves the regression performance between the input and response variables. Copyright © 2014 John Wiley & Sons, Ltd.     

Parallel internally contracted multireference configuration interaction

A parallel implementation of the internally contracted (IC) multireference configuration (MRCI) module of the MOLPRO quantum chemistry program is described. The global array (GA) toolkit has been used in order to map an existing disk-paging small-memory algorithm onto a massively parallel supercomputer, where disk storage is replaced by the combined memory of all processors. This model has enabled a rather complicated code to be ported to the parallel environment without the need for the wholesale redesign of algorithms and data structures. Examples show that the parallel ICMRCI program can deliver results in a fraction of the time needed for equivalent uncontracted MRCI computations. Further examples demonstrate that ICMRCI computations with up to 10⁷ variational parameters, and equivalent to uncontracted MRCI with 10⁹ configurations, are feasible. The largest calculation demonstrates a parallel efficiency of about 80% on 128 nodes of a Cray T3E-300. © 1998 John Wiley & Sons, Inc. J Comput Chem 19: 1215–1228, 1998     

Assessment of the Accuracy of Estimating Adsorption Parameters of Organic Substances from Differential Capacitance Curves: Monte-Carlo Simulation of the Random-Error Effect in Capacitance Measurements

Monte-Carlo method is used for elucidating the effect of a random normally distributed error of capacitance measurements on estimated adsorption parameters of organic substances on an ideally polarizable electrode. Within the model of two parallel capacitors and a Frumkin isotherm with preset adsorption parameters, initial differential capacitance vs. potential dependences are calculated for ten organic-substance concentrations. Then, a random normally distributed error is introduced at every point in these dependences. For estimating the adsorption parameters, 100 series of dependences thus obtained are processed by a nonlinear regression analysis technique. Histograms of distributions of found parameters are plotted. The accuracy of the parameter calculation is assessed for 5 and 10% errors of capacitance measurements.     

Implementation of a parallel direct SCF algorithm on distributed memory computers

A parallel direct self-consistent field (SCF) algorithm for distributed memory computers is described. Key features of the algorithm are its ability to achieve a load balance dynamically, its modest memory requirements per processor, and its ability to utilize the full eightfold index permutation symmetry of the two-electron integrals despite the fact that entire copies of the Fock and density matrices are not present in each processor's local memory. The algorithm is scalable and, accordingly, has the potential to function efficiently on hundreds of processors. With the algorithm described here, a calculation employing several thousand basis functions can be carried out on a distributed memory machine with 100 or more processors each with just 4 MBytes of RAM and no disk. The Fock matrix build portion of the algorithm has been implemented on a 16-node Intel iPSC/2. Results from benchmark calculations are encouraging. The algorithm shows excellent load balance when run on 4, 8, or 16 processors and displays almost ideal speed-up in going from 4 to 16 processors. Preliminary benchmark calculations have also been carried out on an Intel Paragon. © 1995 by John Wiley & Sons, Inc.     

Viscous Flow Through Polydisperse Row of Parallel Fibers: Account for the Slip Effect

A general theory of the flow through the flat periodic row of parallel polydisperse fibers randomly arranged at the fiber period is elaborated. An approximate solution of the problem with account for the slip effect at low Reynolds numbers is obtained for normal incident flow. The system of equations for determining local drag forces acting on each fiber in a row, as well as the system of equations for determining local buoyancy forces, are derived. It was demonstrated that, in the approximation obtained, the buoyancy forces are equal to zero, provided that bilateral symmetry is present in the fiber arrangement. The behavior of local forces as a function of row parameters is considered using bimodal row as an example. The form of flow function is established for the velocity field near the arbitrary fiber.     

Potentiometric determination of iron using a fluoride ion-selective electrode-the application of the Apple II-ISE intelligent ion analyzer

A new method for determining iron is based on both nonlinear regression calibration plots and parabolic interpolation using a fluoride ion-selective electrode (ISE) and the Apple II-ISE intelligent ion Analyzer developed by ourselves. The experimental conditions for determining iron are discussed. The appropriate acidity of the experimental solution is pH 3, controlled by total ionic strength adjustment buffer (TISAB) that is composed of glycine (aminoacetic acid), nitric acid and sodium nitrate. The suitable total concentration of fluoride is equal to the highest concentration of iron in the standard series. Because the mathematical model of the method coincides with the experimental data the Apple II-ISE intelligent ion Analyzer can perform data acquisition and data processing, and the performance of fluoride electrode is excellent, the new method for determination of iron is fast and accurate. This method has been used successfully in the determination of iron in mineral samples.     

Parallelizing a molecular dynamics algorithm on a multiprocessor workstation using OpenMP

The atomistic molecular dynamics program YASP has been parallelized for shared-memory computer architectures. Parallelization was restricted to the most CPU-time-consuming parts: neighbor-list construction, calculation of nonbonded, angle and dihedral forces, and constraints. Most of the sequential FORTRAN code was kept; parallel constructs were inserted as compiler directives using the OpenMP standard. Only in the case of the neighbor list did the data structure have to be changed. The parallel code achieves a useful speedup over the sequential version for systems of several thousand atoms and above. On an IBM Regatta p690+, the throughput increases with the number of processors up to a maximum of 12-16 processors depending on the characteristics of the simulated systems. On dual-processor Xeon systems, the speedup is about 1.7.     

Multiparametric curve fitting XV statistical analysis and goodness-of-fit test by the least-squares algorithm MINOPT

Estimation of nonlinear regression quality leads to examination of quality of parameter estimates, a degree of fit, a prediction ability of model proposed and quality of experimental data. Statistical analysis serves for computation of confidence intervals of parameters and confidence bands, the bias of parameters and bias of residuals. Goodness-of-fit test examines classical residuals using various diagnostics and identifies influential points. Mentioned topics of nonlinear model building and testing contained in MINOPT program from CHEMSTAT package are illustrated.