On learning of Sigmoid Neural Networks

The Probably Approximately Correct (PAC) learning theory creates a framework to assess the learning properties of static models for which the data are assumed to be independently and identically distributed (i.i.d.). One important family of dynamic models to which the conventional PAC learning can not be applied is nonlinear Finite Impulse Response (FIR) models. The present article, using an extension of PAC learning that covers learning with m-dependent data, the learning properties of FIR modeling with sigmoid neural networks are evaluated. These results include upper bounds on the size of the data set required to train FIR sigmoid neural networks, provided that the input data are uniformly distributed. © 2001 John Wiley & Sons, Inc.     

Generalization bounds of ERM algorithm with Markov chain samples

One of the main goals of machine learning is to study the generalization performance of learning algorithms. The previous main results describing the generalization ability of learning algorithms are usually based on independent and identically distributed （i.i.d.） samples. However, independence is a very restrictive concept for both theory and real-world applications. In this paper we go far beyond this classical framework by establishing the bounds on the rate of relative uniform convergence for the Empirical Risk Minimization （ERM） algorithm with uniformly ergodic Markov chain samples. We not only obtain generalization bounds of ERM algorithm, but also show that the ERM algorithm with uniformly ergodic Markov chain samples is consistent. The established theory underlies application of ERM type of learning algorithms.     

Measuring overall efficiency and effectiveness using DEA

This paper presents a framework where data envelopment analysis (DEA) is used to measure overall efficiency and show how to apply this framework to assess effectiveness for more general behavioral goals. The relationships between various cone-ratio DEA models and models to measure overall efficiency are clarified. Specifically it is shown that as multiplier cones tighten, the cone-ratio DEA models converge to measures of overall efficiency. Furthermore, it is argued that multiplier cone and cone-ratio model selection must be consistent with the behavioral goals assigned or assumed for purposes of analysis. Consistent with this reasoning, two new models are introduced to measure effectiveness when value measures are represented by separable or linked cones, where the latter can be used to analyze profit-maximizing effectiveness.     

A neural fuzzy control system with structure and parameter learning

A general connectionist model, called neural fuzzy control network (NFCN), is proposed for the realization of a fuzzy logic control system. The proposed NFCN is a feedforward multilayered network which integrates the basic elements and functions of a traditional fuzzy logic controller into a connectionist structure which has distributed learning abilities. The NFCN can be constructed from supervised training examples by machine learning techniques, and the connectionist structure can be trained to develop fuzzy logic rules and find membership functions. Associated with the NFCN is a two-phase hybrid learning algorithm which utilizes unsupervised learning schemes for structure learning and the backpropagation learning scheme for parameter learning. By combining both unsupervised and supervised learning schemes, the learning speed converges much faster than the original backpropagation algorithm. The two-phase hybrid learning algorithm requires exact supervised training data for learning. In some real-time applications, exact training data may be expensive or even impossible to obtain. To solve this problem, a reinforcement neural fuzzy control network (RNFCN) is further proposed. The RNFCN is constructed by integrating two NFCNs, one functioning as a fuzzy predictor and the other as a fuzzy controller. By combining a proposed on-line supervised structure-parameter learning technique, the temporal difference prediction method, and the stochastic exploratory algorithm, a reinforcement learning algorithm is proposed, which can construct a RNFCN automatically and dynamically through a reward-penalty signal (i.e., “good” or “bad” signal). Two examples are presented to illustrate the performance and applicability of the proposed models and learning algorithms.     

Distributed Online Frequency Assignment in Cellular Networks

A cellular network is generally modeled as a subgraph of the triangular lattice. The distributed online frequency assignment problem can be abstracted as a multicoloring problem on a weighted graph, where the weight vector associated with the vertices models the number of calls to be served at the vertices and is assumed to change over time. In this paper, we develop a framework for studying distributed online frequency assignment in cellular networks. We present the first distributed online algorithms for this problem with proven bounds on their competitive ratios. We show a series of algorithms that use at each vertex information about increasingly larger neighborhoods of the vertex, and that achieve better competitive ratios. In contrast, we show lower bounds on the competitive ratios of some natural classes of online algorithms.     

Stochastic analysis of a n-unit repairable system with slow switch

This paper deals with the costn–benefit analysis of a cold standby system composed of n identical repairable units, subject to slow switch. Two models of system functioning are studied in this paper. In model 1, the repair time of a unit is assumed to follow exponential distribution and the other time distributions as arbitrary, while in model 2, the repair time of a unit is assumed to be arbitrarily distributed and the other time distributions follow exponential law. For both the models, the system characteristics, namely

(i) the expected upn–time of the system during the period (O,t]

(ii) the expected busyn–period of the repair facility during the period (0,t] and

(iii) the expected time the units spend in the switchover/installation state during the period (O,t]

are studied by identifying the system a t suitable regeneration epochs. The cost-benefit analysis is carried out using these characteristics     

Learning from dependent observations

In most papers establishing consistency for learning algorithms it is assumed that the observations used for training are realizations of an i.i.d. process. In this paper we go far beyond this classical framework by showing that support vector machines (SVMs) only require that the data-generating process satisfies a certain law of large numbers. We then consider the learnability of SVMs for α-mixing (not necessarily stationary) processes for both classification and regression, where for the latter we explicitly allow unbounded noise.     

Learning from uniformly ergodic Markov chains

Evaluation for generalization performance of learning algorithms has been the main thread of machine learning theoretical research. The previous bounds describing the generalization performance of the empirical risk minimization (ERM) algorithm are usually established based on independent and identically distributed (i.i.d.) samples. In this paper we go far beyond this classical framework by establishing the generalization bounds of the ERM algorithm with uniformly ergodic Markov chain (u.e.M.c.) samples. We prove the bounds on the rate of uniform convergence/relative uniform convergence of the ERM algorithm with u.e.M.c. samples, and show that the ERM algorithm with u.e.M.c. samples is consistent. The established theory underlies application of ERM type of learning algorithms.     

The effects of learning in production and group size on the lot-sizing problem

One of the lot-sizing problem extensions that received noticeable attention in the literature is the one that investigated the effects of learning in production. The studies along this line of research assumed learning to improve with the number of repetitions following a power form. There is evidence also that the group size, i.e., the number of workers learning in a group affects performance (time per unit). This note revisits the problem and modifies it by incorporating the group size, along with cumulative production, as a proxy for measuring performance. Numerical examples are provided to illustrate the behavior of the modified model. The results of the two models are also compared to draw some meaningful insights and conclusions. Although the results favor using a simple univariate learning curve, considering group size when modeling lot-sizing problems can significantly affect the unit production cost.     

Rank order data in DEA: A general framework

In data envelopment analysis (DEA), performance evaluation is generally assumed to be based upon a set of quantitative data. In many real world settings, however, it is essential to take into account the presence of qualitative factors when evaluating the performance of decision making units (DMUs). Very often rankings are provided from best to worst relative to particular attributes. Such rank positions might better be presented in an ordinal, rather than numerical sense. The paper develops a general frame work for modeling and treating qualitative data in DEA and provides a unified structure for embedding rank order data into the DEA framework. The existing techniques are discussed and their equivalence is demonstrated. Both continuous and discrete projection models are provided. It is shown that qualitative data can be treated in conventional DEA methodology.     

Models and estimation methods for clinical HIV-1 data

Clinical HIV-1 data include many individual factors, such as compliance to treatment, pharmacokinetics, variability in respect to viral dynamics, race, sex, income, etc., which might directly influence or be associated with clinical outcome. These factors need to be taken into account to achieve a better understanding of clinical outcome and mathematical models can provide a unifying framework to do so. The first objective of this paper is to demonstrate the development of comprehensive HIV-1 dynamics models that describe viral dynamics and also incorporate different factors influencing such dynamics. The second objective of this paper is to describe alternative estimation methods that can be applied to the analysis of data with such models. In particular, we consider: (i) simple but effective two-stage estimation methods, in which data from each patient are analyzed separately and summary statistics derived from the results, (ii) more complex nonlinear mixed effect models, used to pool all the patient data in a single analysis. Bayesian estimation methods are also considered, in particular: (iii) maximum posterior approximations, MAP, and (iv) Markov chain Monte Carlo, MCMC. Bayesian methods incorporate prior knowledge into the models, thus avoiding some of the model simplifications introduced when the data are analyzed using two-stage methods, or a nonlinear mixed effect framework. We demonstrate the development of the models and the different estimation methods using real AIDS clinical trial data involving patients receiving multiple drugs regimens.     

QML-Morven: A novel framework for learning qualitative differential equation models using both symbolic and evolutionary approaches

In this paper, a novel qualitative differential equation model learning (QML) framework named QML-Morven is presented. QML-Morven employs both symbolic and evolutionary approaches as its learning strategies to deal with models of different complexity. Based on this framework, a series of experiments were designed and carried out to: (1) investigate factors that influence the learning precision and minimum data requirement for successful learning; (2) address the scalability issue of QML systems.     

Fast Inference for the Latent Space Network Model Using a Case-Control Approximate Likelihood

Multilevel modeling is a popular statistical technique for analyzing data in hierarchical format, and thus naturally fits within a distributed database framework. We consider the computational aspects of multilevel modeling across distributed databases. In addition, we consider these aspects under a generalization of the multilevel model where the distributed groups (or databases) are allowed to specify different models at both level-1 (individual) and level-2 (group). For a variety of scenarios, we develop the distributed computation algorithm for two-step least squares (LS) estimators and also for iterative MLE estimators of the parameters of interest; in particular, we determine the required data structure at each computing site, the necessary information (original data, cross-product matrices, coefficient vectors), and the order in which such information needs to be passed between sites. Finally, we discuss recursive updating, fault tolerance, and security issues.     

Architecture for development of adaptive on-line prediction models

This work presents an architecture for the development of on-line prediction models. The architecture defines unified modular environment based on three concepts from machine learning, these are: (i) ensemble methods, (ii) local learning, and (iii) meta learning. The three concepts are organised in a three layer hierarchy within the architecture. For the actual prediction making any data-driven predictive method such as artificial neural network, support vector machines, etc. can be implemented and plugged in. In addition to the predictive methods, data pre-processing methods can also be implemented as plug-ins. Models developed according to the architecture can be trained and operated in different modes. With regard to the training, the architecture supports the building of initial models based on a batch of training data, but if this data is not available the models can also be trained in incremental mode. In a scenario where correct target values are (occasionally) available during the run-time, the architecture supports life-long learning by providing several adaptation mechanisms across the three hierarchical levels. In order to demonstrate its practicality, we show how the issues of current soft sensor development and maintenance can be effectively dealt with by using the architecture as a construction plan for the development of adaptive soft sensing algorithms.     

AN ADAPTIVE LEARNING FRAMEWORK FOR FORECASTING SEASONAL WATER ALLOCATIONS IN IRRIGATED CATCHMENTS

Abstract This paper describes an adaptive learning framework for forecasting end‐season water allocations using climate forecasts, historic allocation data, and results of other detailed hydrological models. The adaptive learning framework is based on artificial neural network (ANN) method, which can be trained using past data to predict future water allocations. Using this technique, it was possible to develop forecast models for end‐irrigation‐season water allocations from allocation data available from 1891 to 2005 based on the allocation level at the start of the irrigation season. The model forecasting skill was further improved by the incorporation of a set of correlating clusters of sea surface temperature (SST) and the Southern oscillation index (SOI) data. A key feature of the model is to include a risk factor for the end‐season water allocations based on the start of the season water allocation. The interactive ANN model works in a risk‐management context by providing probability of availability of water for allocation for the prediction month using historic data and/or with the incorporation of SST/SOI information from the previous months. All four developed ANN models (historic data only, SST incorporated, SOI incorporated, SST‐SOI incorporated) demonstrated ANN capability of forecasting end‐of‐season water allocation provided sufficient data on historic allocation are available. SOI incorporated ANN model was the most promising forecasting tool that showed good performance during the field testing of the model.     

Discovering Phase Field Models from Image Data with the Pseudo-Spectral Physics Informed Neural Networks

In this paper,we introduce a new deep learning framework for discovering the phase-field models from existing image data.The new framework embraces the approxim...     

Are ‘misconceptions’ or alternative frameworks of force and motion spontaneous or formed prior to instruction?

It has often been assumed that misconceptions of force and motion are part of an alternative framework and that conceptual change takes place when that framework is challenged and replaced with the Newtonian framework. There have also been variations of this theme, such as this structure is not coherent and conceptual change does not involve the replacement of concepts, conceptions or ideas but consists of the development of scientific ideas that can exist alongside ideas of the everyday. This article argues that misconceptions (or preconceptions, intuitive ideas, synthetic models, p-prims etc.) may not be formed until the learner considers force and motion within the learning situation and reports on a classroom observation (that is replicated with similar results) that suggest misconceptions arise, not because of prior experience, but spontaneously in the attempt at making sense of the terms of the discourse. The implications are that misconceptions may not be preformed, that research ought to consider the possible spontaneity in the students’ reasoning and then, if possible, attempt to discern any preformed elements or antecedents, and that we ought to reconsider what is meant by ‘conceptual change’. The classroom observation also suggests gravity as a particular stumbling-block for students. The implications for further research are discussed.     

Data association based on optimization in graphical models with application to sensor networks

We propose techniques based on graphical models for efficiently solving data association problems arising in multiple target tracking with distributed sensor networks. Graphical models provide a powerful framework for representing the statistical dependencies among a collection of random variables, and are widely used in many applications (e.g., computer vision, error-correcting codes). We consider two different types of data association problems, corresponding to whether or not it is known a priori which targets are within the surveillance range of each sensor. We first demonstrate how to transform these two problems to inference problems on graphical models. With this transformation, both problems can be solved efficiently by local message-passing algorithms for graphical models, which solve optimization problems in a distributed manner by exchange of information among neighboring nodes on the graph. Moreover, a suitably reweighted version of the max–product algorithm yields provably optimal data associations. These approaches scale well with the number of sensors in the network, and moreover are well suited to being realized in a distributed fashion. So as to address trade-offs between performance and communication costs, we propose a communication-sensitive form of message-passing that is capable of achieving near-optimal performance using far less communication. We demonstrate the effectiveness of our approach with experiments on simulated data.     

Inventory Positioning in Multiple Product Supply Chains

In this paper we address the problem of inventory positioning, i.e., the determination of the supply chain node where inventory should be held, to minimize holding costs given a pre-specified order fill rate. A single-echelon inventory system with multiple products models the problem. The value of inventory is assumed to be an increasing function of the amount of processing performed at upstream nodes, while achieved fill-rates are dependent on the distance or time between the inventory storage and customer locations. We propose a novel analytical approach to solve the problem for the case of normally distributed demand that is based on iterative calculations of inventory holding costs at the various potential inventory locations.