Application of a Near-Optimal Reinforcement Learning Controller to a Robotics Problem in Manufacturing: A Hybrid Approach

Optimization theory provides a framework for determining the best decisions or actions with respect to some mathematical model of a process. This paper focuses on learning to act in a near-optimal manner through reinforcement learning for problems that either have no model or the model is too complex. One approach to solving this class of problems is via approximate dynamic programming. The application of these methods are established primarily for the case of discrete state and action spaces. In this paper we develop efficient methods of learning which act in complex systems with continuous state and action spaces. Monte-Carlo approaches are employed to estimate function values in an iterative, incremental procedure. Derivative-free line search methods are used to obtain a near-optimal action in the continuous action space for a discrete subset of the state space. This near-optimal control policy is then extended to the entire continuous state space via a fuzzy additive model. To compensate for approximation errors, a modified procedure for perturbing the generated control policy is developed. Convergence results under moderate assumptions and stopping criteria are established.     

Bayesian Probabilities and the Histories Algebra

We attempt a justification of a generalisation of the consistent histories programme using a notion of probability that is valid for all complete sets of history propositions. This consists of introducing Cox's axioms of probability theory and showing that our candidate notion of probability obeys them. We also give a generalisation of Bayes' theorem and comment upon how Bayesianism should be useful for the quantum gravity/cosmology programmes. PACS: 02.50.Cw;03.65.Ta;04.60.-m.     

Bayesian Object Identification: Variants

We present a Bayesian theory of object identification. Here, identifying an object means selecting a particular observation from a group of observations (variants), this observation (the regular variant) being characterized by a distributional model. In this sense, object identification means assigning a given model to one of several observations. Often, it is the statistical model of the regular variant, only, that is known. We study an estimator which relies essentially on this model and not on the characteristics of the “irregular” variants. In particular, we investigate under what conditions this variant selector is optimal. It turns out that there is a close relationship with exchangeability and Markovian reversibility. We finally apply our theory to the case of irregular variants generated from the regular variant by a Gaussian linear model.     

Consistency of a myopic Bayesian algorithm for one-dimensional global optimization

A sequential Bayesian method for finding the maximum of a function based on myopically minimizing the expected dispersion of conditional probabilities is described. It is shown by example that an algorithm that generates a dense set of observations need not converge to the correct answer for some priors on continuous functions on the unit interval. For the Brownian motion prior the myopic algorithm is consistent; for any continuous function, the conditional probabilities converge weakly to a point mass at the true maximum.     

最大似然方法和Bayes方法在结构方程模型分析中的讨论(英文)

本文通过模拟研究,讨论了最大似然方法和Bayes方法在分析结构方程模型中的相似点和不同之处。     

Reference priors for exponential families with simple quadratic variance function

Reference analysis is one of the most successful general methods to derive noninformative prior distributions. In practice, however, reference priors are often difficult to obtain. Recently developed theory for conditionally reducible natural exponential families identifies an attractive reparameterization which allows one, among other things, to construct an enriched conjugate prior. In this paper, under the assumption that the variance function is simple quadratic, the order-invariant group reference prior for the above parameter is found. Furthermore, group reference priors for the mean- and natural parameter of the families are obtained. A brief discussion of the frequentist coverage properties is also presented. The theory is illustrated for the multinomial and negative-multinomial family. Posterior computations are especially straightforward due to the fact that the resulting reference distributions belong to the corresponding enriched conjugate family. A substantive application of the theory relates to the construction of reference priors for the Bayesian analysis of two-way contingency tables with respect to two alternative parameterizations.     

A new type SVM-projected SVM

Support vector machine (SVM), developed by Vapnik et al., is a new and promising technique for classification and regression and has been proved to be competitive with the best available learning machines in many applications. However, the classification speed of SVM is substantially slower than that of other techniques with similar generalization ability. A new type SVM named projected SVM (PSVM), which is a combination of feature vector selection (FVS) method and linear SVM (LSVM), is proposed in present paper. In PSVM, the FVS method is first used to select a relevant subset (feature vectors, FVs) from the training data, and then both the training data and the test data are projected into the subspace constructed by FVs, and finally linear SVM(LSVM) is applied to classify the projected data. The time required by PSVM to calculate the class of new samples is proportional to the count of FVs. In most cases, the count of FVs is smaller than that of support vectors (SVs), and therefore PSVM is faster than SVM in running. Compared with other speeding-up techniques of SVM, PSVM is proved to possess not only speeding-up ability but also de-noising ability for high-noised data, and is found to be of potential use in mechanical fault pattern recognition.     

Sum-fuzzy implementation of a choice function using artificial learning procedure with fixed fraction

In one if his paper Luo transformed the problem of sum-fuzzy rationality into artificial learning procedure and gave an algorithm which used the learning rule of perception. This paper extends the Luo method for finding a sum-fuzzy implementation of a choice function and offers an algorithm based on the artificial learning procedure with fixed fraction. We also present a concrete example which uses this algorithm.     

无失效数据的Bayes和多层Bayes分析

本文推广了文献[6]的结果，对指数分布无失效数据的失效率，给出了Bayes估计、Bayes置信上限以及多层Bayes估计，从而可以得到无失效数据可靠度的估计，最后，结合实际问题进行了计算。     

Global Optimal Image Reconstruction from Blurred Noisy Data by a Bayesian Approach

In this paper, a procedure is presented which allows the optimal reconstruction of images from blurred noisy data. The procedure relies on a general Bayesian approach, which makes proper use of all the available information. Special attention is devoted to the informative content of the edges; thus, a preprocessing phase is included, with the aim of estimating the jump sizes in the gray level. The optimization phase follows; existence and uniqueness of the solution is secured. The procedure is tested against simple simulated data and real data.