MEDSS: A Multi-Agent Environmental Decision Support System framework for model management

Model Management Systems (MMS) have become increasingly important in handling complicated problems in Decision Support Systems (DSS). The primary goal of MMS is to facilitate the development and the utilization of quantitative models to improve decision performance. Much current research focuses on model construction. Where early research used deductive reasoning approaches to construct new models, more recent efforts use inductive reasoning mechanisms. Both approaches have their drawbacks. Deductive reasoning methods require a strong domain theory (which may not exist or may be too complex to apply) and ignore previous solving experience. Inductive reasoning methods can take advantage of precedents or prototypical cases, but do not employ domain knowledge. Both methods are limited in learning capacity. This study proposes a Multi-Agent Environmental Decision Support System, which integrates an Inductive Reasoning Agent, and an Environmental Learning Agent to perform new model formation and problem solving. New models can be generated by the coordination of both the Inductive Agent and the Deductive Agent. At the same time, a model repair process is undertaken by the Environmental Learning Agent when the prediction resulting from existing knowledge fails.     

Accelerating autonomous learning by using heuristic selection of actions

This paper investigates how to make improved action selection for online policy learning in robotic scenarios using reinforcement learning (RL) algorithms. Since finding control policies using any RL algorithm can be very time consuming, we propose to combine RL algorithms with heuristic functions for selecting promising actions during the learning process. With this aim, we investigate the use of heuristics for increasing the rate of convergence of RL algorithms and contribute with a new learning algorithm, Heuristically Accelerated Q-learning (HAQL), which incorporates heuristics for action selection to the Q-Learning algorithm. Experimental results on robot navigation show that the use of even very simple heuristic functions results in significant performance enhancement of the learning rate.     

Reinforcement learning versus heuristics for order acceptance on a single resource

Order Acceptance (OA) is one of the main functions in business control. Accepting an order when capacity is available could disable the system to accept more profitable orders in the future with opportunity losses as a consequence. Uncertain information is also an important issue here. We use Markov decision models and learning methods from Artificial Intelligence to find decision policies under uncertainty. Reinforcement Learning (RL) is quite a new approach in OA. It is shown here that RL works well compared with heuristics. It is demonstrated that employing an RL trained agent is a robust, flexible approach that in addition can be used to support the detection of good heuristics.     

Learning objects and objectives towards automatic learning construction

This paper presents a method that creates instructionally sound learning experiences by means of learning objects. The method uses a mathematical model, distinguishes two kinds of Learning Objects Properties and proceeds in two major steps: first, the Course Creation is transformed into Set Covering under specific requirements derived from Learning Theories and practice; second, the Alternative Learning Sources are selected by using a similarity measure specially defined for this purpose.     

Basis Function Adaptation in Temporal Difference Reinforcement Learning

Reinforcement Learning (RL) is an approach for solving complex multi-stage decision problems that fall under the general framework of Markov Decision Problems (MDPs), with possibly unknown parameters. Function approximation is essential for problems with a large state space, as it facilitates compact representation and enables generalization. Linear approximation architectures (where the adjustable parameters are the weights of pre-fixed basis functions) have recently gained prominence due to efficient algorithms and convergence guarantees. Nonetheless, an appropriate choice of basis function is important for the success of the algorithm. In the present paper we examine methods for adapting the basis function during the learning process in the context of evaluating the value function under a fixed control policy. Using the Bellman approximation error as an optimization criterion, we optimize the weights of the basis function while simultaneously adapting the (non-linear) basis function parameters. We present two algorithms for this problem. The first uses a gradient-based approach and the second applies the Cross Entropy method. The performance of the proposed algorithms is evaluated and compared in simulations. This research was partially supported by the Fund for Promotion of Research at the Technion. The work of S.M. was partially supported by the National Science Foundation under grant ECS-0312921.     

Learning dynamic prices in electronic retail markets with customer segmentation

In this paper, we use reinforcement learning (RL) techniques to determine dynamic prices in an electronic monopolistic retail market. The market that we consider consists of two natural segments of customers, captives and shoppers. Captives are mature, loyal buyers whereas the shoppers are more price sensitive and are attracted by sales promotions and volume discounts. The seller is the learning agent in the system and uses RL to learn from the environment. Under (reasonable) assumptions about the arrival process of customers, inventory replenishment policy, and replenishment lead time distribution, the system becomes a Markov decision process thus enabling the use of a wide spectrum of learning algorithms. In this paper, we use the Q-learning algorithm for RL to arrive at optimal dynamic prices that optimize the seller’s performance metric (either long term discounted profit or long run average profit per unit time). Our model and methodology can also be used to compute optimal reorder quantity and optimal reorder point for the inventory policy followed by the seller and to compute the optimal volume discounts to be offered to the shoppers.     

The role of local partial independence in learning of Bayesian networks

Bayesian networks are one of the most widely used tools for modeling multivariate systems. It has been demonstrated that more expressive models, which can capture additional structure in each conditional probability table (CPT), may enjoy improved predictive performance over traditional Bayesian networks despite having fewer parameters. Here we investigate this phenomenon for models of various degree of expressiveness on both extensive synthetic and real data. To characterize the regularities within CPTs in terms of independence relations, we introduce the notion of partial conditional independence (PCI) as a generalization of the well-known concept of context-specific independence (CSI). To model the structure of the CPTs, we use different graph-based representations which are convenient from a learning perspective. In addition to the previously studied decision trees and graphs, we introduce the concept of PCI-trees as a natural extension of the CSI-based trees. To identify plausible models we use the Bayesian score in combination with a greedy search algorithm. A comparison against ordinary Bayesian networks shows that models with local structures in general enjoy parametric sparsity and improved out-of-sample predictive performance, however, often it is necessary to regulate the model fit with an appropriate model structure prior to avoid overfitting in the learning process. The tree structures, in particular, lead to high quality models and suggest considerable potential for further exploration.     

基于随机时滞模型的学习成绩波动规律

依据学习机制和教育心理学、认知心理学等有关研究成果,运用动力系统、随机过程等理论,构建描述学习成绩变化的随机时滞模型,并通过模型动力学性态的研究来揭示学习成绩波动特性,预测学习成绩对学习系统中内外因素的反应,分析造成学习成绩波动的原因,探讨相应的教学干预策略.结果表明,学习成绩波动是由4种动力学模式主导;学习成绩对学习系统内外因素具有高度的敏感性.这将从理论上加深对认知涌现机制的认识,并对于拓展和改进教学思维有积极意义.     

Evolving fuzzy rule based controllers using genetic algorithms

The synthesis of genetics-based machine learning and fuzzy logic is beginning to show promise as a potent tool in solving complex control problems in multi-variate non-linear systems. In this paper an overview of current research applying the genetic algorithm to fuzzy rule based control is presented. A novel approach to genetics-based machine learning of fuzzy controllers, called a Pittsburgh Fuzzy Classifier System # 1 (P-FCS1) is proposed. P-FCS1 is based on the Pittsburgh model of learning classifier systems and employs variable length rule-sets and simultaneously evolves fuzzy set membership functions and relations. A new crossover operator which respects the functional linkage between fuzzy rules with overlapping input fuzzy set membership functions is introduced. Experimental results using P-FCS 1 are reported and compared with other published results. Application of P-FCS1 to a distributed control problem (dynamic routing in computer networks) is also described and experimental results are presented.     

Improved Generalization via Tolerant Training

Theoretical and computational justification is given for improved generalization when the training set is learned with less accuracy. The model used for this investigation is a simple linear one. It is shown that learning a training set with a tolerance improves generalization, over zero-tolerance training, for any testing set satisfying a certain closeness condition to the training set. These results, obtained via a mathematical programming formulation, are placed in the context of some well-known machine learning results. Computational confirmation of improved generalization is given for linear systems (including nine of the twelve real-world data sets tested), as well as for nonlinear systems such as neural networks for which no theoretical results are available at present. In particular, the tolerant training method improves generalization on noisy, sparse, and overparameterized problems.     

生成学习系统的混沌与设计

认知学习是一种高度复杂的非线性现象.试图依据生成学习理论和经验学习的思想,建构含有记忆效应的生成学习系统动力学模型,探讨认知学习过程的复杂现象和变化特征,揭示学习系统波动的内生机制和学生认知的混沌规律,并在此基础上提出基于学习混沌的教学系统设计模式,期望能促进学生认知结构的发展.     

Iterative sliding mode control

Iterative Learning Control (ILC) methods are described and applied ever-increasingly as powerful tools to control dynamics nowadays.

ILC’s methods in most studies are described as based on repetitive process from the beginning to the end of process or as a kind of repetitive control.

Our newly designed controllers based on a particular case of iterative learning control radically differ from conventional methods in attempting to stabilize a class of non linear systems.

In this paper two kinds of ILC method are introduced in two separate sections. In the first, our newly designed method satisfies the condition of a Lyapunov stability theorem in a class of non linear systems in which their structures have the Lipschitz property. In the second, by freezing the time and moving to a new virtual axis, called the index axis, this newly designed method tries to find the best value for control at this time step and can be used in two modes, on-line and off-line.

In both methods, by satisfying the convergence condition of our designed ILC, closed loop stability is obtained automatically.     

Using video representations of teaching in practice-based professional development programs

This article explores how video can be used in practice-based professional development (PD) programs to serve as a focal point for teachers’ collaborative exploration of the central activities of teaching. We argue that by choosing video clips, posing substantive questions, and facilitating productive conversations, professional developers can guide teachers to examine central aspects of learning and instruction. We draw primarily from our experiences developing and studying two mathematics PD programs, the Problem-Solving Cycle (PSC) and Learning and Teaching Geometry (LTG). While both programs feature classroom video in a central role, they illustrate different approaches to practice-based PD. The PSC, an adaptive model of PD, provides a framework within which facilitators tailor activities to suit their local context. By contrast, LTG is a highly specified model of PD, which details in advance particular learning goals, design characteristics, and extensive support materials for facilitators. We propose a continuum of video use in PD from highly adaptive to highly specified and consider the affordances and constraints of different approaches exemplified by the PSC and LTG programs.     

多因素分析的线性结构模型及其应用

线性结构模型是含有潜在因子的通径分析,该模型可以分析外生的潜在因子与内生的潜在因子之间相互影响的结构关系.研究应用线性结构模型探讨对《算法与数据结构》的学习兴趣有影响的主要因素,结果显示学生的专业基础和教学方式对学习兴趣的影响分别占各因素影响总和的49.56%和43.11%.注重专业基础课程的教学效果和采取通俗易懂的讲授方式是提高学生学习兴趣的有效途径.     

Learning the Structure of Mixed Graphical Models

We consider the problem of learning the structure of a pairwise graphical model over continuous and discrete variables. We present a new pairwise model for graphical models with both continuous and discrete variables that is amenable to structure learning. In previous work, authors have considered structure learning of Gaussian graphical models and structure learning of discrete models. Our approach is a natural generalization of these two lines of work to the mixed case. The penalization scheme involves a novel symmetric use of the group-lasso norm and follows naturally from a particular parameterization of the model. Supplementary materials for this article are available online.     

Algebraic results and bottom-up algorithm for policies generalization in reinforcement learning using concept lattices

The generalization of policies in reinforcement learning is a main issue, both from the theoretical model point of view and for their applicability. However, generalizing from a set of examples or searching for regularities is a problem which has already been intensively studied in machine learning. Thus, existing domains such as Inductive Logic Programming have already been linked with reinforcement learning. Our work uses techniques in which generalizations are constrained by a language bias, in order to regroup similar states. Such generalizations are principally based on the properties of concept lattices. To guide the possible groupings of similar states of the environment, we propose a general algebraic framework, considering the generalization of policies through a partition of the set of states and using a language bias as an a priori knowledge. We give a practical application as an example of our theoretical approach by proposing and experimenting a bottom-up algorithm.     

Optimal incremental learning under covariate shift

Learning strategies under covariate shift have recently been widely discussed. The density of learning inputs under covariate shift is different from that of test inputs. Learning machines in such environments need to employ special learning strategies to acquire greater capabilities of generalizing through learning. However, incremental learning methods are also used for learning in non-stationary learning environments, which represent a kind of covariate shift. However, the relation between covariate-shift environments and incremental-learning environments has not been adequately discussed. This paper focuses on the covariate shift in incremental-learning environments and our re-construction of a suitable incremental-learning method. Then, the model-selection criterion is also derived, which is to be an essential object function for memetic algorithms to solve these kinds of learning problems.     

“Your truth isn’t the Truth”: Data activities and informal inferential reasoning

This paper investigates data activities in an afterschool setting, offering a deeper understanding of the social nature of students’ informal inferences by investigating how informal inferences are negotiated in group interactions, influenced by social norms, and how statistical concepts come into play in learners’ informal inferential reasoning (IIR). Analyses take up a multi-sited orientation to investigate how youth used quantitative and contextual resources during a research activity to make meaning of data and negotiate emergent social tensions. Findings show how data activities that are part of informal inferential reasoning, such as collection, interpretation, generalization, inference, and representation unfolded as social, political, and personal. Implications call for designs for learning that better support working with data and understanding real-world phenomena and sociopolitical issues in ways that leverage youths’ experiences, enabling them to take part in social action as critical community actors.     

关于G-代数的一些注记

引进了基于一般剩余格的G-代数-G(RL)-代数的概念,并且分别给出了G-滤子和(全序)G(RL)-代数的一系列特征刻画,同时还证明了任何正则的G(RL)-代数必为Boolean代数,本文所得结果分别是已有结果的一般化.