Learning of shared attention in sociable robotics

Sociable robots are embodied agents that are part of a heterogeneous society of robots and humans. They should be able to recognize human beings and each other, and to engage in social interactions. The use of a robotic architecture may strongly reduce the time and effort required to construct a sociable robot. Such architecture must have structures and mechanisms to allow social interaction, behavior control and learning from environment. Learning processes described on Science of Behavior Analysis may lead to the development of promising methods and structures for constructing robots able to behave socially and learn through interactions from the environment by a process of contingency learning. In this paper, we present a robotic architecture inspired from Behavior Analysis. Methods and structures of the proposed architecture, including a hybrid knowledge representation, are presented and discussed. The architecture has been evaluated in the context of a nontrivial real problem: the learning of the shared attention, employing an interactive robotic head. The learning capabilities of this architecture have been analyzed by observing the robot interacting with the human and the environment. The obtained results show that the robotic architecture is able to produce appropriate behavior and to learn from social interaction.     

Robot motion classification from the standpoint of learning control

In robot learning control, the learning space for executing the general motions of multi-joint robot manipulators is very complicated. Thus, when the learning controllers are employed as major roles in motion governing, the motion variety requires them to consume excessive amount of memory. Therefore, in spite of their ability to generalize, the learning controllers are usually used as subordinates to conventional controllers or the learning process needs to be repeated each time a new trajectory is encountered. To simplify learning space complexity, we propose, from the standpoint of learning control, that robot motions be classified according to their similarities. The learning controller can then be designed to govern groups of robot motions with high degrees of similarity without consuming excessive memory resources. Motion classification based on using the PUMA 560 robot manipulator demonstrates the effectiveness of the proposed scheme.     

Reciprocity between the cerebellum and the cerebral cortex: Nonlinear dynamics in microscopic modules for generating voluntary motor commands

The cerebellum and basal ganglia are reciprocally connected with the cerebral cortex, forming many loops that function as distributed processing modules. Here we present a detailed model of one microscopic loop between the motor cortex and the cerebellum, and we show how small arrays of these microscopic loops (CB modules) can be used to generate biologically plausible motor commands for controlling movement. A fundamental feature of CB modules is the presence of positive feedback loops between the cerebellar nucleus and the motor cortex. We use nonlinear dynamics to model one microscopic loop and to investigate its bistable properties. Simulations demonstrate an ability to program a motor command well in advance of command generation and an ability to vary command duration. However, control of command intensity is minimal, which could interfere with the control of movement velocity. To assess these hypotheses, we use a minimal nonlinear model of the neuromuscular (NM) system that translates motor commands into actual movements. Simulations of the combined CB‐NM modular model indicate that movement duration is readily controlled, whereas velocity is poorly controlled. We then explore how an array of eight CB‐NM modules can be used to control the direction and endpoint of a planar movement. In actuality, thousands of such microscopic loops function together as an array of adjustable pattern generators for programming and regulating the composite motor commands that control limb movements. We discuss the biological plausibility and limitations of the model. We also discuss ways in which an agent‐based representation can take advantage of the modularity in order to model this complex system. © 2008 Wiley Periodicals, Inc. Complexity, 2008     

Feasible command strategies for the control of motion of a simple mobile robot

The concept of feasible command strategies is introduced and its applicability is demonstrated by solving a guidance and control problem. This problem concerns the motion of a system which is composed of a rolling disk and a controlled slender rod that is pivoted, through its endpoint, about the disk center. The motion of the disk-rod system is subjected to state and control constraints, and it serves as a model for the motion of a simple mobile robot. In addition, the concept of path controllability is introduced and a condition is derived for the system motion path controllability. The derivation of this condition enables one to design closed-loop control laws for the system motion.     

Large Deviations and Importance Sampling for Systems of Slow-Fast Motion

In this paper we develop the large deviations principle and a rigorous mathematical framework for asymptotically efficient importance sampling schemes for general, fully dependent systems of stochastic differential equations of slow and fast motion with small noise in the slow component. We assume periodicity with respect to the fast component. Depending on the interaction of the fast scale with the smallness of the noise, we get different behavior. We examine how one range of interaction differs from the other one both for the large deviations and for the importance sampling. We use the large deviations results to identify asymptotically optimal importance sampling schemes in each case. Standard Monte Carlo schemes perform poorly in the small noise limit. In the presence of multiscale aspects one faces additional difficulties and straightforward adaptation of importance sampling schemes for standard small noise diffusions will not produce efficient schemes. It turns out that one has to consider the so called cell problem from the homogenization theory for Hamilton-Jacobi-Bellman equations in order to guarantee asymptotic optimality. We use stochastic control arguments.     

Navigation of underwater robot based on dynamically adaptive harmony search algorithm

The current research work has employed an evolutionary based novel navigational strategy to trace the collision free near optimal path for underwater robot in a three-dimensional scenario. The population based harmony search algorithm has been dynamically adapted and used to search next global best pose for underwater robot while obstacle is identified near about robot’s current pose. Each pose is evaluated based on their respective value for objective function which incorporates features of path length minimization as well as obstacle avoidance. Dynamic adaptation of control parameters and new perturbation schemes for solution vectors of harmony search has been proposed to strengthen both exploitation and randomization ability of present search process in a balanced manner. Such adaptive tuning process has found to be more effective for avoiding early convergence during underwater motion in comparison with performances of other popular variants of Harmony Search. The proposed path planning method has also shown better navigational performance in comparison with improved version of ant colony optimization and heuristic potential field method for avoiding static obstacles of different shape and sizes during underwater motion. Simulation studies and corresponding experimental verification for three-dimensional navigation are performed to check the accuracy, robustness and efficiency of proposed dynamically adaptive harmony search algorithm.     

Fast reinforcement learning for simple physical robots

In the past few years, the field of autonomous robot has been rigorously studied and non-industrial applications of robotics are rapidly emerging. One of the most interesting aspects of this field is the development of the learning ability which enables robots to autonomously adapt to given environments without human guidance. As opposed to the conventional methods of robots’ control, where human logically design the behavior of a robot, the ability to acquire action strategies through some learning processes will not only significantly reduce the production costs of robots but also improves the applicability of robots in wider tasks and environments. However, learning algorithms usually require large calculation cost, which make them unsuitable for robots with limited resources. In this study, we propose a simple two-layered neural network that implements a novel and fast Reinforcement Learning. The proposed learning method requires significantly less calculation resources, hence is applicable to small physical robots running in the real world environments. For this study, we built several simple robots and implemented the proposed learning mechanism to them. In the experiments, to evaluate the efficacy of the proposed learning mechanism, several robots were simultaneously trained to acquire obstacle avoidance strategies in the same environment, thus, forming a dynamic environment where the learning task is substantially harder than in the case of learning in a static environment and promising result was obtained.     

Transient RankingMethods for the Simplification of Nonlinear DAE Systems in Analog Circuit Design

Symbolic analysis of analog circuits using computer algebra is limited by the complexity problem: even for small circuits the symbolic equation systems get too large to be handled efficiently. In the past years simplification techniques have been developed which reduce the complexity of such equation systems. In this paper we are focusing on simpli fication techniques for equations modeling nonlinear transient analog circuits. Here it is necessary to control the dynamic behavior of the simplified system. For this, we developed two different methods which predict the influence of a simplification on the equations' transient solution. We will describe both methods and compare their efficiency and accuracy.     

Computed torque control of an under-actuated service robot platform modeled by natural coordinates

The paper investigates the motion planning of a suspended service robot platform equipped with ducted fan actuators. The platform consists of an RRT robot and a cable suspended swinging actuator that form a subsequent parallel kinematic chain and it is equipped with ducted fan actuators. In spite of the complementary ducted fan actuators, the system is under-actuated. The method of computed torques is applied to control the motion of the robot.The under-actuated systems have less control inputs than degrees of freedom. We assume that the investigated under-actuated system has desired outputs of the same number as inputs. In spite of the fact that the inverse dynamical calculation leads to the solution of a system of differential–algebraic equations (DAE), the desired control inputs can be determined uniquely by the method of computed torques.We use natural (Cartesian) coordinates to describe the configuration of the robot, while a set of algebraic equations represents the geometric constraints. In this modeling approach the mathematical model of the dynamical system itself is also a DAE.The paper discusses the inverse dynamics problem of the complex hybrid robotic system. The results include the desired actuator forces as well as the nominal coordinates corresponding to the desired motion of the carried payload. The method of computed torque control with a PD controller is applied to under-actuated systems described by natural coordinates, while the inverse dynamics is solved via the backward Euler discretization of the DAE system for which a general formalism is proposed. The results are compared with the closed form results obtained by simplified models of the system. Numerical simulation and experiments demonstrate the applicability of the presented concepts.     

Decentralized iterative learning control for a class of large scale interconnected dynamical systems

The problem of decentralized iterative learning control for a class of large scale interconnected dynamical systems is considered. In this paper, it is assumed that the considered large scale dynamical systems are linear time-varying, and the interconnections between each subsystem are unknown. For such a class of uncertain large scale interconnected dynamical systems, a method is presented whereby a class of decentralized local iterative learning control schemes is constructed. It is also shown that under some given conditions, the constructed decentralized local iterative learning controllers can guarantee the asymptotic convergence of the local output error between the given desired local output and the actual local output of each subsystem through the iterative learning process. Finally, as a numerical example, the system coupled by two inverted pendulums is given to illustrate the application of the proposed decentralized iterative learning control schemes.     

The Use of Real-World Computer Models for Computer-Aided Design of Tracking Control Schemes for Induction Motors

The design of tracking controllers for induction motors is usually developed by neglecting the presence of power-supply devices, such as inverters, and measurement apparatuses, e.g., encoders. However, these components represent unmodeled dynamics that are present during the real operating conditions of the induction motor. Since the development of a numerical simulation study represents a low-cost, safe, and fast test to validate the design of tracking control schemes, the need arises to build a computer model of the overall system (i.e., motor, power supply, measurement devices, and tracking controller) as realistic as possible. In this context, the paper describes a computer model for simulation of an induction motor under a tracking control scheme including many real-world effects; namely, encoder's quantization, current sensors' noise, stator current dynamics, presence of a current-controlled voltage-source inverter within a stator current regulator loop, flux observer dynamics, saturation of the control signal, and discrete-time implementation of the control algorithm. The developed computer model is finally used in a case study and the simulation results obtained for an induction motor driving a single-link robotic arm under an H8 tracking control scheme are reported.     

AIS Based Distributed Wireless Sensor Network for Mobile Search and Rescue Robot Tracking

This research presents the implementation of GSCF, an AIS-based control framework, on a distributed wireless sensor network for tracking search and rescue robots in open fields. The General Suppression Control Framework (GSCF) is a framework inspired by the suppression hypothesis of the immune discrimination theory. The framework consists of five distinct components; each carries a specific function that can generate long-term and short-term influences to other components by the use of humoral and cellular signals. The goal of the research is to develop mathematical models that can assist the control and analyses of robots behavior through the use of Suppressor Cells in the Suppression Modulator. Acquire data from the physical robot will be used as simulation parameters in future search and rescue research.     

Design an interactive virtual physics environment with uncertainties control to support motion tracking

The development of robot or character motion tracking algorithms is inherently a challenging task. This is more than ever true when the latest trends in motion tracking are considered. Some researchers can deal with kinematic and dynamic constraints induced by the mechanical structure. Another class of researchers fulfills various types of optimality conditions, yet others include means of dealing with uncertainties about the robot or character and its environment. In order to deal with the complexity of developing motion tracking algorithms, it is proposed in this paper to design an interactive virtual physics environment with uncertainties for motion tracking based on sliding mode control. The advantages of doing so are outlined and a virtual environment presented which is well suited to support motion tracking development. The environment makes full use of multi-body system dynamics and a robust sliding mode controller independent of model as simulation kernel. So the environment is capable of simulating setups which fulfill the requirements posed by state-of-the-art motion tracking algorithm development. The demonstration results verified the validity of the environment.     

Heuristic fuzzy-neuro network and its application to reactive navigation of a mobile robot

A novel pattern recognition approach to reactive navigation of a mobile robot is presented in this paper. A heuristic fuzzy-neuro network is developed for pattern-mapping between quantized ultrasonic sensory data and velocity commands to the robot. The design goal was to enable an autonomous mobile robot to navigate safely and efficiently to a target position in a previously unknown environment. Useful heuristic rules were combined with the fuzzy Kohonen clustering network (FKCN) to build the desired mapping between perception and motion. This method provides much faster response to unexpected events and is less sensitive to sensor misreading than conventional approaches. It allows continuous, fast motion of the mobile robot without any need to stop for obstacles. The effectiveness of the proposed method is demonstrated in a series of practical tests on our experimental mobile robot.     

Motion Planning of Robot Locomotion

Since the treatment of robot locomotion is quite closed with this one of human locomotion, the present paper treats the problems of modelling, simulation and motion planning of robot locomotion on the base of knowledge of the skeletal system, as well as on the base of multibody system modelling, simulation and control using some ideas from Lie group theory and differential geometry.     

Bayesian decoding of neural spike trains

Perception, memory, learning, and decision making are processes carried out in the brain. The performance of such intelligent tasks is made possible by the communication of neurons through sequences of voltage pulses called spike trains. It is of great interest to have methods of extracting information from spike trains in order to learn about their relationship to behavior. In this article, we review a Bayesian approach to this problem based on state-space representations of point processes. We discuss some of the theory and we describe the way these methods are used in decoding motor cortical activity, in which the hand motion is reconstructed from neural spike trains.     

Flexible-link robots with combined trajectory tracking and vibration control

This work deals with asymptotic trajectory tracking and active damping injection on a flexible-link robot by application of Multiple Positive Position Feedback. The flexible-link robot is modeled and validated by using finite element methods and experimental modal analysis, and then a reduced order model of the flexible-link robot dynamics, up to the first dominant vibration modes, is employed for experimental evaluation on a test rig. Then, a combined control scheme is synthesized in two parts: first, a Sliding-Mode Control based on a cascaded Proportional-Integral-Derivative for regulation and trajectory tracking tasks, via a direct current motor torque as the control input for the overall system dynamics, and, second, a Multiple Positive Position Feedback for active vibration control and attenuation of residual vibrations on the tip position, via the input voltage applied to a piezoelectric patch actuator attached directly on the flexible beam. The results are evaluated on an experimental platform, where the dynamic performance of the overall active vibration control scheme leads to fast and effective tracking results, with damping ratios increased up to 300%.     

Euler estimates for rough differential equations

We consider controlled ordinary differential equations and give new estimates for higher order Euler schemes. Our proofs are inspired by recent work of A.M. Davie who considers first and second order schemes. In order to implement the general case we make systematic use of geodesic approximations in the free nilpotent group. Such Euler estimates have powerful applications. By a simple limit argument they apply to rough path differential equations (RDEs) in the sense of T. Lyons and hence also to stochastic differential equations driven by Brownian motion or other random rough paths with sufficient integrability. In the context of the latter, we obtain strong remainder estimates in stochastic Taylor expansions a la Azencott, Ben Arous, Castell and Platen. Although our findings appear novel even in the case of driving Brownian motion our main insight is the genuine rough path nature of (quantitative) remainder estimates in stochastic Taylor expansions. There are several other applications of which we discuss in detail Lq-convergence in Lyons' Universal Limit Theorem and moment control of RDE solutions.     

Learning agents for the multi-mode project scheduling problem

Intelligent optimization refers to the promising technique of integrating learning mechanisms into (meta-)heuristic search. In this paper, we use multi-agent reinforcement learning for building high-quality solutions for the multi-mode resource-constrained project scheduling problem (MRCPSP). We use a network of distributed reinforcement learning agents that cooperate to jointly learn a well-performing constructive heuristic. Each agent, being responsible for one activity, uses two simple learning devices, called learning automata, that learn to select a successor activity order and a mode, respectively. By coupling the reward signals for both learning tasks, we can clearly show the advantage of using reinforcement learning in search. We present some comparative results, to show that our method can compete with the best performing algorithms for the MRCPSP, yet using only simple learning schemes without the burden of complex fine-tuning.     

Numerical methods for analytic continuation and mesh generation

The problem of computing the analytic continuation of a holomorphic function known on a circle is considered. Several fast numerical schemes based on solving an initial-value problem for the Cauchy-Riemann equations are analyzed. To avoid instability problems, some of the schemes consist of two parts: one for integrating the Cauchy-Riemann equations, and one for smoothing the function values so obtained. We show that with appropriate integration and smoothing methods, the stability and accuracy of such schemes is sufficient for many applications. The schemes are well-suited for generating level curves and stream lines of conformal mappings. Computed examples are presented. We also indicate how the schemes can be used to generate near-orthogonal boundary-fitted grids with given mesh sizes along the boundary.