A new approach to filtering and adaptive control

A simple distribution-free method is proposed for directly estimating and updating a criterion function without recourse to prior state space specification, updated state probabilities, and Bayes' rule. Optimality properties and efficiency advantages of the method are illustrated in terms of a two-armed bandit problem. The relationship between direct criterion function estimation and Kalman-Bucy filtering is clarified.