STOCHASTIC MODEL OF LAKE SYSTEM INVASION AND ITS OPTIMAL CONTROL: NEURODYNAMIC PROGRAMMING AS A SOLUTION METHOD

Abstract We develop a stochastic model for the process of spread of an aquatic invader in a lake system. The invader is transported along with recreation boats, and treatment of boats allows one to control the invader spread. Optimization of invasion and control costs leads to a dynamic programming problem. However, standard stochastic dynamic programming (SDP) algorithms allow us to solve the problem for at most 13–14 lakes, which is significantly less than the size of systems arising in applications. We have developed a new technique, which is based upon the ideas of reinforcement learning and neurodynamic programming. This approach allows us to obtain approximate but reasonable control policy for essentially greater lake systems. The resulting numerical technique is comparatively easy to implement, and it can be applied to other spatially extended optimal control problems as well.