The Compositional Construction of Markov Processes

We describe a symmetric monoidal category whose arrows are automata in which the actions have probabilities. The endomorphisms of the identity for the tensor are classical finite Markov processes. The operations of the category permit the compositional description Markov processes. We illustrate by describing a Markov process with 12 ⁿ states, which represents a model of the classical Dining Philosopher problem with n dining philosophers, showing how to calculate the probability of reaching deadlock in k steps. A straightforward application of the Perron-Frobenius Theorem yields that this probability tends to 1 as k tends to infinity.