An intelligent agent for closed-loop sedation of simulated ICU patients

Abstract

Classical control methods enjoy widespread industrial application due to their simplicity in design and implementation, as well as their success in governing most control tasks. However, some industrial processes may not be handled well by classical methods. Such processes may involve highly-dimensional inputs, significantly nonlinear transfer functions, nonstationary time parameters, and nondeterminism. More robust methods, such as fuzzy logic and nonlinear control may be applied to these tasks; however, these methods are believed to entail complexity in design and implementation.

Recent advances in intelligent control techniques have successfully addressed some of the challenges faced in "real-world" control tasks. Most notably, neural networks and evolutionary computing have been applied to industrial processes with good results. However, intelligent agent control remains an underdeveloped discipline. For example, reinforcement leaming has demonstrated success in select problem domains, but the extent of this technique's ability to control industrial processes remains largely unexplored.

The objective of this study was to evaluate the suitability of reinforcement learning in a "real-world" control task. An intelligent agent was developed to control sedation of simulated intensive care unit patients; training was accomplished using a temporal differencing form of reinforcement leaming. The agent demonstrated an ability to regulate the simulated patient's consciousness level and compared favorably to the conventional PED control method.