Control of ball and beam with neural networks

The ball-and-beam problem is a benchmark for testing new control algorithms. In the Worid Congress On Neural Networks, 1994, Prof Lotfi Zadeh proposed a more difficult version which he claimed required a fuzzy logic controller. This experiment uses a beam, partially covered with a sticky substance, increasing the difficulty of predicting the ball's motion. We complicated the problem even more by not using any information concerning the ball's velocity. Although it is common to use the first differences of the ball's consecutive positions as a measure of velocity and explicit input to the controller, we preferred to exploit recurrent neural networks inputting only consecutive positions instead. We have used truncated backpropagation through time with the Node-Decoupled Extended Kalman Filter (NDEKF) algorithm to update the weights in the networks. The neurocontroller uses a form of approximate dynamic programming called an adaptive critic design. A hierarchy of such designs exists. Our system uses Dual Heuristic Programming (DHP), an upper-level design. To our best knowledge, our results are the first use of DHP to control a physical system. It is also the first system we know of to meet Zadeh's challenge.