Episodic Task Planning And Learning In Pervasive Environments

During planning and control of autonomous robots in a pervasive environment designed to serve people, we will inevitably face the situations of needing to perform multiple complex tasks. Management and optimization of the execution of complex tasks involve the design of efficient approach and framework based on algorithm, artificial intelligence, machine learning, cognitive science, etc. In this dissertation, we have developed a new method for complex task planning of robots, so that they can improve the service for the elderly and the disabled. The word "episode" comes from Greek, which means "event", or "occurrence". Humans learn and plan from past episodes by connecting them to the current environment and the task at hand. In cognitive science, episodic memory refers to a human memory subsystem that is concerned with storing and remembering specific sequences and occurrences of events pertaining to a person's ongoing perceptions, experiences, decisions and actions [1]. It helps a human plan the next task. In recent years, researchers have begun to realize the importance of episodic memory to artificial intelligence and cognitive robots, and the episodic like approaches to general event processing.In this dissertation, we propose a computational framework that utilizes the idea of episodic memory to cope with robot planning on complex tasks. Our approach is based on the traditional mathematical model of Markov decision processes, combining the episodic memory approach. In this way, it provides a human-like thinking for autonomous robots, so that they can accomplish complex tasks in pervasive assistive environments, and thus achieve the goal of assisting the everyday living of people. In regard to the traditional hierarchical algorithms for Markov decision processes, although they have been proved to be useful for the problem domains with multiple subtasks due to their strength in task decomposition, they are weak in task abstraction, something that is more important for task analysis and modeling. Using episodic task planning and learning, we propose a task-oriented design approach, which addresses the functionality of task abstraction. Our approach builds an episodic task model from different problem domains, which the robot uses to plan at every step, with more concise structure and much improved performance than the traditional hierarchical model. According to our analysis and experimental evaluation, our approach has shown to have better performance than the existing hierarchical algorithms, such as MAXQ [2] and HEXQ [3].We further introduce a hierarchical multimodal framework for robot planning in multiple-sensor pervasive environments, using multimodal POMDPs. Considering realistic assistive applications may be time-critical and highly related with the risk of planning, we develop a risk-aware approach, allowing robots to possess risk attitudes [4] in their planning. Thus, we have answered the question of how to plan and make sequential decisions efficiently and effectively under complex tasks in pervasive assistive environments, which is very important for the design of applications to assist the living of the elderly and the disabled.