Browsing by Subject "Agent"
Now showing 1 - 4 of 4
Results Per Page
Sort Options
Item Development and evaluation of an arterial adaptive traffic signal control system using reinforcement learning(2009-05-15) Xie, YuanchangThis dissertation develops and evaluates a new adaptive traffic signal control system for arterials. This control system is based on reinforcement learning, which is an important research area in distributed artificial intelligence and has been extensively used in many applications including real-time control. In this dissertation, a systematic comparison between the reinforcement learning control methods and existing adaptive traffic control methods is first presented from the theoretical perspective. This comparison shows both the connections between them and the benefits of using reinforcement learning. A Neural-Fuzzy Actor-Critic Reinforcement Learning (NFACRL) method is then introduced for traffic signal control. NFACRL integrates fuzzy logic and neural networks into reinforcement learning and can better handle the curse of dimensionality and generalization problems associated with ordinary reinforcement learning methods. This NFACRL method is first applied to isolated intersection control. Two different implementation schemes are considered. The first scheme uses a fixed phase sequence and variable cycle length, while the second one optimizes phase sequence in real time and is not constrained to the concept of cycle. Both schemes are further extended for arterial control, with each intersection being controlled by one NFACRL controller. Different strategies used for coordinating reinforcement learning controllers are reviewed, and a simple but robust method is adopted for coordinating traffic signals along the arterial. The proposed NFACRL control system is tested at both isolated intersection and arterial levels based on VISSIM simulation. The testing is conducted under different traffic volume scenarios using real-world traffic data collected during morning, noon, and afternoon peak periods. The performance of the NFACRL control system is compared with that of the optimized pre-timed and actuated control. Testing results based on VISSIM simulation show that the proposed NFACRL control has very promising performance. It outperforms optimized pre-timed and actuated control in most cases for both isolated intersection and arterial control. At the end of this dissertation, issues on how to further improve the NFACRL method and implement it in real world are discussed.Item Multi-Agent System for predictive reconfiguration of Shipboard Power Systems(Texas A&M University, 2005-02-17) Srivastava, Sanjeev KumarThe electric power systems in U.S. Navy ships supply energy to sophisticated systems for weapons, communications, navigation and operation. The reliability and survivability of the Shipboard Power System (SPS) are critical to the mission of a surface combatant ship, especially under battle conditions. In the event of battle, various weapons might attack a ship. When a weapon hits the ship it can cause severe damage to the electrical system on the ship. This damage can lead to de-energization of critical loads on a ship that can eventually decrease a ship?s ability to survive the attack. It is very important, therefore, to maintain availability of energy to the connected loads that keep the power systems operational. Technology exists that enables the detection of an incoming weapon and prediction of the geographic area where the incoming weapon will hit the ship. This information can then be used to take reconfiguration actions before the actual hit so that the actual damage caused by the weapon hit is reduced. The Power System Automation Lab (PSAL) has proposed a unique concept called "Predictive Reconfiguration" which refers to performing reconfiguration of a ship?s power system before a weapon hit to reduce the potential damage to the electrical system caused by the impending weapon hit. The concept also includes reconfiguring the electrical system to restore power to as much of the healthy system as possible after the weapon hit. This dissertation presents a new methodology for Predictive Reconfiguration of a Shipboard Power System (SPS). This probabilistic approach includes a method to assess the damage that will be caused by a weapon hit. This method calculates the expected probability of damage for each electrical component on the ship. Also a heuristic method is included, which uses the expected probability of damage to determine reconfiguration steps to reconfigure the ship?s electrical network to reduce the damage caused by a weapon hit. This dissertation also presents a modified approach for performing a reconfiguration for restoration after the weapon hits the system. In this modified approach, an expert system based restoration method restores power to loads de-energized due to the weapon hit. These de-energized loads are restored in a priority order. The methods were implemented using multi-agent technology. A test SPS model based on the electrical layout of a non-nuclear surface combatant ship was presented. Complex scenarios representing electrical casualties caused due to a weapon hit, on the test SPS model, were presented. The results of the Predictive Reconfiguration methodology for complex scenarios were presented to illustrate the effectiveness of the developed methodology.Item Role-based and agent-oriented teamwork modeling(Texas A&M University, 2005-11-01) Cao, SenTeamwork has become increasingly important in many disciplines. To support teamwork in dynamic and complex domains, a teamwork programming language and a teamwork architecture are important for specifying the knowledge of teamwork and for interpreting the knowledge of teamwork and then driving agents to interact with the domains. Psychological studies on teamwork have also shown that team members in an effective team often maintain shared mental models so that they can have mutual expectation on each other. However, existing agent/teamwork programming languages cannot explicitly express the mental states underlying teamwork, and existing representation of the shared mental models are inefficient and further become an obstacle to support effective teamwork. To address these issues, we have developed a teamwork programming language called Role-Based MALLET (RoB-MALLET) which has rich expressivity to explicitly specify the mental states underlying teamwork. By using roles and role variables, the knowledge of team processes is specified in terms of conceptual notions, instead of specific agents and agent variables, allowing joint intentions to be formed and this knowledge to be reused by different teams of agents. Further, based on roles and role variables, we have developed mechanisms of task decomposition and task delegation, by which the knowledge of a team process is decomposed into the knowledge of a team process for individuals and then delegate it to agents. We have also developed an efficient representation of shared mental models called Role-Based Shared Mental Model (RoB-SMM) by which agents only maintain individual processes complementary with others?? individual process and a low level of overlapping called team organizations. Based on RoB-SMMs, we have developed tworeasoning mechanisms to improve team performance, including Role-Based Proactive Information Exchange (RoB-PIE) and Role-Based Proactive Helping Behaivors (RoBPHB). Through RoB-PIE, agents can anticipate other agents?? information needs and proactively exchange information with them. Through RoB-PHB, agents can identify other agents?? help needs and proactively initialize actions to help them. Our experiments have shown that RoB-MALLET is flexible in specifying reusable plans, RoB-SMMs is efficient in supporting effective teamwork, and RoB-PHB improves team performance.Item Sensory invariance driven action (SIDA) framework for understanding the meaning of neural spikes(Texas A&M University, 2004-09-30) Bhamidipati, Sarvani KumarWhat does the spike of a sensory neuron mean? This is a fundamental question in computational neuroscience. Conventional approaches provide an answer based on correlation between spike pattern and the stimulus that caused it. However, these approaches do not satisfactorily explain how the brain, which does not have direct knowledge of the world or the stimuli, can achieve this task. This thesis frames the problem in terms of a task for a simulated agent and provides a solution based on an approach which regards action as necessary for acquiring the meaning of neural spikes. This approach differs from some others in that it proposes a new criterion called the sensory invariance criterion, which can be used to associate meaning to spike patterns in terms of action sequences the agent generates. This criterion forms the basis of the Sensory Invariance Driven Action (SIDA) framework presented in this thesis. This framework is implemented in a reinforcement learning agent and the results indicate that the agent can successfully learn to associate meaning to the sensor activity in terms of specific actions which reflect the properties of the stimulus. Further behavioral experiments on the agent show that this framework allows the agent to learn the meaning of complex (spatiotemporal) spike patterns. The successful learning exhibited by the agent raises hopes that SIDA can be used to build agents with natural semantics.