Browsing by Subject "Information retrieval"

Now showing 1 - 10 of 10
  • No Thumbnail Available
    Item
    Break down the walls : how the “folder effect” influences the transfer of learning
    (2011-05) He, Jingjie; Svinicki, Marilla D., 1946-; Markman, Arthur
    Categorizing knowledge into different disciplines and units may block knowledge within separate “folders”, which could limit its later retrieval and transfer to new contexts. To test this hypothesis, two experiments had been conducted. In one experiment, participants memorized a list of words with or without cuing which category these words belonged to. One week later, they were asked to recall all the positive adjectives, which required them to retrieve words that came from different categories. In the other experiment, participants read exactly the same story but embedded in two different subject domains or no context. A survey report was presented to test whether people from different contexts would have different transfer effect. The current study replicated previous results that successful transfer was hard to observe in the laboratory settings without explicit prompts. The memory test and transfer task in this study were too difficult and resulted into to the poor performance of the participants. The initial hypothesis had been neither supported nor rejected. To test the hypothesis, future studies could reduce the time interval between study and test, and modified the transfer task to lower the difficulty of the experiment.
  • No Thumbnail Available
    Item
    A collaborative approach to IR evaluation
    (2014-05) Sheshadri, Aashish; Grauman, Kristen Lorraine, 1979-; Lease, Matthew A.
    In this thesis we investigate two main problems: 1) inferring consensus from disparate inputs to improve quality of crowd contributed data; and 2) developing a reliable crowd-aided IR evaluation framework. With regard to the first contribution, while many statistical label aggregation methods have been proposed, little comparative benchmarking has occurred in the community making it difficult to determine the state-of-the-art in consensus or to quantify novelty and progress, leaving modern systems to adopt simple control strategies. To aid the progress of statistical consensus and make state-of-the-art methods accessible, we develop a benchmarking framework in SQUARE, an open source shared task framework including benchmark datasets, defined tasks, standard metrics, and reference implementations with empirical results for several popular methods. Through the development of SQUARE we propose a crowd simulation model that emulates real crowd environments to enable rapid and reliable experimentation of collaborative methods with different crowd contributions. We apply the findings of the benchmark to develop reliable crowd contributed test collections for IR evaluation. As our second contribution, we describe a collaborative model for distributing relevance judging tasks between trusted assessors and crowd judges. Based on prior work's hypothesis of judging disagreements on borderline documents, we train a logistic regression model to predict assessor disagreement, prioritizing judging tasks by expected disagreement. Judgments are generated from different crowd models and intelligently aggregated. Given a priority queue, a judging budget, and a ratio for expert vs. crowd judging costs, critical judging tasks are assigned to trusted assessors with the crowd supplying remaining judgments. Results on two TREC datasets show significant judging burden can be confidently shifted to the crowd, achieving high rank correlation and often at lower cost vs. exclusive use of trusted assessors.
  • No Thumbnail Available
    Item
    Crowdsourcing construction of information retrieval test collections for conversational speech
    (2015-05) Zhou, Haofeng; Lease, Matthew A.; Wallace, Byron
    Building a test collection for an ad hoc information retrieval system on conversational speech raises new challenges for researchers. Traditional methods for building test collections are costly, and thus they are not feasible to apply to large scale conversational speech data. Constructing a large test collection on conversational speech with high quality at low cost is challenging. Crowdsourcing may represent a promising approach. Crowd workers tend to be less expensive than professional assessors, and crowd workers can work simultaneously to perform jobs on a large scale. However, despite the benefits of scale and cost, the quality of the results delivered by crowd workers may suffer. This thesis focuses on relevance judging, one of the key components of a test collection. We adopt two crowdsourcing platforms: oDesk and MTurk, use audio clips and various versions of transcripts, conduct multiple experiments under diverse settings, and analyze the results qualitatively and quantitatively. We delve into what factors influence the quality of relevance judgments on conversational speech. We also investigate differences between relevance judgements from experts and crowd workers. This thesis also describes best practices for the design of crowdsourcing tasks to improve crowd workers' performance. Ultimately, these may assist researchers in building high-quality test collections on conversational speech at low cost and scale through crowdsourcing.
  • No Thumbnail Available
    Item
    The Gander search engine for personalized networked spaces
    (2012-12) Michel, Jonas Reinhardt; Julien, Christine; Garg, Vijay
    The vision of pervasive computing is one of a personalized space populated with vast amounts of data that can be exploited by humans. Such Personalized Networked Spaces (PNetS) and the requisite support for general-purpose expressive spatiotemporal search of the “here” and “now” have eluded realization, due primarily to the complexities of indexing, storing, and retrieving relevant information within a vast collection of highly ephemeral data. This thesis presents the Gander search engine, founded on a novel conceptual model of search in PNetS and targeted for environments characterized by large volumes of highly transient data. We overview this model and provide a realization of it via the architecture and implementation of the Gander search engine. Gander connects formal notions of sampling a search space to expressive, spatiotemporal-aware protocols that perform distributed query processing in situ. This thesis evaluates Gander through a user study that examines the perceived usability and utility of our mobile application, and benchmarks the performance of Gander in large PNetS through network simulation.
  • No Thumbnail Available
    Item
    Knowledge-Rich Event Coreference Resolution
    (2021-05-04) Lu, Jing
    Information extraction, a key area of research in Natural Language Processing (NLP), concerns the extraction of structured information from natural language documents. Recent years have seen a gradual shift of focus from entity-based tasks to event-based tasks in information extraction research. Being a core event-based task, event coreference resolution, the task of determining which event mentions in a document refer to the same real-world event, is generally considered one of the most challenging tasks in NLP. More specifically, for two event mentions to be coreferent, both their triggers (i.e., the words realizing the occurrence of events) and their corresponding arguments (e.g., time, places, and people involved in them) have to be compatible. However, identifying potential arguments (which is typically performed by an entity extraction system), linking arguments to their event mentions (which is typically performed by an event extraction system), and determining the compatibility between two event arguments (which is provided by an entity coreference resolver), are all non-trivial tasks. In other words, end-to-end event coreference resolution is complicated in part by the fact that an event coreference resolver has to rely on the noisy outputs produced by its upstream components in the standard information extraction pipeline. Many existing event coreference resolvers avoid the hassle of dealing with noisy information and simply adopt a knowledge-lean approach consisting of a pipeline of two components, a trigger detection component that identifies triggers and corresponding subtypes, followed by an event coreference component. We hypothesize that knowledge-lean approaches are not the right way to go if the ultimate goal is to take event coreference resolvers to the next level of performance. With this in mind, we investigate knowledge-rich approaches in which we derive potentially useful knowledge for event coreference resolution from a variety of sources, including models that are trained on tasks that we believe are closely related to event coreference, statistical and linguistic features that are directly relevant to the prediction of event coreference links, as well as constraints that encode commonsense knowledge of when two event mentions should or should not be coreferent. We start by designing a multi-pass sieve approach that first resolves easy coreference links and then exploits these easy-to-identify coreference links as a source of knowledge to identify difficult coreference links. We then investigate two types of joint models for event coreference resolution, including a joint inference model and a joint learning model, where we encode commonsense knowledge of the inter-dependencies between the various components via hard or soft constraints. In addition, we incorporate non-local information extracted from the broader context preceding an event mention via learning a supervised topic model and modeling discourse salience. Further, we present an unsupervised method for deriving argument compatibility information from a large, unannotated corpus, and develop a transfer-learning framework that transfers the resulting argument (in)compatibility knowledge to an event coreference resolution resolver. Finally, we investigate a multi-tasking neural model that involves simultaneously learning six tasks related to event coreference, and guide the model learning process using cross-task consistency constraints.
  • No Thumbnail Available
    Item
    Medical Question Answering and Patient Cohort Retrieval
    (2018-05) Goodwin, Travis R.
    With the advent of the electronic health record (EHR), there has been an explosion of rich medical information available for automatic and manual analyses. While the majority of current medical informatics research focuses on easily accessible structured information stored in medical databases, it is widely believed that the majority of information in EHRs remains locked within unstructured text. This dissertation aims to present research that will unlock the knowledge encoded in clinical texts by automatically (1) identifying clinical texts relevant to a specific information need and (2) reasoning about the information encoded in clinical text to answer medical questions posed in natural language. Specifically, we address the tasks of medical question answering -- analyzing the knowledge encoded by EHRs documenting medical practice and experience as well as medical research articles to automatically produce answers to medical questions posed by a physician -- and patient cohort retrieval -- identifying patients who satisfy a given natural language description of specific inclusion and exclusion criteria. Novel systems addressing both of these task are presented and discussed. Moreover, this dissertation presents a number of approaches for overcoming some of the most significant complexities of processing electronic health records. We present new approaches for (1) modeling the temporal aspects of electronic health records -- that is, the fact that the clinical picture of a patient varies throughout his or her medical care -- and show how these approaches can be used to infer, represent, and predict temporal interactions of clinical findings and observations; (2) inferring underspecified information and recovering missing sections of records; and (3) applying machine learning to learn an optimal set of relevance criteria for a specific set of information needs and collection of clinical texts. Combined, this work demonstrates the importance of harnessing the natural language content of electronic health records and highlights the promise of medical question answering and patient cohort retrieval for enabling more informed patient care and improved patient outcomes.
  • No Thumbnail Available
    Item
    PHONETIC MATCHING TOOLKIT WITH STATE-OF-THE-ART META-SOUNDEX ALGORITHM (ENGLISH AND SPANISH)
    (2016-10-27) Koneru, Keerthi; Varol, Cihan; Karpoor, Shashidhar; Zhou, Bing
    Researchers confront major problems while searching for various kinds of data in large imprecise databases, as they are not spelled correctly or in the way they were expected to be spelled. As a result, they cannot find the word they sought. Over the years of struggle, pronunciation of words was considered to be one of the practices to solve the problem effectively. The technique used to acquire words based on sounds is known as “Phonetic Matching”. Soundex was the first algorithm developed and other algorithms like Metaphone, Caverphone, DMetaphone, Phonex etc., are also used for information retrieval in different environments. This project mainly deals with the analysis and implementation of newly proposed Meta-Soundex algorithm for English and Spanish languages which retrieves suggestions for the misspelled words. The newly developed Meta-Soundex algorithm addresses the limitations of Metaphone and Soundex algorithms. Specifically, the new algorithm has more accuracy compared to both Soundex and Metaphone algorithm. The new algorithm also has higher precision compared to Soundex, thus reducing the noise in the considered arena. A phonetic matching toolkit is also developed enclosing the different phoneticmatching algorithms along with the state-of-the-art Meta-Soundex algorithm for both Spanish and English languages.
  • No Thumbnail Available
    Item
    Semantic and Logic Based Routing Algorithms for Service Discovery and Composition in Dynamic IoT-Edge Networks
    (2019-11-22) Moeini, Hessam
    Internet of Things (IoT) interconnects billions of smart sensors, devices, actuators, as well as people, over a distributed environment. Many existing IoT systems are built with predefined tasks and system architecture. To make better use of the large number of “things” available in the IoT infrastructure, we consider to dynamically discover and compose IoT services in the IoT network for new or dynamically arising tasks. To achieve this goal, we develop semantic-based routing protocols for IoT service discovery. We also introduce a distributed planning algorithm in order to efficiently compose distributed IoT services to address dynamically arising tasks. We first consider how to specify the capabilities of the IoT devices and the discovery queries so as to facilitate effective matchmaking. We need to address multi-keyword based queries in the IoT systems. We have studied IoT services and capabilities and noticed that their specification is very different from handling documents. Generally, an IoT device can be specified by a main functionality together with some additional attributes. Thus, we introduce a two-level specification model in which a single keyword is used to specify the main functionality and the Bloom filters are used for specifying the attribute keywords of an IoT service. Next, we consider the memory constraints on IoT and edge devices and focus on the design of routing tables to achieve space-efficiency. We design a capability summarization algorithm to retain the most routing information within the routing table size bound of each IoT node. Our routing table summarization is based on a predefined ontology of IoT capabilities, i.e., the child capabilities can be summarized to a parent capability following the ontology tree. However, this approach will require each node in the IoT network to store the whole ontology tree, which defeats the space reduction goal. Thus, we design an OnBF coding scheme for our two-level approach. For the main functionality specification, we introduce an ontology coding scheme to support ontology-based summarization without needing the ontology tree. The attributes for the IoT service specification are captured in a Bloom filter. Based on OnBF, we design the routing table, the summarization algorithm, etc., to achieve efficient, multi-keyword based service discovery in IoT networks. We also consider how to compose IoT services to achieve the goals of some dynamically arising tasks. When a dynamic task arises, distributed planning is performed over the IoT network to discover the available IoT services and determine how to select and compose them. To ensure planning efficiency, a Logic-based Overlay Network of Services (LONS) is maintained and updated periodically. We then develop a decentralized Graphplan algorithm to traverse through the LONS to derive the desired service composition. Our approach also considers the mutual exclusion relations in the planning graph. Unlike some existing service composition models, our algorithm is comprehensive, fully distributed and efficient in terms of network traffic and the depth of output plan workflow.
  • No Thumbnail Available
    Item
    Supervised language models for temporal resolution of text in absence of explicit temporal cues
    (2013-12) Kumar, Abhimanu; Ghosh, Joydeep
    This thesis explores the temporal analysis of text using the implicit temporal cues present in document. We consider the case when all explicit temporal expressions such as specific dates or years are removed from the text and a bag of words based approach is used for timestamp prediction for the text. A set of gold standard text documents with times- tamps are used as the training set. We also predict time spans for Wikipedia biographies based on their text. We have training texts from 3800 BC to present day. We partition this timeline into equal sized chronons and build a probability histogram for a test document over this chronon sequence. The document is assigned to the chronon with the highest probability. We use 2 approaches: 1) a generative language model with Bayesian priors, and 2) a KL divergence based model. To counter the sparsity in the documents and chronons we use 3 different smoothing techniques across models. We use 3 diverse datasets to test our mod- els: 1) Wikipedia Biographies, 2) Guttenberg Short Stories, and 3) Wikipedia Years dataset. Our models are trained on a subset of Wikipedia biographies. We concentrate on two prediction tasks: 1) time-stamp prediction for a generic text or mid-span prediction for a Wikipedia biography , and 2) life-span prediction for a Wikipedia biography. We achieve an f-score of 81.1% for life-span prediction task and a mean error of around 36 years for mid-span prediction for biographies from present day to 3800 BC. The best model gives a mean error of 18 years for publication date prediction for short stories that are uniformly distributed in the range 1700 AD to 2010 AD. Our models exploit the temporal distribu- tion of text for associating time. Our error analysis reveals interesting properties about the models and datasets used. We try to combine explicit temporal cues extracted from the document with its implicit cues and obtain combined prediction model. We show that a combination of the date-based predictions and language model divergence predictions is highly effective for this task: our best model obtains an f-score of 81.1% and the median error between actual and predicted life span midpoints is 6 years. This would be one of the emphasis for our future work. The above analyses demonstrates that there are strong temporal cues within texts that can be exploited statistically for temporal predictions. We also create good benchmark datasets along the way for the research community to further explore this problem.
  • No Thumbnail Available
    Item
    Trust filter for disease surveillance : Identity
    (2016-12) Lin, Guangyu; Barber, Suzanne
    A flexible and extensible mobile application was delivered for evaluation and optimal inclusion of NextGen (Next Generation) data sources into biosurveillance for early detection, situational awareness and prediction. We present trust analysis of NextGen data sources to increase data confidence. One of the trust filters is the Identity filter, which helps us determine the degree of separation between the sender and the subject of a sentence. In this thesis, the author presents the definition of Identity. To help us distinguish different degrees of separation, the author uses relation distance along with a family tree to weight different relationships. Then the author compares a discriminative algorithm and a generative algorithm to calculate a user's Identity score. In the end, the author concludes that it is a good choice to apply a binary classification algorithm combined with a Natural Language Processing algorithm because of the trade-off between accuracy and runtime complexity.