Browsing by Subject "Information retrieval"
Now showing 1 - 7 of 7
Results Per Page
Sort Options
Item Break down the walls : how the “folder effect” influences the transfer of learning(2011-05) He, Jingjie; Svinicki, Marilla D., 1946-; Markman, ArthurCategorizing knowledge into different disciplines and units may block knowledge within separate “folders”, which could limit its later retrieval and transfer to new contexts. To test this hypothesis, two experiments had been conducted. In one experiment, participants memorized a list of words with or without cuing which category these words belonged to. One week later, they were asked to recall all the positive adjectives, which required them to retrieve words that came from different categories. In the other experiment, participants read exactly the same story but embedded in two different subject domains or no context. A survey report was presented to test whether people from different contexts would have different transfer effect. The current study replicated previous results that successful transfer was hard to observe in the laboratory settings without explicit prompts. The memory test and transfer task in this study were too difficult and resulted into to the poor performance of the participants. The initial hypothesis had been neither supported nor rejected. To test the hypothesis, future studies could reduce the time interval between study and test, and modified the transfer task to lower the difficulty of the experiment.Item A collaborative approach to IR evaluation(2014-05) Sheshadri, Aashish; Grauman, Kristen Lorraine, 1979-; Lease, Matthew A.In this thesis we investigate two main problems: 1) inferring consensus from disparate inputs to improve quality of crowd contributed data; and 2) developing a reliable crowd-aided IR evaluation framework. With regard to the first contribution, while many statistical label aggregation methods have been proposed, little comparative benchmarking has occurred in the community making it difficult to determine the state-of-the-art in consensus or to quantify novelty and progress, leaving modern systems to adopt simple control strategies. To aid the progress of statistical consensus and make state-of-the-art methods accessible, we develop a benchmarking framework in SQUARE, an open source shared task framework including benchmark datasets, defined tasks, standard metrics, and reference implementations with empirical results for several popular methods. Through the development of SQUARE we propose a crowd simulation model that emulates real crowd environments to enable rapid and reliable experimentation of collaborative methods with different crowd contributions. We apply the findings of the benchmark to develop reliable crowd contributed test collections for IR evaluation. As our second contribution, we describe a collaborative model for distributing relevance judging tasks between trusted assessors and crowd judges. Based on prior work's hypothesis of judging disagreements on borderline documents, we train a logistic regression model to predict assessor disagreement, prioritizing judging tasks by expected disagreement. Judgments are generated from different crowd models and intelligently aggregated. Given a priority queue, a judging budget, and a ratio for expert vs. crowd judging costs, critical judging tasks are assigned to trusted assessors with the crowd supplying remaining judgments. Results on two TREC datasets show significant judging burden can be confidently shifted to the crowd, achieving high rank correlation and often at lower cost vs. exclusive use of trusted assessors.Item Crowdsourcing construction of information retrieval test collections for conversational speech(2015-05) Zhou, Haofeng; Lease, Matthew A.; Wallace, ByronBuilding a test collection for an ad hoc information retrieval system on conversational speech raises new challenges for researchers. Traditional methods for building test collections are costly, and thus they are not feasible to apply to large scale conversational speech data. Constructing a large test collection on conversational speech with high quality at low cost is challenging. Crowdsourcing may represent a promising approach. Crowd workers tend to be less expensive than professional assessors, and crowd workers can work simultaneously to perform jobs on a large scale. However, despite the benefits of scale and cost, the quality of the results delivered by crowd workers may suffer. This thesis focuses on relevance judging, one of the key components of a test collection. We adopt two crowdsourcing platforms: oDesk and MTurk, use audio clips and various versions of transcripts, conduct multiple experiments under diverse settings, and analyze the results qualitatively and quantitatively. We delve into what factors influence the quality of relevance judgments on conversational speech. We also investigate differences between relevance judgements from experts and crowd workers. This thesis also describes best practices for the design of crowdsourcing tasks to improve crowd workers' performance. Ultimately, these may assist researchers in building high-quality test collections on conversational speech at low cost and scale through crowdsourcing.Item The Gander search engine for personalized networked spaces(2012-12) Michel, Jonas Reinhardt; Julien, Christine; Garg, VijayThe vision of pervasive computing is one of a personalized space populated with vast amounts of data that can be exploited by humans. Such Personalized Networked Spaces (PNetS) and the requisite support for general-purpose expressive spatiotemporal search of the “here” and “now” have eluded realization, due primarily to the complexities of indexing, storing, and retrieving relevant information within a vast collection of highly ephemeral data. This thesis presents the Gander search engine, founded on a novel conceptual model of search in PNetS and targeted for environments characterized by large volumes of highly transient data. We overview this model and provide a realization of it via the architecture and implementation of the Gander search engine. Gander connects formal notions of sampling a search space to expressive, spatiotemporal-aware protocols that perform distributed query processing in situ. This thesis evaluates Gander through a user study that examines the perceived usability and utility of our mobile application, and benchmarks the performance of Gander in large PNetS through network simulation.Item PHONETIC MATCHING TOOLKIT WITH STATE-OF-THE-ART META-SOUNDEX ALGORITHM (ENGLISH AND SPANISH)(2016-10-27) Koneru, Keerthi; Varol, Cihan; Karpoor, Shashidhar; Zhou, BingResearchers confront major problems while searching for various kinds of data in large imprecise databases, as they are not spelled correctly or in the way they were expected to be spelled. As a result, they cannot find the word they sought. Over the years of struggle, pronunciation of words was considered to be one of the practices to solve the problem effectively. The technique used to acquire words based on sounds is known as “Phonetic Matching”. Soundex was the first algorithm developed and other algorithms like Metaphone, Caverphone, DMetaphone, Phonex etc., are also used for information retrieval in different environments. This project mainly deals with the analysis and implementation of newly proposed Meta-Soundex algorithm for English and Spanish languages which retrieves suggestions for the misspelled words. The newly developed Meta-Soundex algorithm addresses the limitations of Metaphone and Soundex algorithms. Specifically, the new algorithm has more accuracy compared to both Soundex and Metaphone algorithm. The new algorithm also has higher precision compared to Soundex, thus reducing the noise in the considered arena. A phonetic matching toolkit is also developed enclosing the different phoneticmatching algorithms along with the state-of-the-art Meta-Soundex algorithm for both Spanish and English languages.Item Supervised language models for temporal resolution of text in absence of explicit temporal cues(2013-12) Kumar, Abhimanu; Ghosh, JoydeepThis thesis explores the temporal analysis of text using the implicit temporal cues present in document. We consider the case when all explicit temporal expressions such as specific dates or years are removed from the text and a bag of words based approach is used for timestamp prediction for the text. A set of gold standard text documents with times- tamps are used as the training set. We also predict time spans for Wikipedia biographies based on their text. We have training texts from 3800 BC to present day. We partition this timeline into equal sized chronons and build a probability histogram for a test document over this chronon sequence. The document is assigned to the chronon with the highest probability. We use 2 approaches: 1) a generative language model with Bayesian priors, and 2) a KL divergence based model. To counter the sparsity in the documents and chronons we use 3 different smoothing techniques across models. We use 3 diverse datasets to test our mod- els: 1) Wikipedia Biographies, 2) Guttenberg Short Stories, and 3) Wikipedia Years dataset. Our models are trained on a subset of Wikipedia biographies. We concentrate on two prediction tasks: 1) time-stamp prediction for a generic text or mid-span prediction for a Wikipedia biography , and 2) life-span prediction for a Wikipedia biography. We achieve an f-score of 81.1% for life-span prediction task and a mean error of around 36 years for mid-span prediction for biographies from present day to 3800 BC. The best model gives a mean error of 18 years for publication date prediction for short stories that are uniformly distributed in the range 1700 AD to 2010 AD. Our models exploit the temporal distribu- tion of text for associating time. Our error analysis reveals interesting properties about the models and datasets used. We try to combine explicit temporal cues extracted from the document with its implicit cues and obtain combined prediction model. We show that a combination of the date-based predictions and language model divergence predictions is highly effective for this task: our best model obtains an f-score of 81.1% and the median error between actual and predicted life span midpoints is 6 years. This would be one of the emphasis for our future work. The above analyses demonstrates that there are strong temporal cues within texts that can be exploited statistically for temporal predictions. We also create good benchmark datasets along the way for the research community to further explore this problem.Item Trust filter for disease surveillance : Identity(2016-12) Lin, Guangyu; Barber, SuzanneA flexible and extensible mobile application was delivered for evaluation and optimal inclusion of NextGen (Next Generation) data sources into biosurveillance for early detection, situational awareness and prediction. We present trust analysis of NextGen data sources to increase data confidence. One of the trust filters is the Identity filter, which helps us determine the degree of separation between the sender and the subject of a sentence. In this thesis, the author presents the definition of Identity. To help us distinguish different degrees of separation, the author uses relation distance along with a family tree to weight different relationships. Then the author compares a discriminative algorithm and a generative algorithm to calculate a user's Identity score. In the end, the author concludes that it is a good choice to apply a binary classification algorithm combined with a Natural Language Processing algorithm because of the trade-off between accuracy and runtime complexity.