Browsing by Subject "Sketch Recognition"

Now showing 1 - 5 of 5
  • No Thumbnail Available
    Item
    Acoustic Based Sketch Recognition
    (2012-10-19) Li, Wenzhe
    Sketch recognition is an active research field, with the goal to automatically recognize hand-drawn diagrams by a computer. The technology enables people to freely interact with digital devices like tablet PCs, Wacoms, and multi-touch screens. These devices are easy to use and have become very popular in market. However, they are still quite costly and need more time to be integrated into existing systems. For example, handwriting recognition systems, while gaining in accuracy and capability, still must rely on users using tablet-PCs to sketch on. As computers get smaller, and smart-phones become more common, our vision is to allow people to sketch using normal pencil and paper and to provide a simple microphone, such as one from their smart-phone, to interpret their writings. Since the only device we need is a single simple microphone, the scope of our work is not limited to common mobile devices, but also can be integrated into many other small devices, such as a ring. In this thesis, we thoroughly investigate this new area, which we call acoustic based sketch recognition, and evaluate the possibilities of using it as a new interaction technique. We focus specifically on building a recognition engine for acoustic sketch recognition. We first propose a dynamic time wrapping algorithm for recognizing isolated sketch sounds using MFCC(Mel-Frequency Cesptral Coefficients). After analyzing its performance limitations, we propose improved dynamic time wrapping algorithms which work on a hybrid basis, using both MFCC and four global features including skewness, kurtosis, curviness and peak location. The proposed approaches provide both robustness and decreased computational cost. Finally, we evaluate our algorithms using acoustic data collected by the participants using a device's built-in microphone. Using our improved algorithm we were able to achieve an accuracy of 90% for a 10 digit gesture set, 87% accuracy for the 26 English characters and over 95% accuracy for a set of seven commonly used gestures.
  • No Thumbnail Available
    Item
    Analysis of Children's Sketches to Improve Recognition Accuracy in Sketch-Based Applications
    (2012-12-06) Kim, Hong-Hoe
    The current education systems in elementary schools are usually using traditional teaching methods such as paper and pencil or drawing on the board. The benefit of paper and pencil is their ease of use. Researchers have tried to bring this ease of use to computer-based educational systems through the use of sketch-recognition. Sketch-recognition allows students to draw naturally while at the same time receiving automated assistance and feedback from the computer. There are many sketch-based educational systems for children. However, current sketch-based educational systems use the same sketch recognizer for both adults and children. The problem of this approach is that the recognizers are trained by using sample data drawn by adults, even though the drawing patterns of children and adults are markedly different. We propose that if we make a separate recognizer for children, we can increase the recognition accuracy of shapes drawn by children. By creating a separate recognizer for children, we improved the recognition accuracy of children?s drawings from 81.25% (using the adults? threshold) to 83.75% (using adjusted threshold for children). Additionally, we were able to automatically distinguish children?s drawings from adults? drawings. We correctly identified the drawer?s age (age 3, 4, 7, or adult) with 78.3%. When distinguishing toddlers (age 3 and 4) from matures (age 7 and adult), we got a precision of 95.2% using 10-fold cross validation. When we removed adults and distinguished between toddlers and 7 year olds, we got a precision of 90.2%. Distinguishing between 3, 4, and 7 year olds, we got a precision of 86.8%. Furthermore, we revealed that there is a potential gender difference since our recognizer was more accurately able to recognize the drawings of female children (91.4%) than the male children (85.4%). Finally, this paper introduces a sketch-based teaching assistant tool for children, EasySketch, which teaches children how to draw digits and characters. Children can learn how to draw digits and characters by instructions and feedback.
  • No Thumbnail Available
    Item
    Chinese Calligraphist: A Sketch Based Learning Tool for Learning Written Chinese
    (2014-08-28) Yin, Zhengliang
    Learning Chinese as a foreign language is becoming more and more popular in western countries, however it is also very hard to be proficient, especially in writing. The involvement of the teachers in the process of learning Chinese writing is extremely necessary because they can give timely critiques and feedbacks as well as correct the students? bad writing habits. However, it is inadequate and inefficient of the large class capacity therefore it is urgent and necessary to design a computer-based system to help students in practice Chinese writing, correct their bad writing habits early, and give feedback personally. The current written Chinese learning tools such as online tutorials emphasize writing rules including stroke order, but it could not provide practicing sessions and feedback. Hashigo, a novel CALL (Computer Assisted Language Learning) system, introduced the concept of sketch-based learning, but it?s low level recognizer is not proper for Chinese character domain. Therefore in order to help western students learn Chinese with better understanding, we adopted LADDER description language, machine learning techniques, and sketch recognition algorithms to improve handwritten Chinese stroke recognition rate. With our multilayer perceptron recognizer, it improved Chinese stroke recognition accuracy by 15.7% than the average of the four basic recognizer. In feature selection study we found that the most important features were ?the aspect of the bounding box?, and the ?density metrics?, and ?curviness?. We chose 8 most important features after the recursive selecting stabilized. We discovered that in most situations, feature recognition is more important than template recognition. Since the writing technique is emphasized while they are taught, only 2 templates is enough. It worked as well as 20 templates, which improved recognition speed dramatically. In conclusion, in this thesis our contribution is that we (1) proposed a natural way to describe Chinese characters; (2) implemented a hierarchical Chinese character recognizer combining LADDER with the multilayer perceptron low level recognizer; (3) analyzed the performance of different recognition schemes; (4) designed a sketch-based Chinese writing learning tool, Chinese Calligraphist; and (5) find the best feature combination to recognize Chinese strokes while improving the recognition accuracy.
  • No Thumbnail Available
    Item
    Sketch Recognition on Mobile Devices
    (2012-11-29) Lucchese, George 1987-
    Sketch recognition allows computers to understand and model hand drawn sketches and diagrams. Traditionally sketch recognition systems required a pen based PC interface, but powerful mobile devices such as tablets and smartphones can provide a new platform for sketch recognition systems. We describe a new sketch recognition library, Strontium (SrL) that combines several existing sketch recognition libraries modified to run on both personal computers and on the Android platform. We analyzed the recognition speed and accuracy implications of performing low-level shape recognition on smartphones with touch screens. We found that there is a large gap in recognition speed on mobile devices between recognizing simple shapes and more complex ones, suggesting that mobile sketch interface designers limit the complexity of their sketch domains. We also found that a low sampling rate on mobile devices can affect recognition accuracy of complex and curved shapes. Despite this, we found no evidence to suggest that using a finger as an input implement leads to a decrease in simple shape recognition accuracy. These results show that the same geometric shape recognizers developed for pen applications can be used in mobile applications, provided that developers keep shape domains simple and ensure that input sampling rate is kept as high as possible.
  • No Thumbnail Available
    Item
    TAYouKi: A Sketch-Based Tutoring System for Young Kids
    (2012-10-19) Vides Ceron, Francisco
    Intelligent tutoring systems (ITS) have proven to be effective tools for aiding in the instruction of new skills for young kids; however, interaction methods that employ traditional input devices such as the keyboard and mouse may present barriers to children who have yet learned how to write. Existing applications which utilize pen-input devices better mimic the physical act of writing, but few provide useful feedback to the users. This thesis presents a system specifically designed to serve as a useful tool in teaching children how to draw basic shapes, and helping them develop basic drawing and writing skills. The system uses a combination of sketch recognition techniques to interpret the handwritten strokes from sketches of the children, and then provides intelligent feedback based on what they draw. Our approach provides a virtual coach to assist teachers teaching the critical skills of drawing and handwriting. We do so by guiding children through a set of exercises of increasing complexity according to their progress, and at the same time keeping track of students' performance and engagement, giving them differentiated instruction and feedback. Our system would be like a virtual Teaching Assistant for Young Kids, hence we call it TAYouKi. We collected over five hundred hand-drawn shapes from grownups that had a clear understanding of what a particular geometric shape should look like. We used this data to test the recognition of our system. Following, we conducted a series of case studies with children in age group three to six to test the interactivity efficacy of the system. The studies served to gain important insights regarding the research challenges in different domains. Results suggest that our approach is appealable and engaging to children and can help in more effectively teach them how to draw and write.