Freehand Sketch Recognition for Computer-Assisted Language Learning of Written East Asian Languages

One of the challenges students face in studying an East Asian (EA) language (e.g., Chinese, Japanese, and Korean) as a second language is mastering their selected language?s written component. This is especially true for students with native fluency of English and deficient written fluency of another EA language. In order to alleviate the steep learning curve inherent in the properties of EA languages? complicated writing scripts, language instructors conventionally introduce various written techniques such as stroke order and direction to allow students to study writing scripts in a systematic fashion. Yet, despite the advantages gained from written technique instruction, the physical presence of the language instructor in conventional instruction is still highly desirable during the learning process; not only does it allow instructors to offer valuable real-time critique and feedback interaction on students? writings, but it also allows instructors to correct students? bad writing habits that would impede mastery of the written language if not caught early in the learning process. The current generation of computer-assisted language learning (CALL) applications specific to written EA languages have therefore strived to incorporate writing-capable modalities in order to allow students to emulate their studies outside the classroom setting. Several factors such as constrained writing styles, and weak feedback and assessment capabilities limit these existing applications and their employed techniques from closely mimicking the benefits that language instructors continue to offer. In this thesis, I describe my geometric-based sketch recognition approach to several writing scripts in the EA languages while addressing the issues that plague existing CALL applications and the handwriting recognition techniques that they utilize. The approach takes advantage of A Language to Describe, Display, and Editing in Sketch Recognition (LADDER) framework to provide users with valuable feedback and assessment that not only recognizes the visual correctness of students? written EA Language writings, but also critiques the technical correctness of their stroke order and direction. Furthermore, my approach provides recognition independent of writing style that allows students to learn with natural writing through size- and amount-independence, thus bridging the gap between beginner applications that only recognize single-square input and expert tools that lack written technique critique.