A validation study of an instrument designed to measure metacognition of problem solving

Abstract

Metacognition is the ability to monitor, regulate, and control any cognitive enterprise (Flavell, 1993). This construct may be the key to understanding learning differences and learning difficulties experienced by many individuals. It has been noted (Wong, 1994) that learning disabled students have problems applying and utilizing effective strategies in many learning situations and that metacognitive strategies are a key to remediating their problems. There are many instruments available for measuring metacognition. One method utilizes a structured interview. However, the psychometric soundness and the validity of these instruments have not always been evaluated. This study analyzed some properties of one of the instruments utilized in current research. This instrument was the Swanson, Christie and Rubadeau (1993) interview-style questionnaire which was designed to measure metacognition of general problem solving.

A multiple-choice exam designed to assess knowledge of classical problem solving theory and the structured interview used by Swanson, et al. (1993) were used to assess metacognitive ability. Forty-three participants were used in this study: 21 professors and 22 graduate students in the fields of psychology and educational psychology.

Overall, the Swanson instrument has some problems in parts of the instrument, which was demonstrated through the ranking task developed in this study. The experts in the field of problem solving theory did agree on many of the response rankings amongst themselves, however even when their rankings had consistency, few questions matched the responses from the Swanson instrument.

This information adds to the body of knowledge for the development of an improved instrument designed to measure metacognition. This allows for the evaluation of individuals' capabilities in this area and advances the study of metacognitive processes.