Interactive musical visualization based on emotional and color theory

Influenced by synesthesia, the creators of such ?visual musics? as abstract art, color organs, abstract film, and most recently visualizers, have attempted to illustrate correspondences between the senses. This thesis attempts to develop a framework for music visualization founded on emotional analogues between visual art and music. The framework implements audio signal spectrum analysis, mood modeling, and color theory to produce pertinent data for use in visualizations. The research is manifest as a computer program that creates a simple visualizer. Built in Max/MSP/Jitter, a programming environment especially for musical and multimedia processing, it analyzes data and produces images in real-time. The program employs spectrum analysis to extract musical data such as loudness, brightness, and note attacks from the audio signals of AIFF song files. These musical features are used to calculate the Energy and Stress of the song, which determine the general mood of the music. The mood can fall into one of the four general categories of Exuberance, Contentment, Depression, and Anxious/Frantic. This method of automatic mood classification resulted in an eighty-five percent accuracy rate. Applying color expression theory yields a color palette that reflects the musical mood. The color palette and the musical features are then supplied to four different animation schemes to produce visuals. The visualizer generates shapes and forms in a three-dimensional environment and animates them in response to the real-time musical data. The visualizer allows user input to actively direct the creation of a variety of different visualizations. This personalization of the synesthetic effects of the visualizer invites the viewer to actively consider his or her own unique associations and facilitates understanding of the phenomenon of synesthesia and sensory fusion.