Pattern detection in natural images

A fundamental visual task is to detect target objects within a background scene. Using relatively simple stimuli, vision science has identified several major factors that affect detection thresholds, such as the luminance of the background, the contrast of the background, the spatial similarity of the background to the target, and uncertainty due to random variations in the properties of the background and in the amplitude of the target. Here I use a new experimental approach together with a theoretical analysis based on signal detection theory, to discover how these factors affect detection in natural scenes. First, I sorted a large collection of natural image backgrounds into multidimensional bins, where each bin corresponds to a narrow range of luminance, contrast and similarity. Detection thresholds were measured by randomly sampling a natural image background from a bin on each trial. In low uncertainty conditions both the bin and the amplitude of the target were blocked and in high uncertainty conditions the bin and amplitude varied randomly on each trial. I found that thresholds increased approximately linearly along all three dimensions and that detection accuracy was unaffected by bin and amplitude uncertainty. The entire set of results was predicted from first principles by a normalized matched template detector, where the dynamic normalizing factor follows directly from the statistical properties of the natural backgrounds. This model assumed that the properties of the background underneath the target were constant across the image, but in natural images this is often not the case. Therefore, in a separate experiment, I measured detection thresholds on backgrounds where the contrast was modulated underneath the target. I found that varying the contrast underneath the target signal had a substantial effect on detectability, and that the pattern of results was predicted by an ideal observer that weighted its response based on an estimate of the local contrast (under the target). This suggests that the human visual system is able to use the varying properties of the background under the target in an near optimal way. Taken together, the results provide a new explanation for some classic laws of psychophysics and their underlying neural mechanisms.