Image Saliency through Spatial Surprise


Salient (region 2) and non-salient (region 1) regions differ in the surprise they induce in the observers expectations

Overview

The human visual attention system has been for long a subject of research in psychophysics and cognitive sciences, due to its prominent role in biological vision. Significant efforts have also been made in computer vision to construct a computational model of this system, due to the potential for efficient, application-specific and perceptual resource allocation. Attention in this context has been used to achieve critical improvements in applications as diverse as object recognition, video summarization and image quality assessment among others.

Using an information-theoretic approach to study bottom-up spatial saliency, we show how Bayesian surprise can be interpreted to explain spatial saliency. Applications include attention modeling and fixation-prediction, image region detector and image quality assesement.


Fixations, spatial surprise saliency map and ground-truth (mouse-tracking)

Results

Additional image results on:

Fixation location prediction (Dataset 1) (page)
Fixation location prediction (Dataset 2) (page)
Region detection (page)


The fixation in green is the first, and the red lines connect consecutive fixations

Data

Dataset 1 (mouse-tracking data)

Size: 40 images of ValidAttention interface
Source: http://tcts.fpms.ac.be/~mousetrack
Info: M. Mancas. Computational Attention: Towards attentive computers. PhD thesis, Faculty of Engineering, Mons, 2007

Dataset 2 (eye-tracking data)

Samples: 120 images
Source: http://www-sop.inria.fr/members/Neil.Bruce
Info: N. Bruce and J. Tsotsos. Saliency based on information maximization. In Proc. NIPS, volume 18, pages 155-162, 2006

Region Detection

Sample images: http://www.robots.ox.ac.uk/~vgg/research/affine/index.html, The Berkeley Segmentation Dataset
Info: K. Mikolajczyk, et. al. A comparison of affine region detectors. IJCV, 65(1):43-72, 2005.


Image region detection through spatial surprise saliency

People

Ioannis Gkioulekas (SEAS, Harvard Univ., USA) igkiou@gmail
Giorgos Evangelopoulos
Petros Maragos

Publications

- I. Gkioulekas, G. Evangelopoulos, and P. Maragos,
  Spatial Bayesian Surprise for Image Saliency & Quality Assessment,
  Proc. IEEE Int’l Conf. on Image Processing (ICIP-10), Hong-Kong, Sep. 26-29, 2010.