Towards a psychophysical evaluation of colour constancy algorithms

Computational colour constancy tries to solve the problem of recovering the illuminant of a scene from an acquired image. The most popular algorithms developed to deal with this problem use heuristics to select a unique solution from within the feasible set. Their performance has shown that there is still a long way to go to globally solve this problem as a preliminary step in computer vision. Recent works tried to insert high-level constraints to improve the selection step, whose plausibility could be evaluated according to their performance on the final visual task. To allow comparisons of constraints independently of the task, in this work we present a new performance measure, the perceptual angular error. It tries to evaluate the performance of a colour constancy algorithm according to the perceptual preferences of humans instead of the actual optimal solution. To this end, we present a new version of our "MaxName" algorithm, which aims at solving the illuminant problem using high-level information such as the number of identifiably colours on a scene. Afterwards, we show the results of a psychophysical experiment comparing three colour constancy algorithms. Our results show that in more than half of the judgements the preferred solution is not the one closest to the optimal solution. This makes us conclude that such a perceptual comparison is feasible, and we could benefit from the construction of a large colour constancy database of calibrated images, labelled according to the illuminant preferred by human observers.