User Tools

Site Tools


resources:pdf_document_resources:topic_face_color

Summary

  • Color information can improve face perception
  • Several cortical regions contain single neurons whose responses are influenced by color and shape simultaneously
  • Cortical regions supporting color and face perception abilities are close
  • ADC found literature supporting behavioral improvements in two domains: face detection and gender recognition
  • Color may aid emotion recognition, but ADC only found computer vision literature supporting this

The references listed below are ones that support a role for color information in aiding face perception. The importance of the role depends strongly on the perceptual task in question, however. There is a long literature showing that ventral occipitotemporal cortical regions from V1 onwards contain single neurons that respond to specific combinations of shape and color features; only two references are included here. In as much as there exists a “color region” of cortex, it is located just posterior and medial to “face regions.” Color aids face detection, especially in naturalistic backgrounds, when it is done by both human observers and computer vision algorithms. Color appears to aid gender discrimination, although the specific color/gender associations seem up in the air. Not much research has been done on the role of color in recognizing facial expressions of emotion, but there is at least one computer vision article that found a use for color in that domain.

FIRST OF ALL

Refer to this very comprehensive list of computer vision articles:

http://www.visionbib.com/bibliography/people902.html#Finding%20Faces%20by%20Color%20Features

For a list of computer vision face image data sets, navigate here and skip to Dataset, Faces. Note that this is not a complete list.

http://datasets.visionbib.com/index.html

See also VNLab's list of face image data sets available online:

Face data sets

Close relationship between cortical regions for color and face perception

  • Clark, V. P., Parasuraman, R., Keil, K., Kulansky, R., Fannon, S., Maisog, J. M., … Haxby, J. V. (1997). Selective attention to face identity and color studied with f MRI. Human Brain Mapping, 5(4), 293–297. doi:10.1002/(SICI)1097-0193(1997)5:4<293::AID-HBM15>3.0.CO;2-F
  • Tanaka, K., Saito, H., Fukada, Y., & Moriya, M. (1991). Coding visual images of objects in the inferotemporal cortex of the macaque monkey. Journal of Neurophysiology, 66(1), 170–189.

Color aids face detection

Behavioral

  • Yip, Andrew W., and Pawan Sinha. “Contribution of Color to Face Recognition.” Perception 31, no. 8 (2002): 995–1003. doi:10.1068/p3376.

NOTE: at least one article hints that if individuals can be distinguished based solely on color information, then ordinary, holistic face recognition processes might not be used:

  • McKone, Elinor, and Galit Yovel. “Why Does Picture-plane Inversion Sometimes Dissociate Perception of Features and Spacing in Faces, and Sometimes Not? Toward a New Theory of Holistic Processing.” Psychonomic Bulletin & Review 16, no. 5 (October 2009): 778–97. doi:10.3758/PBR.16.5.778.

Computational

  • Maglogiannis, Ilias, Demosthenes Vouyioukas, and Chris Aggelopoulos. “Face Detection and Recognition of Natural Human Emotion Using Markov Random Fields.” Personal and Ubiquitous Computing 13, no. 1 (January 1, 2009): 95–101. doi:10.1007/s00779-007-0165-0.

Color aids gender recognition

Behavioral

  • Hill, H., Bruce, V., & Akamatsu, S. (1995). Perceiving the sex and race of faces: the role of shape and colour. Proceedings. Biological Sciences / The Royal Society, 261(1362), 367–373. doi:10.1098/rspb.1995.0161
  • Tarr, M. J., Kersten, D., Cheng, Y., & Rossion, B. (2001). It’s Pat! Sexing faces using only red and green. Journal of Vision, 1(3), 337–337. doi:10.1167/1.3.337
  • “The Segmental Structure of Faces and Its Use in Gender Recognition” Accessed August 5, 2014. http://repository.cmu.edu/cgi/viewcontent.cgi?article=1392&context=psychology.

Color aids emotion recognition

Computational

  • Maglogiannis, Ilias, Demosthenes Vouyioukas, and Chris Aggelopoulos. “Face Detection and Recognition of Natural Human Emotion Using Markov Random Fields.” Personal and Ubiquitous Computing 13, no. 1 (January 1, 2009): 95–101. doi:10.1007/s00779-007-0165-0.

Detailed information for the references

Sorted by author and date.

Zotero Report

  • Color Face Recognition for Degraded Face Images

    Type Journal Article
    Author Jae-Young Choi
    Author Yong-Man Ro
    Author K.N. Plataniotis
    Volume 39
    Issue 5
    Pages 1217-1230
    Publication IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
    ISSN 1083-4419
    Date October 2009
    DOI 10.1109/TSMCB.2009.2014245
    Library Catalog IEEE Xplore
    Abstract In many current face-recognition (FR) applications, such as video surveillance security and content annotation in a Web environment, low-resolution faces are commonly encountered and negatively impact on reliable recognition performance. In particular, the recognition accuracy of current intensity-based FR systems can significantly drop off if the resolution of facial images is smaller than a certain level (e.g., less than 20 times 20 pixels). To cope with low-resolution faces, we demonstrate that facial color cue can significantly improve recognition performance compared with intensity-based features. The contribution of this paper is twofold. First, a new metric called ldquovariation ratio gainrdquo (VRG) is proposed to prove theoretically the significance of color effect on low-resolution faces within well-known subspace FR frameworks; VRG quantitatively characterizes how color features affect the recognition performance with respect to changes in face resolution. Second, we conduct extensive performance evaluation studies to show the effectiveness of color on low-resolution faces. In particular, more than 3000 color facial images of 341 subjects, which are collected from three standard face databases, are used to perform the comparative studies of color effect on face resolutions to be possibly confronted in real-world FR systems. The effectiveness of color on low-resolution faces has successfully been tested on three representative subspace FR methods, including the eigenfaces, the fisherfaces, and the Bayesian. Experimental results show that color features decrease the recognition error rate by at least an order of magnitude over intensity-driven features when low-resolution faces (25 times 25 pixels or less) are applied to three FR methods.
    Date Added Tuesday, August 05, 2014 12:39:03 PM
    Modified Tuesday, August 05, 2014 12:39:03 PM

    Tags:

    • Algorithms
    • Artificial Intelligence
    • Bayesian
    • Color
    • color face recognition
    • Color face recognition (FR)
    • Colorimetry
    • degraded face images
    • eigenfaces
    • Face
    • face databases
    • face recognition
    • face resolution
    • fisherfaces
    • Humans
    • Identification
    • image colour analysis
    • Image Interpretation, Computer-Assisted
    • Pattern Recognition, Automated
    • Subtraction Technique
    • variation ratio gain
    • variation ratio gain (VRG)
    • verification (VER)
    • video surveillance
    • web-based FR

    Attachments

    • IEEE Xplore Abstract Record
  • Selective attention to face identity and color studied with f MRI

    Type Journal Article
    Author Vincent P. Clark
    Author Raja Parasuraman
    Author Katrina Keil
    Author Rachel Kulansky
    Author Sean Fannon
    Author Jose Ma. Maisog
    Author Leslie G. Ungerleider
    Author James V. Haxby
    URL http://onlinelibrary.wiley.com/doi/10.1002/(SICI)1097-0193(1997)5:4<293::AID-HBM15>3.0.CO;2-F/abstract
    Rights Published 1997 Wiley-Liss, Inc.
    Volume 5
    Issue 4
    Pages 293-297
    Publication Human Brain Mapping
    ISSN 1097-0193
    Date January 1, 1997
    Journal Abbr Hum. Brain Mapp.
    DOI 10.1002/(SICI)1097-0193(1997)5:4<293::AID-HBM15>3.0.CO;2-F
    Accessed Tuesday, August 05, 2014 12:38:08 PM
    Library Catalog Wiley Online Library
    Language en
    Abstract Cortical areas associated with selective attention to the color and identity of faces were located using functional magnetic resonance imaging (fMRI). Six subjects performed tasks which required selective attention to face identity or color similarity using the same color-washed face stimuli. Performance of the color attention task but not the face attention task was associated with a region of activity in the collateral sulcus and nearby regions of the lingual and fusiform gyri. Performance of both tasks was associated with a region of activity in ventral occipitotemporal cortex that was lateral to the color responsive area and had a greater spatial extent. These fMRI results converge with results obtained from PET and ERP studies to demonstrate similar anatomical locations of functional areas for face and color processing across studies. Hum. Brain Mapping5:293–297, 1997. Published 1997 Wiley-Liss, Inc. This article was prepared by a group consisting of both United States government employees and non-United States government employees, and as such is subject to 17 U.S.C. Sec. 105.
    Date Added Tuesday, August 05, 2014 12:38:08 PM
    Modified Tuesday, August 05, 2014 12:38:08 PM

    Tags:

    • Brain Mapping
    • Color Perception
    • extrastriate cortex
    • face recognition
    • Magnetic Resonance Imaging
    • selective attention
    • Visual Cortex
    • Visual Pathways

    Attachments

    • Snapshot
  • Combining color and shape information for illumination-viewpoint invariant object recognition

    Type Journal Article
    Author A Diplaros
    Author T. Gevers
    Author I Patras
    Volume 15
    Issue 1
    Pages 1-11
    Publication IEEE Transactions on Image Processing
    ISSN 1057-7149
    Date January 2006
    DOI 10.1109/TIP.2005.860320
    Library Catalog IEEE Xplore
    Abstract In this paper, we propose a new scheme that merges color- and shape-invariant information for object recognition. To obtain robustness against photometric changes, color-invariant derivatives are computed first. Color invariance is an important aspect of any object recognition scheme, as color changes considerably with the variation in illumination, object pose, and camera viewpoint. These color invariant derivatives are then used to obtain similarity invariant shape descriptors. Shape invariance is equally important as, under a change in camera viewpoint and object pose, the shape of a rigid object undergoes a perspective projection on the image plane. Then, the color and shape invariants are combined in a multidimensional color-shape context which is subsequently used as an index. As the indexing scheme makes use of a color-shape invariant context, it provides a high-discriminative information cue robust against varying imaging conditions. The matching function of the color-shape context allows for fast recognition, even in the presence of object occlusion and cluttering. From the experimental results, it is shown that the method recognizes rigid objects with high accuracy in 3-D complex scenes and is robust against changing illumination, camera viewpoint, object pose, and noise.
    Date Added Tuesday, August 05, 2014 12:39:59 PM
    Modified Tuesday, August 05, 2014 12:39:59 PM

    Tags:

    • Algorithms
    • Artificial Intelligence
    • camera viewpoint
    • cameras
    • cluttering
    • Color
    • Colorimetry
    • color-invariant derivatives
    • Color-shape context
    • composite information
    • geometric invariants
    • illumination-viewpoint invariant object recognition
    • image colour analysis
    • Image Enhancement
    • Image Interpretation, Computer-Assisted
    • image matching
    • Image recognition
    • image retrieval
    • Imaging, Three-Dimensional
    • indexing scheme
    • Information Storage and Retrieval
    • Layout
    • Lighting
    • matching function
    • multidimensional color-shape context
    • Multidimensional systems
    • object occlusion
    • object pose
    • object recognition
    • Pattern Recognition, Automated
    • photometric changes
    • photometric invariants
    • Photometry
    • Robustness
    • Shape
    • similarity invariant shape descriptors
    • Subtraction Technique

    Attachments

    • IEEE Xplore Abstract Record
  • Perceiving the sex and race of faces: the role of shape and colour

    Type Journal Article
    Author H. Hill
    Author V. Bruce
    Author S. Akamatsu
    Volume 261
    Issue 1362
    Pages 367-373
    Publication Proceedings. Biological Sciences / The Royal Society
    ISSN 0962-8452
    Date Sep 22, 1995
    Extra PMID: 8587879
    Journal Abbr Proc. Biol. Sci.
    DOI 10.1098/rspb.1995.0161
    Library Catalog NCBI PubMed
    Language eng
    Abstract Theories of object recognition have emphasized the information conveyed by shape information, whereas theories of face recognition have emphasized properties of superficial features. In the experiments reported here we used novel technology to investigate the relative contributions of shape and superficial colour information to simple categorization decisions about the sex and 'race' of faces. The results show that both shape and colour provide useful information for these decisions; shape information was particularly useful for race decisions while colour dominated sex decisions. When both sources of information were combined, the dominant source depended on viewpoint, with angled views emphasizing the contribution of shape and the full-face view colour. The results are discussed within the context of theories of face recognition and their implications for telecommunication applications are considered.
    Short Title Perceiving the sex and race of faces
    Date Added Tuesday, August 05, 2014 12:36:25 PM
    Modified Tuesday, August 05, 2014 12:36:25 PM

    Tags:

    • Adult
    • Analysis of Variance
    • Asian Continental Ancestry Group
    • Continental Population Groups
    • European Continental Ancestry Group
    • Face
    • Female
    • Form Perception
    • Humans
    • Japan
    • Male
    • Sex Characteristics
    • Skin Pigmentation

    Attachments

    • PubMed entry
  • Face detection and recognition of natural human emotion using Markov random fields

    Type Journal Article
    Author Ilias Maglogiannis
    Author Demosthenes Vouyioukas
    Author Chris Aggelopoulos
    URL http://link.springer.com/article/10.1007/s00779-007-0165-0
    Volume 13
    Issue 1
    Pages 95-101
    Publication Personal and Ubiquitous Computing
    ISSN 1617-4909, 1617-4917
    Date 2009/01/01
    Journal Abbr Pers Ubiquit Comput
    DOI 10.1007/s00779-007-0165-0
    Accessed Tuesday, August 05, 2014 12:40:15 PM
    Library Catalog link.springer.com
    Language en
    Abstract This paper presents an integrated system for emotion detection. In this research effort, we have taken into account the fact that emotions are most widely represented with eye and mouth expressions. The proposed system uses color images and it is consisted of three modules. The first module implements skin detection, using Markov random fields models for image segmentation and skin detection. A set of several colored images with human faces have been considered as the training set. A second module is responsible for eye and mouth detection and extraction. The specific module uses the HLV color space of the specified eye and mouth region. The third module detects the emotions pictured in the eyes and mouth, using edge detection and measuring the gradient of eyes’ and mouth’s region figure. The paper provides results from the system application, along with proposals for further research.
    Date Added Tuesday, August 05, 2014 12:40:15 PM
    Modified Tuesday, August 05, 2014 12:40:15 PM

    Tags:

    • emotion recognition
    • Face detection
    • ICM
    • image segmentation
    • Markov random field
    • User Interfaces and Human Computer Interaction

    Attachments

    • Snapshot
  • Why does picture-plane inversion sometimes dissociate perception of features and spacing in faces, and sometimes not? Toward a new theory of holistic processing

    Type Journal Article
    Author Elinor McKone
    Author Galit Yovel
    Volume 16
    Issue 5
    Pages 778-797
    Publication Psychonomic Bulletin & Review
    ISSN 1531-5320
    Date Oct 2009
    Extra PMID: 19815781
    Journal Abbr Psychon Bull Rev
    DOI 10.3758/PBR.16.5.778
    Library Catalog NCBI PubMed
    Language eng
    Abstract Classically, it has been presumed that picture-plane inversion primarily reduces sensitivity to spacing/configural information in faces (distance between location of the major features) and has little effect on sensitivity to local feature information (e.g., eye shape or color). Here, we review 22 published studies relevant to this claim. Data show that the feature inversion effect varied substantially across studies as a function of the following factors: whether the feature change was shape only or included color/brightness, the number of faces in the stimulus set, and whether the feature was in facial context. For shape-only changes in facial context, feature inversion effects were as large as typical spacing inversion effects. Small feature inversion effects occurred only when a task could be efficiently solved by visual-processing areas outside whole-face coding. The results argue that holistic/configural processing for upright faces integrates exact feature shape and spacing between blobs. We describe two plausible approaches to this process.
    Short Title Why does picture-plane inversion sometimes dissociate perception of features and spacing in faces, and sometimes not?
    Date Added Tuesday, August 05, 2014 12:38:38 PM
    Modified Tuesday, August 05, 2014 12:38:38 PM

    Tags:

    • Face
    • Form Perception
    • Humans
    • Individuality
    • Models, Psychological
    • Space Perception
    • Visual Perception

    Attachments

    • PubMed entry
  • Coding visual images of objects in the inferotemporal cortex of the macaque monkey

    Type Journal Article
    Author K. Tanaka
    Author H. Saito
    Author Y. Fukada
    Author M. Moriya
    Volume 66
    Issue 1
    Pages 170-189
    Publication Journal of Neurophysiology
    ISSN 0022-3077
    Date Jul 1991
    Extra PMID: 1919665
    Journal Abbr J. Neurophysiol.
    Library Catalog NCBI PubMed
    Language eng
    Abstract 1. The inferotemporal cortex (IT) has been thought to play an essential and specific role in visual object discrimination and recognition, because a lesion of IT in the monkey results in a specific deficit in learning tasks that require these visual functions. To understand the cellular basis of the object discrimination and recognition processes in IT, we determined the optimal stimulus of individual IT cells in anesthetized, immobilized monkeys. 2. In the posterior one-third or one-fourth of IT, most cells could be activated maximally by bars or disks just by adjusting the size, orientation, or color of the stimulus. 3. In the remaining anterior two-thirds or three-quarters of IT, most cells required more complex features for their maximal activation. 4. The critical feature for the activation of individual anterior IT cells varied from cell to cell: a complex shape in some cells and a combination of texture or color with contour-shape in other cells. 5. Cells that showed different types of complexity for the critical feature were intermingled throughout anterior IT, whereas cells recorded in single penetrations showed critical features that were related in some respects. 6. Generally speaking, the critical features of anterior IT cells were moderately complex and can be thought of as partial features common to images of several different natural objects. The selectivity to the optimal stimulus was rather sharp, although not absolute. We thus propose that, in anterior IT, images of objects are coded by combinations of active cells, each of which represents the presence of a particular partial feature in the image.
    Date Added Tuesday, August 05, 2014 12:38:29 PM
    Modified Tuesday, August 05, 2014 12:38:29 PM

    Tags:

    • Animals
    • Brain Mapping
    • Color
    • Discrimination (Psychology)
    • Macaca
    • Neurons
    • Photic Stimulation
    • Temporal Lobe
    • Visual Perception

    Attachments

    • PubMed entry
  • It's Pat! Sexing faces using only red and green

    Type Journal Article
    Author M. J. Tarr
    Author D. Kersten
    Author Y. Cheng
    Author B. Rossion
    URL http://www.journalofvision.org/content/1/3/337
    Volume 1
    Issue 3
    Pages 337-337
    Publication Journal of Vision
    ISSN , 1534-7362
    Date 12/12/2001
    Journal Abbr J Vis
    DOI 10.1167/1.3.337
    Accessed Tuesday, August 05, 2014 12:36:04 PM
    Library Catalog www.journalofvision.org
    Language en
    Abstract The reflectance properties of facial hair and skin across sexes produce different degrees of red and green in male (more red) and female (more green) faces. Consequently, measuring the overall ratio of red/green energy in a face is sufficient for accurate sex classification. The optimal red/green threshold for discriminating 200 Caucasian faces by sex yielded an accuracy rate of 75% correct with a d' of 2.0 Faces had no makeup and were edited to remove all hair around the head. A second set of Caucasian faces produced similar results. Preliminary analyses suggest that the red/green ratio is also sufficient for sex classification of Asian and African faces. In contrast, pre-pubescent Caucasian faces were classified at chance. Thus, the red/green difference between males and females may be attributed to post-puberty sexual dimorphism in the spectral properties of human faces. We compared these computational findings with the human ability to discriminate male faces from females faces. To prevent observers from relying on shape information useful for sex classification, the 200 Caucasian faces were dramatically blurred using a Gaussian filter. Faces were presented for 100ms and observers simply judged whether each face was male or female. For female faces there was a -0.66 correlation between red/green ratio and accuracy in sex classification; for males the correlation was +0.42. Reinforcing the relationship between our model and human performance, observers were at chance in their ability to discriminate pre-pubescent faces. Our results may provide a mechanism for rapid sex classification through the differential response of early color opponent processes to male and female faces. In sum, red/green energy appears to be a reliable cue for fast and accurate discrimination of faces by sex.
    Date Added Tuesday, August 05, 2014 12:36:04 PM
    Modified Tuesday, August 05, 2014 12:36:04 PM

    Attachments

    • Snapshot
  • The importance of the color information in face recognition

    Type Conference Paper
    Author L. Torres
    Author J. Y. Reutter
    Author L. Lorente
    Volume 3
    Pages 627-631 vol.3
    Date 1999
    DOI 10.1109/ICIP.1999.817191
    Library Catalog IEEE Xplore
    Conference Name 1999 International Conference on Image Processing, 1999. ICIP 99. Proceedings
    Abstract A common feature found in practically all technical approaches proposed for face recognition is the use of only the luminance information associated to the face image. One may wonder if this is due to the low importance of the color information in face recognition or due to other less technical reasons such as the no availability of color image database. Motivated by this reasoning, we have performed a variety of tests using a global eigen approach developed previously, which has been modified to cope with the color information. Our results show that the use of the color information embedded in a eigen approach, improve the recognition rate when compared to the same scheme which uses only the luminance information
    Proceedings Title 1999 International Conference on Image Processing, 1999. ICIP 99. Proceedings
    Date Added Tuesday, August 05, 2014 12:38:52 PM
    Modified Tuesday, August 05, 2014 12:38:52 PM

    Tags:

    • Availability
    • Color
    • color image database
    • color information
    • Covariance matrix
    • face image
    • face recognition
    • global eigen approach
    • image colour analysis
    • Image databases
    • Image recognition
    • luminance information
    • Performance evaluation
    • principal component analysis
    • reasoning
    • Testing
    • Vectors
    • visual databases

    Attachments

    • IEEE Xplore Abstract Record
  • Contribution of color to face recognition

    Type Journal Article
    Author Andrew W. Yip
    Author Pawan Sinha
    Rights (c) 2012 APA, all rights reserved
    Volume 31
    Issue 8
    Pages 995-1003
    Publication Perception
    ISSN 1468-4233(Electronic);0301-0066(Print)
    Date 2002
    DOI 10.1068/p3376
    Library Catalog APA PsycNET
    Abstract One of the key challenges in face perception lies in determining how different facial attributes contribute to judgments of identity. This study focuses on the role of color cues. Although color appears to be a salient attribute of faces, past research has suggested that it confers little recognition advantage for identifying people. Using 37 subjects (aged 18-40 yrs) with normal or corrected-to-normal vision, the authors report experimental results suggesting that color cues do play a role in face recognition and their contribution becomes evident when shape cues are degraded. Under such conditions, recognition performance with color images is significantly better than that with gray-scale images. The results also indicate that the contribution of color may lie not so much in providing diagnostic cues to identity as in aiding low-level image-analysis processes such as segmentation.
    Date Added Tuesday, August 05, 2014 12:37:36 PM
    Modified Tuesday, August 05, 2014 12:37:36 PM

    Tags:

    • *Color
    • *Cues
    • *Face Perception
    • visual discrimination

    Attachments

    • APA PsycNET Snapshot
  • The segmental structure of faces and its use in gender recognition - viewcontent.cgi

    Type Attachment
    URL http://repository.cmu.edu/cgi/viewcontent.cgi?article=1392&context=psychology
    Accessed Tuesday, August 05, 2014 12:37:04 PM
    Date Added Tuesday, August 05, 2014 12:37:04 PM
    Modified Tuesday, August 05, 2014 12:37:08 PM

resources/pdf_document_resources/topic_face_color.txt · Last modified: 2019/05/22 16:08 (external edit)