Feature Relationships Learning Incorporated Age Estimation Assisted by Cumulative Attribute Encoding

Qing Tian , Meng Cao and Tinghuai Ma

Abstract: The research of human facial age estimation (AE) has attracted increasing attention for its wide applications. Up to date, a number of models have been constructed or employed to perform AE. Although the goal of AE can be achieved by either classification or regression, the latter based methods generally yield more promising results because the continuity and gradualness of human aging can naturally be preserved in age regression. However, the neighbor-similarity and ordinality of age labels are not taken into account yet. To overcome this issue, the cumulative attribute (CA) coding was introduced. Although such age label relationships can be parameterized via CA coding,the potential relationships behind age features are not incorporated to estimate age. To this end, in this paper we propose to perform AE by encoding the potential age feature relationships with CA coding via an implicit modeling strategy. Besides that, we further extend our model to gender-aware AE by taking into account gender variance in aging process. Finally, we experimentally validate the superiority of the proposed methodology.

Keywords: Age estimation, cumulative attribute, gender-aware age estimation,correlation relationship learning.

1 Introduction

Human facial age estimation (AE) is an important research topic and attracted a lot of attention because of its applications in such as age-oriented product and service recommendation [Fjermestad and Romano (2006)], safety monitoring [Guo, Fu, Dyer et al. (2008); Lanitis, Taylor and Cootes (2004)], identity recognition [Jain, Dass and Nandakumar (2004)], etc. For the task of predicting human facial age, it is to estimate the old degree of a face image given its appearance representations (i.e. feature representations). To achieve the goal of AE, great numbers of models have been developed or employed so far. Summing up, they can be grouped into classification based models [Lanitis, Taylor and Cootes (2004); Geng, Yin and Zhou (2013); Alnajar, Shan,Gevers et al. (2012); Sai, Wang and Teoh (2015)], regression-based methods [Lanitis,Taylor and Cootes (2002); Fu, Xu and Huang (2007); Luu, Ricanek, Bui et al. (2009);

Yan, Wang, Tang et al. (2007); Yan, Wang, Huang et al. (2007); Geng, Zhou and Smithmiles (2007); Chang, Chen and Huang (2011); Li, Liu, Liu et al. (2012a, 2012b); Li(2012); Tian, Xue and Qiao (2016); Tian and Chen (2017)] and hybrid-based models[Guo, Fu, Dyer et al. (2008); Guo, Fu, Dyer et al. (2008); Kohli, Prakash and Gupta(2013)]. In the framework of classification based AE, each age is treated as a separated class and by this way the facial age of a given face is typically predicted using the trained AE classifier. Along this line, the authors of Lanitis et al. [Lanitis, Draganova and Christodoulou (2004)] extracted coefficients of active appearance models as feature representations and then employed neural networks to classify the facial age. The authors of Ueki et al. [Ueki, Hayashida and Kobayashi (2006)] constructed a mixed Gaussian model to perform age-group classification. More recently, the conditional probability networks model (abbreviated as CPNN) was generated by the authors of Geng et al. [Geng,Yin and Zhou (2013)] to perform natural AE by incorporating the distributions information of neighboring ages. Further, the authors of Geng et al. [Geng, Yin and Zhou (2013)]proposed fuzzy-set based soft-AE by taking into account the similarities of neighboring ages. Besides, the ELM was also introduced to classify nonlinear age patterns [Sai, Wang and Teoh (2015)]. Motivated by the great success of deep learning paradigm, these deep models with layers of nonlinear representation learning were successively presented to more precisely classify the appearance age of a given face image [Niu, Zhou, Wang et al.(2016); Yang, Gao, Xing et al. (2016); Xing, Li, Hu et al. (2017)].

In fact, human facial appearance is aging gradually, implying that the problem of AE should be more regarded as a regression problem. To this end, the quadratic functions[Lanitis, Taylor and Cootes (2002)] were adopted to regress facial age by function fitting.And, the authors of Fu et al. [Fu, Xu and Huang (2007)] extended AE with multiple directions of linear regression. To eliminate the impact of outlier age patterns, the softversion of SVR model [Luu, Ricanek, Bui et al. (2009)] was also employed for age regression, and more complex SDP models were later adopted to handle AE with annotation noises [Yan, Wang, Tang et al. (2007); Yan, Wang, Huang et al. (2007)].Motivated by the fact that some age patterns are usually missed in most of current benchmark aging databases, component regression type model-AGES was constructed to regress missing age patterns to benefit AE accuracy [Geng, Zhou and Smith-Miles(2007)]. Although the reviewed methods above are reasonable themselves from their motivations, nearly all of these methods ignore the ordinal characteristic in the process of facial appearance aging, thus leading to non-optimal AE accuracy [Chen, Gong, Xiang et al. (2013); Chang, Chen and Huang (2011); Tian, Xue and Qiao (2016)]. To cater such a characteristic, the authors in Chang et al. [Chang, Chen and Hung (2011)] modeled an ordinal regressor called OHRank for age regression. Later, the authors in Li et al. [Li, Liu,Liu et al. (2012)] proposed to construct an ordinal metric space to benefit subsequent AE decision making. Then, Li et al. [Li, Liu, Liu et al. (2012)] extended feature selection learning to AE by removing redundant features. Beyond the work of Li et al. [Li, Liu, Liu et al. (2012)], the authors of Li et al. [Li, Liu, Liu et al. (2015)] modeled an ordinal distance space with taking into account both the ordinality and distribution structure of human aging sequence.

Figure 1: Ordinality and similarity of neighboring ages in facial appearance. The digit under each image indicates its facial age

In fact, apart from the ordinal relationships of age classes, another characteristic of them is that neighboring facial appearances look more similar than those further distributed,just as shown in Fig. 1. As claimed in Tian et al. [Tian, Xue and Qiao (2016); Tian and Chen (2017)] that incorporating these two characteristics of ages in AE learning is beneficial to improve the generalization performance of learned AE estimator. To nobly model the ordinality and similarity of neighboring ages, the authors in Chen et al. [Chen,Gong, Xiang et al. (2013)] constructed the so-called cumulative attribute (CA)representation to encode the age labels. Concretely, they first mapped original feature representation of a face image into CA coding, and then regressed its corresponding scalar age value from the CA coding. The whole pipeline of cumulative attribute coding based age estimation is demonstrated in Fig. 2. Although such a coding scheme yielded better AE results than the previous ones, it just can model the relationships among age semantic labels but ignores the underlying relationships among the original feature representations. Actually, underlying correlations may exist among the entries of original feature representations due to that they are extracted from the same face image. Moreover,the distribution space of face images typically follows some mixed Gaussian distributions[Bocklet, Maier, Bauer et al. (2008)]. Motivated by this fact, in this paper, we propose to model the underlying relationships among age original feature representations nobly by correlation matrix scheme with desirable analytical solutions. By this way, the mutual relationships among the original features can be characterized and incorporated in AE learning. Moreover, we further extend the model to gender-aware scenarios by incorporating gender variance between male and female. Finally, through experiments,we demonstrate the effectiveness of the proposed method. The main contributions of the paper are four-fold as follows:

1.Constructing a correlation learning model for age estimation (AE) to automatically exploit the underlying relationships among facial representations with CA coding.

2.Extending the proposed model to gender-aware AE scenario by incorporating gender variance.

3.Providing closed-form solutions via alternating optimization for the proposed models.

4.Experimentally validating the effectiveness of the proposed methods.

Figure 2: The pipeline of cumulative attribute coding based age estimation

The rest of this paper is organized as follows. Section 2 presents the background contents of CA coding based AE. Section 3 introduces the proposed method. Section 4 reports the validation results. Finally, Section 5 gives the conclusions and future works.

2 Background

For the problem of AE, assume we adopt the regression functionf(x)=wT x+bto predict the age value for a given face imagex∈R, withw∈Randb∈Rbeing the regression coefficients vector and bias. To obtain the parameters{w,b}of the above regression function, we need to optimize the following objective function on the given training instances and their age labelsas follows:

where the first term indicates the empirical loss on the training set, the second term is dedicated to control the complexity of the model, and λare predefined regularization coefficient. However, as analyzed in the introduction section, Eq. (1) is just a general regression function without particularly preserving the ordinality and neighbor-similarity of ages. To overcome this issue, the concept of cumulative attribute (CA) coding was proposed in Chen et al. [Chen, Gong, Xiang et al. (2013)]. We assume the scale of human aging sequence is 100, then the lengthKof CA coding is 100. According to Chen et al.[Chen, Gong, Xiang et al. (2013)], each element of CA coding vector yi∈RKof an instancexiis defined as

Through incorporating Eq. (2) into Eq. (1), Eq. (1) becomes

whereW=∈Rd×Kis the regression coefficient matrix andb∈RKstands for the bias vector. After solving the parameters of Eq. (3), we can obtain the CA coding vector of an instance and then generate its age label by off-the-self regression models.

3 The proposed methodology

3.1 Feature relationships leaning incorporated AE with CA coding

As claimed in the introduction section, although the CA coding can encode the ordinality and neighbor-similarity of age labels, the underlying correlations between the feature representations of face samples are not taken into account, leaving a perform space can be filled. Fortunately, the distribution space of face instances typically comply with some Gaussian distributions [Bocklet, Maier, Bauer et al. (2008)]. Motivated by this fact and the Gaussian modeling scheme [Yu and Yeung (2012)], we here propose to adopt the correlation matrix (denoted as ?) to characterize the mutual relationships between the age feature representations. Inspired by the fact that the feature presentations of an instance is operated inter-product with the regression coefficient matrix (see Eq. (3)), we can resort to model the mutual relationships between the feature representations of the instances nobly by regularizing the correlations between the regression coefficient matrixW∈Rd×Kusing correlation matrix ?.

Objective F unction.Through the above analyzed modeling scheme, the objective function of feature relationships leaning incorporated AE with the assistance of CA coding can be formulated as

whereW=∈Rd×Kis the projection coefficient matrix,b∈RKis the bias vector,?∈Rd×dis the correlation matrix used to model the mutual relationships between the feature rows ofW. The constraints are dedicated to control the complexity of the model.

Because of the semi-definiteness of?, it thus can be characterized the underlying relationships between the rows (corresponding to each component of the feature representations) of projection coefficient matrix. More specifically, because the element?i,jaccordingly characterizes the correlations between thei-th andj-th components of age feature representations, so its magnitude and signal (positive, negative or zero)happen to recognize the degree and type (positive-related, negative-related or unrelated)of mutual relationships between the two feature components. In turn, these recognized correlations can be used to regularize the learning of the projection matrix. By this way,the underlying correlations between the age feature representations can be incorporated to benefit the AE. More importantly, such a modeling strategy can automatically recognize the correlation degrees and types of the feature representations from the training samples,without requiring any prior knowledge. The proposed modeling strategy helps AE escapes from the drawbacks invoked by human defined unnatural correlation priors.

Optimization.It can be found that objective function in Eq. (4) is biconvex w.r.t. the variables ｛W,b, ?｝. Concisely, the objective function is convex w.r.t.｛W,b｝when?is fixed, and convex w.r.t.?if ｛W,b｝are fixed. To obtain the variables in Eq. (4),we can thus take an alternating optimization algorithm to solve it.

· With fixed ?, Eq. (4) essentially becomes

Calculating the gradients of Eq. (5) w.r.t.｛W,b｝yields

whereIdis an identity matrix and

The solutions of ｛W,b｝can be obtained analytically from Eq. (6).

· With fixed ｛W,b｝, Eq. (4) can essentially be rewritten as

which also has analytical solution as

In order to cater for the constraints of Eq. (4),?should be properly initialized. For this,we assume all the feature components between training samples are unrelated, so?can be initially set to. Then, we repeat the above two operation steps alternatingly until maximal iterations meet or convergence. With the obtained ｛W,b｝, we can perform facial image AE.

3.2 Gender-aware AE wi th incorporating w ithin- and between-gender f eature relationships

Although the feature correlations between age feature representations are incorporated to benefit AE, the gender variance between human male and female is not taken into account yet. Related research [Tian and Chen (2018)] shows that the male and female are aging significantly differently, which affect the performance of AE. Motivated by this fact, we here take into account such underlying gender variance to AE by incorporating the underlying correlations between human male and female. To this end, we need to separate the loss term in Eq. (4) for male and female samples, respectively. Besides that,in order to model the underlying relationshipswithin-genderandbetween-gendersimultaneously, we here also construct correlation matrices to characterize them.

Objective Function.Based on the analysis above, we can accordingly formulate the objective function of gender-aware AE as follows

mfEq. (9) can be simplified by introducing projection matrixWto unifyWmandWfas

Optimization.Obviously, Eq. (10) is also biconvex w.r.t. to all variables, thus we can also take an alternating optimization algorithm to solve it.

· With fixed ?mf, Eq. (10) can be written as

We calculate the derivatives of Eq. (11) w.r.t.｛W,bm,bf｝and let them to zeros,yielding that

where

which in form has the same closed-form solution as Eq. (7).

We repeat the above two steps until maximal iterations meet or convergence. Then, all the model variables of Eq. (10) can be obtained. It should be noted that the initialization ofdifferent from that in Section 3.1.

4 Experiment

4.1 Datasets and setting

To validate effectiveness of the proposed method, we conduct age estimation experiments on three widely-used benchmark datasets, i.e. FG-NET [Guo, Fu, Dyer et al. (2008)],Morph Album I and Album II [Geng, Yin and Zhou (2013)]. Specially, FG-NET consists of 1,002 images taken from 82 individuals aging from 0 to 69; Morph Album I contains 1,690 facial pictures from 631 African and European people with age ranging from 15 to 68;the Morph Album II database is relatively large and has over 55,000 face pictures aged between 16 and 77. Image examples of the three datasets are shown in Fig. 3.

Figure 3: Image examples from FG-NET (a), Morph Album I (b) and Album II (c)

For experimental setting, we extract AAM from FG-NET and Morph Album I and BIF from Morph Album II as their feature representations, respectively. All the regularization parameters are tuned through five-fold cross-validation in the searching range｛1e? 5, …,1e5｝. Following related literature [Geng, Yin and Zhou (2013); Fu, Xu and Huang (2007)], we also take the MAE criterion as performance measurement. For the second stage regression from CA codings to age labels, we uniformly adopt the nearest class center regressor for final AE decision making.

4.2 AE Experiment disregarding the gender-variance

To validate the effectiveness of the proposed method, we first perform AE experiments without particularly considering the aging difference between human male and female.More concisely, we randomly select 10%-90% of feature extracted datasets for training and the remaining for test. We repeat the runs ten times with random data splitting. The experimental results are shown in Tab. 1.

We can see from the above results that the proposed method consistently reduces the MAEs by up to 2% (from 5.81 to 5.67 on FG-NET), especially when the percent of training set is less than 40%. This finding shows that when the percent of training samples is low, so limited scale of training samples are not abundant to meet the whole distributions of sample feature space, leading to non-optimal solutions or even biased solutions without proper regularizations. By contrast, when the correlations between feature representations of limited training set are embedded (as a regularizer) in the objective function of AE learning for joint optimization, the underlying distribution relationships between the features can be taken into account to regularize the solutions of AE to better performance. On the other hand, with increasing training samples, the natural spatial distributions and correlations between feature representations can be increasingly covered by the training samples themselves. Besides, we also research the magnitudes and types of the correlations between sample representing features in a visual manner in Fig. 4.

From the correlation visualizations, we can discover that the correlation magnitudes between different elements of age features are not identical but varying. Moreover, these correlations can be grouped into three types: positive-related (corresponding elements of correlation matrix are positive), negative-related (corresponding elements of correlation matrix are negative), or unrelated (corresponding elements of correlation matrix are zeros). More interestingly, they are learned automatically from training data with human interference.

Table 1: Age estimation (AE) results (MAE±STD) comparison on the three datasets

Figure 4: Visualization of the learned average correlation matrix between the age feature representations on FG-NET (left), Morph Album I (middle) and Album II (right)

4.3 Gender-aware AE experiment

Table 2: Gender-aware age estimation (AE) results (MAE±STD) comparison on Morph Album Ⅰ

Table 3: Gender-aware age estimation (AE) results (MAE±STD) comparison on Morph Album Ⅱ

To further evaluate the proposed method in handling gender-variance to AE, we conduct gender-aware AE experiments on the Morph Album I and II, both of which are given gender annotations besides the age labels. We also repeat the runs ten times with random data splitting and report the averaged MAE and standard deviations in Tab. 2 and 3.

We can see from the comparative results that with increasing training samples, in all cases, our proposed method have reduced the MAEs against the original method just with CA coding, on the entire, male and female test sets. These results again demonstrate the effectiveness of the proposed methodology. Interestingly, the MAEs of AE on female are higher than those on male. This is mainly due to that the percentage of female samples is significantly lower than that of male on the Morph Album I and II datasets. So, if we extend these datasets with more female samples, the MAEs of AE on female will be reduced to comparable with that on male.

We also visually demonstrate the underlying correlations between age features withingender and between-gender in Fig. 5.

Figure 5: Visualization of the learned average correlation matrix between the age feature representations on Morph Album I (left) and Album II (right)

We can see from Fig. 5 that not only the features within male and female are correlated,but those between male and female have also relationships. And the types of these relationships arepositive-related, negative-related or unrelated.

5 Conclusion

In this paper, we proposed an AE methodology by encoding the potential age feature relationships with CA coding via an implicit modeling strategy. Although the types of relationships (positive-related, negative-related, or unrelated) between age features are not given in advance, they can be recognized nobly through the proposed model. To incorporate the aging difference between human male and female, we also extended the proposed model to gender-aware AE by taking into account the gender variance. Besides,we also presented alternating optimization algorithms to solve the models with analytical solutions. Finally, we experimentally demonstrated the effectiveness of the proposed methods. In the future, we will consider to extend the proposed models to cross-database age estimation scenarios [Tian and Chen (2017)] with large-margin learning [Gu, Sheng, Tay et al. (2015); Gu, Sun and Sheng (2017)] and deep learning for further performance gain.

Acknowledgment:This work was partially supported by the National Natural Science Foundation of China (61702273 and 61472186), the Natural Science Foundation of Jiangsu Province (BK20170956), the Natural Science Foundation of the Jiangsu Higher Education Institutions of China (17KJB520022), the Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions, and the Startup Foundation for Talents of Nanjing University of Information Science and Technology.