Smile detection plays an important role in human emotion analysis and has wide applications. However, there is still a gap between the performance of the current smile detection algorithms and real-world applications, due to variations of head pose and environment noise. We propose a robust framework based on convolutional neural networks (CNNs) for smile detection. To alleviate the influence of head pose variations and improve performance, the proposed framework customizes two-feature learning layers such as (1) smile feature extraction layer is constructed by hidden factor analysis for learn...