Color temperature, as a crucial attribute influencing image color, plays a critical role in Image Aesthetics Assessment (IAA). Yet, within the existing IAA field, little light has been shed on assessing the aesthetic quality of image color temperature. To bridge this gap, we introduce a new task: Image Color Temperature Aesthetics Assessment (ICTAA). However, this task poses the following challenge: 1) Perceptual Sensitivity: humans exhibit high sensitivity to subtle shifts in color temperature, necessitating a model to enable fine-grained discrimination; 2) Spectral Continuity: The theoretical modeling of color temperature aesthetics requires continuous labels; however, the just-noticeable-difference property of human perception makes continuous labeling infeasible, necessitating a well-designed labeling strategy. 3) Label One-Sidedness: Color temperature annotations exhibit a certain degree of ambiguity and randomness; directly converting annotation results into hard labels discards the potential preference confidence of annotators. To address the aforementioned challenge, we make the following efforts. First, we propose a multi-modal contrastive learning framework, named ICTA2Net, that models color temperature differences between image pairs while strictly controlling other visual attributes. Second, leveraging weak supervision and color temperature transitivity, we discretely sample images based on an anchor image and human perceptibility to establish contrastive relations across different color temperatures. Third, we introduce an Information Entropy-weighted Accuracy (IEA) evaluation metric to better reflect the consistency between model predictions and human preference distributions. Finally, we construct a large-scale color temperature aesthetics dataset (ICTAA240K) and a comprehensive benchmark for validation. Experiments show our method outperforms existing IAA methods on this dataset, thereby setting an effective roadmap for ICTAA.