People of all ages explore the world through looking. Recently, Raz, Cao et al. (2025) built an image-computable model (RANCH) that predicts adults’ and infants’ looking behavior to a large stimulus set, including graded responses to changes in pose, animacy, and number. This model succeeded despite having only a perceptual embedding space of stimuli. However, looking may be influenced by non-perceptual considerations. Using the same data, we found that adults’ behaviors challenge a key assumption of perceptual-only account: since the perceptual distance between two items is symmetrical, behavior guided only by perceptual space should also be symmetrical. Yet, adults did not treat changes in different directions as mere reciprocal transformations. For instance, adults looked longer at magical appearance than disappearance. We suggest that image-computable models of looking behavior would benefit from representations of objects, in addition to perceptual features of images.