What and how do people learn about visual appearance from language? We test the hypothesis that in the absence of sensory evidence, people born blind use abstract causal knowledge to infer object appearance. Congenitally blind (n=19) and sighted adults (n=59) reported how many colors two types of artifacts were likely to have: artifacts for which having many colors is intended to facilitate function (n=30, e.g., fairytale book, fruit candies), and artifacts for which colorfulness is irrelevant or distracting (n=30, e.g., instructional manual, painkillers). The number of colors estimated per object was highly correlated across groups. Blind and sighted people assigned more colors to artifacts for which colorfulness facilitates function and appealed to makers’ intentions in open-ended explanations. A text-only version of GPT-4 generated similar but non-identical colorfulness estimates compared to humans. Our findings suggest that people infer the appearance of unseen objects using causal ‘intuitive theories’ informed by linguistic evidence.