Data visualizations often display averages without raw data to simplify communication and enhance understanding, especially for lay audiences. However, the theory that such simplification improves understanding remains untested. Here, we test this theory’s most basic prediction—that at minimum, the average itself is conveyed better by plotted averages than by plotted raw data. Remarkably, we find the opposite: under a wide range of conditions, overall accuracy of average estimation is higher with raw data. This is due to frequent, severe misinterpretations of both bar and line graphs depicting averages. In contrast, raw data yields some variability but few outright errors; notably, the observed variability is comparable to the uncertainty captured by confidence intervals. We conclude that plotted raw data provides valuable context that helps prevent misunderstandings of the average. Our findings challenge the notion that plotted averages alone yield enhanced understanding and emphasize the value of raw data in communicating evidence.