Learning perception-action contingencies from the environment depends on attention, to efficiently control cognition and selectively process sensory information. Shifting attention between sensory modalities incurs a cost in humans, other primates, and rodents, resulting in slower learning in the new modality. Previous set-shifting work in rats showed that increased difficulty of perceptual discrimination in the preceding modality increases the following shift cost. We studied this in humans by manipulating perceptual attention in one sensory modality, titrating task demand by staircase design, to test the effect on perceptual contingency learning in another modality. To accommodate the complexity of human learning, we introduce a Bayesian method to decompose and estimate learning characteristics from performance data. This method identifies the completion of rule acquisition and consolidation, accounting for individual variation in learning. Results show the expected modality shift cost, and offer new evidence in humans that shift cost is exacerbated by prior demands on attention.