It is now well-established that counterfactual reasoning takes place when people make moral judgments. Less is known about which counterfactuals lead to stronger moral judgment, especially when judging agents who unknowingly produce negative consequences. We explored the relationship between counterfactual salience and responsibility ascription in two experiments. In Experiment 1, we asked people to produce counterfactual alternatives to a vignette they read spontaneously. We manipulated whether agents who produced harm knew the relevant information beforehand and what the reasons for the possible ignorance were. The counterfactual type that people first came up with (e.g., related to external factors or agent's actions) mediated the relationship between the condition and responsibility ratings. Experiment 2 investigated the causal connection between certain counterfactual types and responsibility ascription. We show that guiding people to consider alternative perpetrator's actions leads to a higher tendency to ascribe responsibility than considering victim's actions.