What is the value of probability? Keren and Teigen (2001) demonstrated that people prefer extreme probability (“10%” or “90%”) to medium probability (“50%”) and high probability (“90%”) to low probability (“10%”), and proposed that people’s perception of the value of probability phrases follows the principle of searching for definitive predictions. The present study proposes that this principle aligns with information theory and predicts that people’s judgments of informativeness will vary according to their prior beliefs. Additionally, this study also proposes that surprisingness judgment also obey the prediction from the information theory. To examine these propositions, this study required participants to estimate the valuableness and surprisingness for probability phrases expressing the winning probabilities of gambles. To manipulate prior beliefs about winning a gamble, the study created four conditions where the winning amounts varied. Results indicated that participants' estimations of the informativeness of the probability phrases changed in accordance with predictions from the information theoretic analysis.