Probability theory is a formalization of degrees of belief. Thus, expressing a particular belief in the form of a numerical probability is an attempt to be precise about how sure one is about the belief. The assigned probability is said to be calibrated when the person has the ability to assign a probability in such a way that the probability expresses the frequency with which the person is actually correct. In other words, the weatherman who says that there is an 80% chance of rain tomorrow, is well calibrated when his predictions (namely those which assign an 80% chance) come true 80% of the time. Likewise his predictions which assign a 70% chance come true 70% of the time, and so on.

Human beings do not possess a natural ability to assign such values accurately, but to some extent it can be achieved by training. The Wikipedia article on the Overconfidence effect discusses people’s natural ability to assess such probabilities and cites various studies indicating that untrained persons usually assign a value significantly higher than the actual frequency of being right. Thus for example someone might be right only 60 – 70% of the time when he says that the probability is 90%.

There are good mathematical reasons why this could have been expected in principle, as explained here and here. Nonetheless it is possible for training to correct this effect to a significant extent.

Without training, situations where the evidence would objectively give a reason to assign a probability of around 90% or higher are generally speaking very difficult for people to distinguish from “certainly true.” However, this applies less when the costs of being wrong would be higher, which is why people often buy insurance instead of assuming that it is certain that they won’t have any problems.