Aumann Agreement in Real Life

In an earlier post I discussed Robert Aumann’s mathematical theorem demonstrating that people with common priors who have common knowledge of their opinions cannot disagree about the probability of any fact.

As I said at the time, real human beings do not have a prior probability distribution, and thus the theorem cannot apply to them strictly speaking. To the degree that people do have a prior, that prior can differ from person to person.

A person’s prior can also be modified, something which is not meant to happen to a prior understood in the mathematical sense of Aumann’s paper. We can see this by means of a thought experiment, even if the thought experiment itself cannot happen in real life. Suppose you are given a machine that works like this: you can ask the machine whether some statement is true. It has a 100% chance of printing out a 1 if the statement is in fact true. If the statement is false, it has a 10% chance of printing a 1, and a 90% chance of printing a 0. You are allowed to repeat the question, with the responses having the same probability each time.

Thus if you ask about a false statement, it will have a 10% chance of printing a 1. It will have a 1% chance of printing 1 twice in a row, and a 0.1% chance of printing a 1 three times in a row.

Suppose you ask the question, “Are the Chronicles of Narnia a completely accurate historical account of something that really happened to various children from England?”

The machine outputs a 1. So you ask again. You get another 1. Let’s say this happens 10 times. The probability that this happens this many times with a false statement is one in ten billion.

In real life you would conclude that a machine that did this does not work as stated. But in our thought experiment, you know with absolute certainty that it does work as stated. So you almost certainly will conclude that the Chronicles of Narnia is an accurate historical account. The same will be true pretty much no matter what statement you test, given this result.

But it would be easy to compose far more than 10 billion mutually inconsistent statements. Thus it is logically inconsistent to assign a probability of more than one in ten billion to all such statements. So if you had a consistent and full prior distribution that you were prepared to stick to, then there should be some such statements which you will still believe to be false even after getting a 1 ten times from the machine. This proves that we do not have such a prior: the fact that the machine comes out this way tells us that we should admit that the prior for the particular statement that we are testing should be high enough to accept after the machine’s result. So for example we might think that the actual probability of the Chronicles of Narnia being an accurate historical account is less than one in ten billion. But if we are given the machine and get this result, we will change our mind about the original probability of the claim, in order to justify accepting it as true in those circumstances.

If someone disagrees with the above thought experiment, he can change the 10 to 20, or to whatever is necessary.

Although Aumann’s result depends on unchanging priors, in practice the fact that we can change our priors in this way makes his result apply more to human disagreements than it would in a situation where we had unchanging priors, but still diverse from other people’s priors.

Robin Hanson has published an extension of Aumann’s result, taking into account the fact that people have different priors and can reason about the origin of these priors. By stipulating certain conditions of rationality (just as Aumann does), he can get the result that a disagreement between two people will only be reasonable if they disagree about the origin of their priors, and in a particular way:

This paper presents a theoretical framework in which agents can hold probabilistic beliefs about the origins of their priors, and uses this framework to consider how such beliefs might constrain the rationality of priors. The basic approach is to embed a set of standard models within a larger encompassing standard model. Each embedded model differs only in which agents have which priors, while the larger encompassing model includes beliefs about which possible prior combinations might be realized.

Just as beliefs in a standard model depends on ordinary priors, beliefs in the larger model depend on pre-priors. We do not require that these pre-priors be common; pre-priors can vary. But to keep priors and pre-priors as consistent as possible with each other, we impose a pre-rationality condition. This condition in essence requires that each agent’s ordinary prior be obtained by updating his pre-prior on the fact that nature assigned the agents certain particular priors.

This pre-rationality condition has strong implications regarding the rationality of uncommon priors. Consider, for example, two astronomers who disagree about whether the universe is open (and infinite) or closed (and finite). Assume that they are both aware of the same relevant cosmological data, and that they try to be Bayesians, and therefore want to attribute their difference of opinion to differing priors about the size of the universe.

This paper shows that neither astronomer can believe that, regardless of the size of the universe, nature was equally likely to have switched their priors. Each astronomer must instead believe that his prior would only have favored a smaller universe in situations where a smaller universe was actually more likely. Furthermore, he must believe that the other astronomer’s prior would not track the actual size of the universe in this way; other priors can only track universe size indirectly, by tracking his prior. Thus each person must believe that prior origination processes make his prior more correlated with reality than others’ priors.

Despite the fact that Hanson’s result, like Aumann’s, is based on a particular mathematical analysis which remains much more rigid than real life, and in this sense cannot apply strictly to real life, it is not difficult to see that it does have strong analogies in real human disagreements. Thus for example, suppose a Christian believes that Christianity has a 98% chance of being true, and Islam a 1% chance. A Muslim, with whom he disagrees, believes that Islam has a 98% chance of being true, and Christianity a 1% chance. If they each believe, “Both of us believe in our religions because that is the one in which we were raised,” it is obvious that this disagreement is not reasonable. In order for each of them to be reasonable, they need to disagree about why they believe what they believe. Thus for example one might think, “He believes in his religion because he was raised in it, while I believe in mine because of careful and intelligent analysis of the facts.” The other obviously will disagree with this.

This particular example, of course, does not take into account the fact that belonging to a religion is not a matter of a particular claim, nor the fact that beliefs are voluntary, and both of these affect such a question in real life.

Nonetheless, this kind of disagreement about the origins of our beliefs is clearly a common phenomena in situations where we have a persistent disagreement with someone. In the end each person tends to attribute a particular source to the other person’s opinion, and a different source to his own, one which is much more likely to make his own opinion correct. But all of the same things should apply to these differing opinions about the origins of their beliefs. This suggests that in fact persistent disagreements are usually unreasonable. This corresponds to how people treat them. Once a disagreement is clearly persistent, and clearly will not be resolved by any amount of discussion, people think that the other person is being stubborn and unreasonable.

And in fact, it is very likely that one or both of the two is being stubborn and unreasonable. This will feel pretty much the same from each side, however; thus the fact that it feels to you like the other person is being stubborn and unreasonable, is not a good reason for thinking that this is actually the case. He is very likely to feel the same way about you. This will happen no matter who is actually responsible. Most often both partners contribute to it, since no one is actually perfectly reasonable.

The fact that belief is voluntary can be a mitigating factor here, if people recognize the moral influences on their beliefs. Thus for example the Christian and the Muslim in the above example could simply say, “It is not necessarily that I am more likely to be right, but I choose to believe this rather than that, for these personal reasons.” And in that case in principle they might agree on the probability of the truth of Christian and Islamic doctrines, and nonetheless reasonably hold different beliefs, on account of moral considerations that apply to them in particular.

The fact that people do not like to admit that they are wrong is a reason for a particular approach to disagreement. In the last post, we discussed the fact that since words and thoughts are vague, the particular content of a person’s assertions is not entirely determinate. They may be true in some ways, and not true in others, and the person himself may not be considering in which way he is making the claim. So it is much more productive to interpret the person’s words in the way that contains as much truth as possible. We have talked about this elsewhere. Such an understanding is probably a better understanding of the person in the first place. And it allows him to agree with you while excluding the false interpretations, and without saying, “I was wrong.” And yet he learns from this, because his original statement was in fact open to the false interpretations. There is nothing deceptive about this; our words and beliefs are in fact vague in this way and allow for this sort of learning. And cooperating in this way in a discussion will be mutually profitable. Since absolute precision is not possible, in general there is no one who has nothing at all to learn from another.

True and False Religion

What does it mean to say that a religion is true, or that it is false? The question is not as easy as it appears at first sight. Bertrand Russell, in Why I Am Not a Christianruns up against this difficulty. In order to explain why he is not a Christian, he has to know what it means to be a Christian in the first place:

As your Chairman has told you, the subject about which I am going to speak to you tonight is “Why I Am Not a Christian.” Perhaps it would be as well, first of all, to try to make out what one means by the word Christian. It is used these days in a very loose sense by a great many people. Some people mean no more by it than a person who attempts to live a good life. In that sense I suppose there would be Christians in all sects and creeds; but I do not think that that is the proper sense of the word, if only because it would imply that all the people who are not Christians — all the Buddhists, Confucians, Mohammedans, and so on — are not trying to live a good life. I do not mean by a Christian any person who tries to live decently according to his lights. I think that you must have a certain amount of definite belief before you have a right to call yourself a Christian. The word does not have quite such a full-blooded meaning now as it had in the times of St. Augustine and St. Thomas Aquinas. In those days, if a man said that he was a Christian it was known what he meant. You accepted a whole collection of creeds which were set out with great precision, and every single syllable of those creeds you believed with the whole strength of your convictions.

Nowadays it is not quite that. We have to be a little more vague in our meaning of Christianity. I think, however, that there are two different items which are quite essential to anybody calling himself a Christian. The first is one of a dogmatic nature — namely, that you must believe in God and immortality. If you do not believe in those two things, I do not think that you can properly call yourself a Christian. Then, further than that, as the name implies, you must have some kind of belief about Christ. The Mohammedans, for instance, also believe in God and in immortality, and yet they would not call themselves Christians. I think you must have at the very lowest the belief that Christ was, if not divine, at least the best and wisest of men. If you are not going to believe that much about Christ, I do not think you have any right to call yourself a Christian.

Thus Russell reduces being a Christian to believing in God, in the immortality of the soul, and that Christ was at least the best and wisest of men. Of course there are people who call themselves Christians who do not believe one or more of these things, and do not accept that you cannot call yourself a Christian without them. And other people might well give a different list. Thus for example St. Paul has his own requirements:

Now if Christ is proclaimed as raised from the dead, how can some of you say there is no resurrection of the dead? If there is no resurrection of the dead, then Christ has not been raised; and if Christ has not been raised, then our proclamation has been in vain and your faith has been in vain. We are even found to be misrepresenting God, because we testified of God that he raised Christ—whom he did not raise if it is true that the dead are not raised. For if the dead are not raised, then Christ has not been raised. If Christ has not been raised, your faith is futile and you are still in your sins. Then those also who have died in Christ have perished. If for this life only we have hoped in Christ, we are of all people most to be pitied.

Thus St. Paul says that belief in the resurrection of Christ, and therefore in a general resurrection, is required. Otherwise “your faith has been in vain,” which would certainly seem to say that your religion is not true.

Of course, St. Paul is polemically exaggerating the consequences of the position of his opponents. In the first place someone could believe in the resurrection of Christ without believing in a general resurrection. Likewise, even if Christ did not rise from the dead, it does not follow of necessity that anyone’s faith would be entirely vain, but that it would be vain in some respect, since he would still profit from it in various ways, such as by belonging to a Christian community. Similarly, even if Christians have a false belief in the immortality of the soul, there would still be more pitiable people in the world.

We can learn from these two examples. Russell says that you cannot “properly call yourself a Christian,” if you do not accept his list of three beliefs, while St. Paul says that “your faith is in vain” if you do not believe that Christ is risen. There is something common to the two. Some basic belief or beliefs are proposed, such that without these beliefs, it is not worthwhile to count yourself as a believer at all. For St. Paul, this has the form of saying that you should not bother to put your faith in Christ, while for Russell, this has the form of saying that you should not call yourself a Christian.

The basic difficulty is caused by the fact that being a Christian, considered in itself, is not a belief, but membership in a Christian community. Thus saying that “Christianity is true,” or that “Christianity is false,” ought to mean “belonging to a Christian community is true,” or that “belonging to a Christian community is false,” both of which are evidently absurd, since belonging to a community is not the kind of thing which is true or false. But since a Christian community happens to be a community of believers, we identify Christianity as a belief by saying that it is what that community believes.

But the problem is not resolved by this identification, for “what the Christian community believes” is somewhat indeterminate, since Christians believe different things. Russell and St. Paul resolve the issue in similar ways. Russell does so by saying that you cannot “properly call yourself a Christian,” unless you believe certain things, presumably because it is wrong to deceive people about your beliefs. St. Paul does so by saying that your faith is in vain if you think that Christ did not rise from the dead, presumably meaning that it is pointless for you to belong to a Christian community.

Thus both of them are saying that unless you think that such and such is true, it is a bad idea to be a Christian, that is, to belong to a Christian community.

With this analysis we can say in general what it means to say that a certain religion is true, or that it is false. If I say that Mormonism is true, I mean that there are certain true things usually believed by Mormons, which make it worthwhile to belong to a Mormon community, given that I accept those things. Likewise, if I say that Mormonism is false, I mean that there are things believed by Mormons that would make it worthwhile to be a Mormon, if they were true, but in fact those things are false, and consequently it is not worthwhile to be a Mormon. Or more directly, I mean that there are certain things normally believed by Mormons which happen to be false, and the fact that Mormons normally believe these false things, makes it not worthwhile for me to be a Mormon.

Someone might object that this leads to relativism, since according to this analysis, it seems that a religion might be true for one person, but false for another. For example, in an interview conducted by Sergiu Hart, Robert Aumann, the author of the agreement theorem we discussed earlier, explains, among other things, why he accepts Judaism:

H [Sergiu Hart]: So that’s the Center for Rationality. I know this doesn’t belong, but I’ll ask it here. You are a deeply religious man. How does it fit in with a rational view of the world? How do you fit together science and religion?

A [Robert Aumann]: As you say, it really doesn’t belong here, but I’ll respond anyway. Before responding directly, let me say that the scientific view of the world is really just in our minds. When you look at it carefully, it is not something that is out there in the real world. For example, take the statement “the earth is round.” It sounds like a very simple statement that is either true or false. Either the earth is round or it isn’t; maybe it is square, or elliptical, or whatever. But when you come to think of it, it is a very complex statement. What does roundness mean? Roundness means that there is a point, the “center” of the earth, such that any point on the surface of the earth is at the same distance from that center as any other point on the surface of the earth. Now that already sounds a little complex. But the complexity only begins there. What exactly do we mean by equal distance? For that you need the concept of a distance between two points. The concept of distance between two points is something that is fairly complex even if we are talking about a ball that we can hold in our hands; it involves taking a ruler and measuring the distance between two points. But when we are talking about the earth, it is even more complex, because there is no way that we are going to measure the distance between the center of the earth and the surface of the earth with a ruler. One problem is that we canít get to the center. Even if we could find it we wouldn’t be able to get there. We certainly wouldn’t be able to find a ruler that is big enough. So we have to use some kind of complex theory in order to give that a practical meaning. Even when we have four points and we say the distance from A to B is the same as the distance from C to D, that is fairly complex already. Maybe the ruler changes. We are using a whole big theory, a whole big collection of ideas, in order to give meaning to this very, very simple statement that the earth is round.

Don’t get me wrong. We all agree that the earth is round. What I am saying is that the roundness of the earth is a concept that is in our minds. It’s a product of a very complex set of ideas, and ideas are in people’s minds. So the way I think of science, and even of fairly simple things, is as being in our minds; all the more so for things like gravitation, the energy that is emitted by a star, or even the concept of a “species.” Yes, we are both members of the species homo sapiens. What does that mean? Obviously we are different. My beard is much longer than yours. What exactly does species mean? What exactly does it even mean to say “Bob Aumann” is sitting here? Is it the same Bob Aumann as five minutes ago? These are very complex ideas. Identity, all those things that we think of trivially on a day-to-day basis, are really complex ideas that are in our minds; they are not really out there. Science is built to satisfy certain needs in our minds. It describes us. It does have a relationship with the real world, but this relationship is very, very complex.

Having said that, I’ll get to your question. Religion is very different from science. The main part of religion is not about the way that we model the real world. I am purposely using the word “model.” Religion is an experience, mainly an emotional and aesthetic one. It is not about whether the earth is 5,765 years old. When you play the piano, when you climb a mountain, does this contradict your scientific endeavors? Obviously not. The two things are almost, though not quite, orthogonal. Hiking, skiing, dancing, bringing up your children; you do all kinds of things that are almost orthogonal to your scientific endeavor. That’s the case with religion also. It doesn’t contradict; it is orthogonal. Belief is an important part of religion, certainly; but in science we have certain ways of thinking about the world, and in religion we have different ways of thinking about the world. Those two things coexist side by side without conflict.

Well, it is your way of putting it. Let me enlarge on it. The observance of the Sabbath is extremely beautiful, and is impossible without being religious. It is not even a question of improving society; it is about improving one’s own quality of life. For example, let’s say I’m taking a trip a couple of hours after the Sabbath. Any other person would spend the day packing, going to the office, making final arrangements, final phone calls, this and that. For me it’s out of the question. I do it on Friday. The Sabbath is there. The world stops.

In short, you can be a moral person, but morals are often equivocal. In the eighties, copying software was considered moral by many people. The point I am making is that religion, at least my religion, is a sort of force, a way of making a commitment to conduct yourself in a certain way, which is good for the individual and good for society.

In the first part, Aumann is basically saying that science gives an idealized and approximate description of the world, rather than an exact description. In the second part he attempts to explain why he accepts Judaism, and he seems to be saying that it has little to do with the way the world is, and more to do with what is good for people. In other words, to explain it in the way we analyzed the truth of a religion, “Judaism is true” for Aumann because he believes that it is true that it is good and beautiful to observe the Sabbath, true that it is good to refrain from breaking copyright laws, and so on. And since these things are true it is worthwhile for him to be a member of a Jewish religious community.

You may or may not agree that the Sabbath is beautiful, and you may or may not agree that it is good to refrain from breaking copyright laws. But even if you do agree with these things, you probably don’t conclude that it is worthwhile for you to convert to Judaism. At the same time, you may realize that these things might well make it worthwhile for Robert Aumann to remain a Jew.

Thus our explanation seems to lead to relativism, because Judaism can be true for Aumann, but false for other people. However, there are several problems with calling this result relativism.

First of all, there was some remaining ambiguity in the way we defined the truth or falsity of a religion. Jews might normally believe certain true things, and given that Robert Aumann accepts those things, it might be worthwhile for him to remain a Jew. But it is possible that Jews also normally believe certain false things, such that if Aumann knew they were false, it would no longer be worthwhile for him to remain a Jew. Thus, for example, a Christian would argue that Jews falsely believe that Christ is not the Messiah, and that if Aumann knew that this was false, it would no longer be worthwhile for him to remain a Jew, but to convert to Christianity.

Thus we could make our definition more precise by saying that a religion is true if it is worthwhile to belong to that religion even after you know the truth or falsity of all the beliefs that the members of the religion usually hold, and that it is worthwhile by reason of some of the true things that they hold.

However, this does not sufficiently answer the charge of relativism, because it would still be possible that one religion would be true for one person, and not true for another person.

For example, suppose that theism is true, but that no divine revelation has been given. If Aumann realizes this, he might reasonably believe that it is worthwhile for him to remain a Jew, and unreasonable to convert to Islam, even after knowing the truth or falsity of every concrete belief held by Jews and Muslims. Likewise, a Muslim, knowing the same things, might reasonably believe that it is worthwhile for him to remain a Muslim, and unreasonable to convert to Judaism, even after knowing the truth or falsity of every concrete belief held by Muslims and Jews.

The answer in this case is that the situation simply does not imply relativism, because Aumann and the Muslim do not disagree about anything. Aumann may say, “Judaism is true,” and the Muslim may say, “Islam is true,” but when they explain what they mean and why they say it, they do not disagree with each other about any objective fact. This is no more relativism than it is relativism to admit that one person may prefer vanilla ice cream, and another person chocolate.

Thus, it is possible to mean something reasonable when saying that some religion is true, or that some religion is false. But in the end perhaps it would be better to avoid all the confusion in the first place, by following Robert Aumann’s example and simply distinguishing the question, “What is the world like?” from the question, “Is it, or would it be, good for me to belong to this community of believers?” Of course the answers to these questions are going to be related in various ways, but they are different questions.

Quick to Hear and Slow to Speak

St. James says in 1:19-20 of his letter,Let every man be quick to hear, slow to speak, slow to anger, for the anger of man does not work the righteousness of God.”

What does he mean? How is it possible for every man to be quick to hear and slow to speak? A conversation needs to have an approximately equal amount of listening and speaking. If each of two conversational partners insists on listening instead of speaking, the conversation will go nowhere. Whenever one is speaking, the other should be listening, and if one is listening, the other must be speaking, since it is not listening if neither of the two is saying anything.

The reference to anger is a clue. St. James is speaking of our natural tendencies, and saying that the natural tendency to anger is excessive and must be resisted. Likewise, we tend to have more of a desire to speak than to listen. We would rather explain our own position than listen to that of another. In order for a conversation to go well, each of the partners should restrain his own desire to express his own opinion, in order to listen to the other. This does not imply anything impossible any more than restraining anger is impossible; people have a naturally excessive desire to express themselves in the same way that they have a naturally excessive tendency to become angry. Thus St. James is not against a conversation which is equally composed of listening and speaking; but he is saying that such a conversation requires restraint on both parties to the conversation. A conversation without such restraint leads to situations where someone thinks, “he’s not listening to me,” which then leads precisely to the anger that St. James is opposing.

Robert Aumann has a paper, “Agreeing to Disagree”, which mathematically demonstrates that people having the same prior probability distribution and following the laws of probability, cannot have a different posterior probability regarding any matter, assuming that their opinions of the matter are common knowledge between them. He begins his paper:

If two people have the same priors, and their posteriors for a given event A are common knowledge, then these posteriors must be equal. This is so even though they may base their posteriors on quite different information. In brief, people with the same priors cannot agree to disagree.

We publish this observation with some diffidence, since once one has the appropriate framework, it is mathematically trivial.

The implication is something like this: one person may believe that there is a 50% chance it will rain tomorrow. Another person, having access to other information, such as having seen the weather channel, thinks that there is a 70% chance of rain. Currently these estimates are not common knowledge. But if the two people converse until they both know each other’s current opinion (which will possibly no longer be 50% and 70%), they must agree on the probability of rain, given that they have the same prior distribution.

There are several reasons why this does not apply to real human beings. First of all, people do not have an actual prior probability distribution; such a distribution means having an estimate of the original probability of every possible statement, and obviously people do not actually have such a thing. So not having a prior distribution at all, they cannot possibly have the same prior distribution.

Second, the theorem presumes that each of the two knows that each of the two is reasonable in exactly the sense required, namely having such a prior and updating on it according to the laws of probability. In real life no one does this, even apart from the fact that they do not have such a prior.

Various extensions of the theorem have been published by others, some of which come closer to having a bearing on real human beings. Possibly I will consider some of these results in the future. Even without such extensions, however, Aumann’s result does have some relationship with real disagreements.

We have all had good conversations and bad conversations when we disagreed with someone, and it is not so difficult to recognize the difference. In the best conversations, we may have actually come to partial or even full agreement, even if not exactly on the original position of either partner. In the worst conversations, neither partner budged, and both concluded that the other was being stubborn and unreasonable. Possibly the conversation descended to the point of anger, insults and attributing bad will to the other. And on the other hand we have also had conversations which were somewhat in the middle between these two extremes.

These facts are related to Aumann’s result because his result is that reasonable conversational partners must end up agreeing, this being understood in a simplified mathematical sense. Because of the simplifications it does not strictly apply to real life, but something like it is also true in real life, and we can see that in our experiences with conversations with others involving disagreements. In other words, basically whenever we get to the point where neither partner will budge, we begin to think that someone is being stubborn and at least somewhat unreasonable.

St. James is explaining how to avoid the bad conversations and have the good conversations. And that is by being “quick to hear.” It is a question of listening to the other. And basically that implies asking the question, “How is this right, in what way is it true?” If someone approaches a conversation with the idea that he is going to prove that the other is wrong, the other will get the impression that he is not being listened to. And this impression, in this case, is basically correct. A person sees what he is saying as true, not as false, so one who does not see how it could be true does not even understand it. If you say something, I do not even understand you, until I see a way that what you are saying could be so. And on the other hand, if I do approach a conversation with the idea of seeing what is true in the position that is in disagreement with mine, the conversation will be far more likely to end up as one of the good conversations, and far more likely to end in agreement.

Often even if a person is wrong in his conclusion, part of that conclusion is correct, and it is important for someone speaking with him to acknowledge the part that is correct before criticizing the part that is wrong. And on the other hand, even if a person’s conclusion is completely wrong, insofar as that is possible, there will always be some evidence for his conclusion. It is important to acknowledge that evidence, rather than simply pointing out the evidence against his conclusion.