Evidence of Intent

In “Free Will – Even for Robots“, John McCarthy describes what he feels are the minimal criteria for a complex system to be considered as possessing free will, namely the ability to state “I can, but I won’t”.

The decision process of such an entity is divided into two phases, starting with the consideration of all possible actions, i.e. what it can do, and then further searching within that set for what it wants most to do. (more…)

Thanks to some very good points made by wakaba I’ve decided to write a followup post.

More cash is always worth more than less cash. Rationally trading two differing amounts of the same currency is thus impossible. A stock has risk; its future dividend is not guaranteed. Estimating the value of a stock involves constructing a probability distribution for the future stock dividends.

Different people have different information, and may thus infer different distributions. However, even when they have the same beliefs and infer the same distribution, they will still price the stock differently depending on their differing appetites for risk. For example, a grad student would be willing to pay less than a millionaire for the same 50% chance at winning $10,000. See also the St Petersburg Paradox.

A corollary of this line of logic: given that two people infer the same probability distribution A, they will arrive at the same price for the stock as the uncertainty of A becomes smaller and the appetite for risk becomes irrelevant.

In a two-way barter trade, two people (1 and 2) exchange two objects (A and B). From the voluntary nature of the transaction, we conclude that both people value their new possessions more than their old; to one person A > B while to the other B > A. These two statements contradict each other, but that’s okay because they apply to different people; Person 1 prefers A to B while Person 2 prefers B to A. In fact, this contradiction is what makes trade possible. At this point, there is little temptation to label either person as being wrong.

Now imagine that prior to the trade, the two people had in fact just walked out of the same store, having bought the objects of the trade in the store. At best, if both objects cost the same, the trade seems pointless. In all other cases, one of the people could have done better by just paying less for his post-trade object in the store. This untaken better option prompts us to label the choice as irrational.

(more…)