Applied Abstraction – the terminal / instrumental dichotomy

Note: I use the words “value” and “goal” interchangeably in this post.

Terminal values are pursued for their own sake. Instrumental values are pursued as a means to attaining terminal values, or other instrumental values closer to the terminal values.

For example, if I had the ultimate goal of maximizing profit, I might break that up into the instrumental goals of increasing revenue and decreasing cost. Depending on the situation, those two instrumental goals can be broken up into even more specific instrumental sub-goals.

Instrumental sub-goals help to reduce repetition in thought. Conditions change in such a way as to change the actions demanded by an instrumental sub-goal, but not change the validity of the way in which the terminal goal is divided into sub-goals.

Irrationality is often caused by holding on to instrumental sub-goals even though they no longer serve your terminal goal. This is what Eliezer calls thought caching.

One Response to “Applied Abstraction – the terminal / instrumental dichotomy”