Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2012 Oct 8:6:138.
doi: 10.3389/fnins.2012.00138. eCollection 2012.

Don'T let me do that! - models of precommitment

Zeb Kurth-Nelson  1 , A David Redish
Affiliations

Don'T let me do that! - models of precommitment

Zeb Kurth-Nelson et al. Front Neurosci. .

Abstract

Precommitment, or taking away a future choice from oneself, is a mechanism for overcoming impulsivity. Here we review recent work suggesting that precommitment can be best explained through a distributed decision-making system with multiple discounting rates. This model makes specific predictions about precommitment behavior and is especially interesting in light of the emerging multiple-systems view of decision-making, in which functional systems with distinct neural substrates use different computational strategies to optimize decisions. Given the growing consensus that impulsivity constitutes a common point of breakdown in decision-making processes, with common neural and computational mechanisms across multiple psychiatric disorders, it is useful to translate precommitment into the common language of temporal difference reinforcement learning that unites many of these behavioral and neural data.

Keywords: decision-making; discounting function; neuroeconomics; precommitment; temporal diference reinforcement learning.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Precommitment arises from hyperbolic but not exponential discounting. (A) A state-space for precommitment (from Kurth-Nelson and Redish, 2010). The agent first chooses whether to enter state C or state N. From state C, a standard intertemporal choice is available, between a larger reward available later (LL) and a smaller reward available sooner (SS). This choice is outlined with a dashed box. But from state N, only LL is available. Thus choosing N represents precommitment. (B) In exponential discounting, values decay by the same percentage for each unit of delay, so if SS is preferred at state C, it must also be preferred at state N. (C), In hyperbolic discounting, values decay more steeply proximally to the outcome, so it is possible for SS to be preferred at state C, but for LL to be preferred at state N.
Figure 2
Figure 2
Distributed discounting enables precommitment in temporal difference learning. (A) Twenty exponential curves with discounting rates spread uniformly between 0 and 1 are shown in black. The average of these curves is shown in red. This average curve closely approximates a hyperbolic function. (B) Standard TD models cannot precommit because, at each state transition, discounting starts over, ensuring that if SS is preferred over LL at the time of C, then it is also preferred at the time of P (top pair of curves). When averaging a set of exponential discount curves, discounting is not reset at each state transition, so preferences can reverse between C and P (bottom pair of curves).
Figure 3
Figure 3
Shape of discounting curve strongly influences precommitment. (A) The actual discounting curves of two individuals are shown in solid lines, and the best-fit hyperbolic curves are shown in dashed lines. These two subjects were both fit by a hyperbolic function with ln(K) of approximately 0 (from a range of −13 to +4 across subjects). (B) Predicted precommitment behavior, based on actual discounting curve shape of each subject, using the following parameters: DC = 6 days, DL = 1 day, DS = 0, RL = $150, RS = $100. Subject 1 is expected to have a modest preference for SS over LL, and to be averse to precommitment. Meanwhile, subject 2 is expected to have a strong preference for SS over LL, but to favor precommitment. (Data from Chopra et al., , used with permission.)
See this image and copyright information in PMC

References

    1. Ainslie G. (1974). Impulse control in pigeons. J. Exp. Anal. Behav. 21, 485–489 10.1901/jeab.1974.21-485 - DOI - PMC - PubMed
    1. Ainslie G. (1992). Picoeconomics: The Strategic Interaction of Successive Motivational States Within the Person. New York, NY: Cambridge University Press
    1. Ainslie G. (2001). Breakdown of Will. New York, NY: Cambridge University Press
    1. Ainslie G., Monterosso J. R. (2003). Building blocks of self-control: increased tolerance for delay with bundled rewards. J. Exp. Anal. Behav. 79, 37–48 10.1901/jeab.2003.79-37 - DOI - PMC - PubMed
    1. Alexander W. H., Brown J. W. (2010). Hyperbolically discounted temporal difference learning. Neural. Comput. 22, 1511–1527 10.1162/neco.2010.08-09-1080 - DOI - PMC - PubMed

LinkOut - more resources