Devaluation of outcomes due to their cost: Extending discounting models beyond delay

Impulsivity is a profound source of poor decision making, often bringing suffering to both person and polity. Although impulsivity attends psychiatric disorders such as addiction, pathological gambling, attention deficit/hyperactivity disorder, and obsessive-compulsive disorder, almost everyone makes impulsive decisions that disregard the long-term consequences of our actions in favor of the near-term allure of immediate temptations. Deliberating between long-term benefits and short-term rewards is also a hallmark of foraging decisions, probably the most fundamental of all challenges confronted by mobile organisms. Behavioral studies confirm theoretical predictions that foragers compute the value of current offers, track background reward rates over different temporal and spatial scales, and update strategies in response to changes in the environment. These observations suggest that the execution of foraging computations is fundamental for understanding the organization of the nervous system. Here, we describe a process model for making foraging choices that integrates the value of short-term options and compares that value to a decision threshold determined by long-term reward rates. In addition, the role of interrupts and optimization routines are here incorporated for the first time into a foraging framework, by adapting decision thresholds to changes in the environment. A core network of brain areas, including the ventromedial prefrontal cortex, the anterior cingulate cortex, and the posterior cingulate cortex, under the modulatory influence of dopamine and norepinephrine, executes these computations and implements these processes. Our model provocatively implies that maladaptive impulsive choices can result from dysregulated foraging neurocircuitry.