Subjects should make choices by engaging in a form of planning to assess the expected long-run utility of possible actions based on a characterization of the current circumstance and then choose accordingly (note the term “circumstance” is used to refer to the detailed aspects of the current and past sensory environment that suffice to determine as best as possible the future effects of the subject’s current choice). However, uncertainty permeates both the determination of the current circumstance, for instance because of sensory noise, and the evaluation of the utility of actions, for instance because of ignorance stemming from incomplete

learning. As we will see, multiple, partially independent, systems Enzalutamide concentration are involved in the overall processes of choice and are thus tied up with utility and uncertainty, and all the systems are influenced by neuromodulators. Our restriction

to decision making leads to a concentration on the four major ascending neuromodulators: acetylcholine (ACh), dopamine (DA), norepinephrine (NE), and serotonin (5-HT). Even just for these four, there is not the space to discuss many of their operations or to provide the mathematical details of the models that underlie the analysis (as described in detail in the cited papers). The focus will be on data from rodents and primates, although click here there is substantial commonality of neuromodulator effects (if not

always their identities) in invertebrates (Katz, 2011). This analysis is influenced by Doya (2002) and the contributions in Doya et al. (2002). It is important to note that almost none of the computationally richer cases discussed is yet universally accepted. Utility or affective value is a central piece of information that influences behavior. In Parvulin terms of reinforcement learning (RL; Sutton and Barto, 1998), predictions about future values are made based on the current circumstance to determine choice and action; and, at least when disconfirmed, command learning. Utility should be influenced by aspects of a subject’s motivational state—the prospect of food is more valuable to a hungry than a thirsty animal. When choices can (perhaps also) avoid punishments, it is net utility that counts—it may not be worth stopping to collect either outcome in the face of mortal threat. Utility also plays roles other than determining the suitability of discrete choices. For instance, one can argue (Niv et al., 2007) that the average rate of (positive) utility quantifies the effective cost of the passage of time, in that the larger the expected rate, the more costly it is to deny oneself that much utility through failing to act for a given length of time. This can energize behavior (Guitart-Masip et al., 2011).