Models of neuromodulation for computational psychiatry

Advanced Review

Sandra Iglesias, Sara Tomiello, Maya Schneebeli, Klaas E. Stephan

Published Online: Sep 21 2016

DOI: 10.1002/wcs.1420

Full article on Wiley Online Library: HTML PDF

Can't access this content? Tell your librarian.

Psychiatry faces fundamental challenges: based on a syndrome‐based nosology, it presently lacks clinical tests to infer on disease processes that cause symptoms of individual patients and must resort to trial‐and‐error treatment strategies. These challenges have fueled the recent emergence of a novel field—computational psychiatry—that strives for mathematical models of disease processes at physiological and computational (information processing) levels. This review is motivated by one particular goal of computational psychiatry: the development of ‘computational assays’ that can be applied to behavioral or neuroimaging data from individual patients and support differential diagnosis and guiding patient‐specific treatment. Because the majority of available pharmacotherapeutic approaches in psychiatry target neuromodulatory transmitters, models that infer (patho)physiological and (patho)computational actions of different neuromodulatory transmitters are of central interest for computational psychiatry. This article reviews the (many) outstanding questions on the computational roles of neuromodulators (dopamine, acetylcholine, serotonin, and noradrenaline), outlines available evidence, and discusses promises and pitfalls in translating these findings to clinical applications. WIREs Cogn Sci 2017, 8:e1420. doi: 10.1002/wcs.1420 This article is categorized under: Psychology > Brain Function and Dysfunction Neuroscience > Computation Neuroscience > Plasticity

Model‐based functional magnetic resonance imaging (fMRI) analysis. A schematic demonstration of the steps involved in a model‐based fMRI analysis. (1) Fitting computational models to behavioral data provides a trajectory of trial‐wise (k) estimates of computational quantities. Here, we consider the example of a precision‐weighted outcome prediction error (PE; μ₂) from the hierarchical Gaussian filter (HGF). This consists of a precision weight (green) and the PE (purple). (2) Trial‐by‐trial trajectories of these quantities are (3) entered into a general linear model (GLM) as parametric modulators (third and fifth columns), modeling a specific time window in the trial (e.g., the outcome phase). All regressors are convolved with the canonical hemodynamic response function and its temporal derivative. (4) Voxel‐wise contrasts applied to the parametric modulators result in statistical parametric maps, which, following correction for multiple comparisons, reveal significant relations between the blood oxygen level‐dependent (BOLD) signal in a particular region and the computational quantity of interest. SFG: superior frontal gyrus.

[ Normal View | Magnified View ]

Modulatory neurotransmitter systems. An illustrative representation of the (1) dopaminergic, (2) cholinergic, (3) serotonergic, and (4) noradrenergic systems.

[ Normal View | Magnified View ]

Computational psychiatry. One key goal of computational psychiatry is to develop (1) computational assays, (2) apply them to neuroimaging data from individual patients to infer disease‐relevant mechanisms which allow for (3) detection of subgroups in order to (4) improve individual treatment prediction. (Reprinted with permission from Ref . Copyright 2015 Elsevier; Panel 1: Reprinted with permission from Ref . Copyright 2009 Elsevier)

[ Normal View | Magnified View ]

Computational quantities of neuromodulation. This figure summarizes results from previous model‐based functional magnetic resonance imaging (fMRI) studies of neuromodulation. The following studies reported correlations between BOLD signal in the human midbrain and (1) reward prediction errors (PEs; Reprinted with permission from Ref . Copyright 2008 Nature Publishing Group), (2) reward PEs (Reprinted with permission from Ref . Copyright 2011 Elsevier), (3) sensory PEs (Reprinted with permission from Ref . Copyright 2013 Elsevier), (4) precision (Reprinted with permission from Ref . Copyright 2015 Oxford University Press). Additionally, (5) expected uncertainty (specifically: precision‐weighted PE about outcome probability) correlated with BOLD signal in the cholinergic septum (Reprinted with permission from Ref . Copyright 2013 Elsevier), (6) whereas unexpected uncertainty showed an inverse correlation with BOLD activity in the noradrenergic locus coeruleus (Reprinted with permission from Ref . Copyright 2013 Elsevier)

[ Normal View | Magnified View ]

References

Krystal, JH, State, MW. Psychiatric disorders: diagnosis to therapy. Cell 2014, 157:201–214.
Owen, MJ. New approaches to psychiatric diagnostic classification. Neuron 2014, 84:564–571.
Stephan, KE, Bach, DR, Fletcher, PC, Flint, J, Frank, MJ, Friston, KJ, Heinz, A, Huys, QJM, Owen, MJ, Binder, EB, et al. Charting the landscape of priority problems in psychiatry, part 1: classification and diagnosis. Lancet Psychiatry 2015, 3:77–83.
Stephan, KE, Binder, EB, Breakspear, M, Dayan, P, Johnstone, EC, Meyer‐Lindenberg, A, Schnyder, U, Wang, X‐J, Bach, DR, Fletcher, PC, et al. Charting the landscape of priority problems in psychiatry, part 2: pathogenesis and aetiology. Lancet Psychiatry 2015, 3:84–90.
Kapur, S, Phillips, AG, Insel, TR. Why has it taken so long for biological psychiatry to develop clinical tests and what to do about it? Mol Psychiatry 2012, 17:1174–1179.
Rush, AJ, Trivedi, MH, Wisniewski, SR, Nierenberg, AA, Stewart, JW, Warden, D, Niederehe, G, Thase, ME, Lavori, PW, Lebowitz, BD, et al. Acute and longer‐term outcomes in depressed outpatients requiring one or several treatment steps: a STAR*D report. Am J Psychiatry 2006, 163:1905–1917.
Hyman, SE. Revolution stalled. Sci Transl Med 2012, 4:155cm11.
Wang, X‐J, Krystal, JH. Computational psychiatry. Neuron 2014, 84:638–654.
Adams, RA, Huys, QJM, Roiser, JP. Computational psychiatry: towards a mathematically informed understanding of mental illness. J Neurol Neurosurg Psychiatry 2016, 87:53–63.
Friston, KJ, Stephan, KE, Montague, R, Dolan, RJ. Computational psychiatry: the brain as a phantastic organ. Lancet Psychiatry 2014, 1:148–158.
Huys, QJM, Maia, T, Frank, MJ. Computational psychiatry as a bridge from neuroscience to clinical applications. Nat Neurosci 2016, 19:404–413.
Maia, T, Frank, MJ. From reinforcement learning models to psychiatric and neurological disorders. Nat Neurosci 2011, 14:154–162.
Montague, PR, Dolan, RJ, Friston, KJ, Dayan, P. Computational psychiatry. Trends Cogn Sci 2012, 16:72–80.
Stephan, KE, Iglesias, S, Heinzle, J, Diaconescu, AO. Translational perspectives for computational neuroimaging. Neuron 2015, 87:716–732.
Stephan, KE, Mathys, C. Computational approaches to psychiatry. Curr Opin Neurobiol 2014, 25:85–92.
Wiecki, TV, Poland, J, Frank, MJ. Model‐based cognitive neuroscience approaches to computational psychiatry clustering and classification. Clin Psychol Sci 2015, 3:378–399.
Chen, CC, Henson, RN, Stephan, KE, Kilner, JM, Friston, KJ. Forward and backward connections in the brain: a DCM study of functional asymmetries. Neuroimage 2009, 45:453–462.
Karg, K, Burmeister, M, Shedden, K, Sen, S. The serotonin transporter promoter variant (5‐HTTLPR), stress, and depression meta‐analysis revisited: evidence of genetic moderation. Arch Gen Psychiatry 2011, 68:444–454.
Biederman, J, Spencer, T. Attention‐deficit/hyperactivity disorder (ADHD) as a noradrenergic disorder. Biol Psychiatry 1999, 46:1234–1242.
Howes, OD, Kapur, S. The dopamine hypothesis of schizophrenia: version III—the final common pathway. Schizophr Bull 2009, 35:549–562.
Raedler, TJ, Bymaster, FP, Tandon, R, Copolov, D, Dean, B. Towards a muscarinic hypothesis of schizophrenia. Mol Psychiatry 2007, 12:232–246.
Stephan, KE, Baldeweg, T, Friston, KJ. Synaptic plasticity and dysconnection in schizophrenia. Biol Psychiatry 2006, 59:929–939.
Gu, Q. Neuromodulatory transmitter systems in the cortex and their role in cortical plasticity. Neuroscience 2002, 111:815–835.
von Bohlen und Halbach, O, Dermietzel, R. Neurotransmitters and Neuromodulators: Handbook of Receptors and Biological Effects. Weinheim, Germany: Wiley; 2006.
McCormick, DA, Wang, Z, Huguenard, J. Neurotransmitter control of neocortical neuronal activity and excitability. Cereb Cortex 1993, 3:387–398.
Faber, ES, Sah, P. Calcium‐activated potassium channels: multiple contributions to neuronal function. Neuroscientist 2003, 9:181–194.
Martin, KA, Spühler, IA. The fine structure of the dopaminergic innervation of area 10 of macaque prefrontal cortex. Eur J Neurosci 2013, 37:1061–1071.
Sara, SJ. The locus coeruleus and noradrenergic modulation of cognition. Nat Rev Neurosci 2009, 10:211–223.
Taber, KH, Hurley, RA. Volume transmission in the brain: beyond the synapse. J Neuropsychiatry Clin Neurosci 2014, 26:iv, 1–4.
Friston, KJ, Tononi, G, Reeke, GN Jr, Sporns, O, Edelman, GM. Value‐dependent selection in the brain: simulation in a synthetic neural model. Neuroscience 1994, 59:229–243.
Montague, PR, Hyman, SE, Cohen, JD. Computational roles for dopamine in behavioural control. Nature 2004, 431:760–767.
Marder, E. Neuromodulation of neuronal circuits: back to the future. Neuron 2012, 76:1–11.
Doya, K. Modulators of decision making. Nat Neurosci 2008, 11:410–416.
Dayan, P. Twenty‐five lessons from computational neuromodulation. Neuron 2012, 76:240–256.
Schultz, W, Dayan, P, Montague, P. A neural substrate of prediction and reward. Science 1997, 275:1593.
D`Ardenne, K, McClure, SM, Nystrom, LE, Cohen, JD. BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science 2008, 319:1264–1267.
Klein‐Flügge, MC, Hunt, LT, Bach, DR, Dolan, RJ, Behrens, TE. Dissociable reward and timing signals in human midbrain and ventral striatum. Neuron 2011, 72:654–664.
Pessiglione, M, Seymour, B, Flandin, G, Dolan, RJ, Frith, CD. Dopamine‐dependent prediction errors underpin reward‐seeking behaviour in humans. Nature 2006, 442:1042–1045.
O`Doherty, JP, Dayan, P, Friston, K, Critchley, H, Dolan, RJ. Temporal difference models and reward‐related learning in the human brain. Neuron 2003, 38:329–337.
Gläscher, J, O`Doherty, J. Model based approaches to neuroimaging: combining reinforcement learning theory with fMRI data. WIREs Cogn Sci 2010, 1:501–510.
Stephan, KE, Schlagenhauf, F, Huys, QJM, Raman, S, Aponte, EA, Brodersen, KH, Rigoux, L, Moran, RJ, Daunizeau, J, Dolan, RJ, et al. Computational neuroimaging strategies for single patient predictions. Neuroimage 2016. Available at: http://dx.doi.org/10.1016/j.neuroimage.2016.06.038.
Ludvig, EA, Bellemare, MG, Pearson, KG. A primer on reinforcement learning in the brain: psychological, computational, and neural perspectives. AlonsoE,, MondragónE, (eds). In: Alonso E, Mondragón E, eds.Computational Neuroscience for Advancing Artificial Intelligence: Models, Methods and Applications. Hershey, PA: IGI Global; 2011, 111–144.
Gradin, VB, Kumar, P, Waiter, G, Ahearn, T, Stickle, C, Milders, M, Reid, I, Hall, J, Steele, JD. Expected value and prediction error abnormalities in depression and schizophrenia. Brain 2011, 134:1751–1764.
Rescorla, RA, Wagner, AW. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In: Black, AH, Prokasy, WF, eds. Classical Conditioning II: Current Research and Theory. Appleton Century Crofts, NY: Appleton‐Century‐Crofts; 1972, 64–99.
Schultz, W. Predictive reward signal of dopamine neurons. J Neurophysiol 1998, 80:1–27.
Mathys, C, Daunizeau, J, Friston, KJ, Stephan, KE. A bayesian foundation for individual learning under uncertainty. Front Hum Neurosci 2011, 5:39.
Friston, KJ, Harrison, L, Penny, W. Dynamic causal modelling. Neuroimage 2003, 19:1273–1302.
Stephan, KE, Penny, WD, Moran, RJ, den Ouden, HE, Daunizeau, J, Friston, KJ. Ten simple rules for dynamic causal modeling. Neuroimage 2010, 49:3099–3109.
den Ouden, HE, Daunizeau, J, Roiser, J, Friston, KJ, Stephan, KE. Striatal prediction error modulates cortical coupling. J Neurosci 2010, 30:3210–3219.
Roy, M, Shohamy, D, Daw, N, Jepma, M, Wimmer, GE, Wager, TD. Representation of aversive prediction errors in the human periaqueductal gray. Nat Neurosci 2014, 17:1607–1612.
Moran, RJ, Jung, F, Kumagai, T, Endepols, H, Graf, R, Dolan, RJ, Friston, KJ, Stephan, KE, Tittgemeyer, M. Dynamic causal models and physiological inference: a validation study using isoflurane anaesthesia in rodents. PLoS One 2011, 6:e22790.
Moran, RJ, Symmonds, M, Stephan, KE, Friston, KJ, Dolan, RJ. An in vivo assay of synaptic function mediating human cognition. Curr Biol 2011, 21:1320–1325.
Beaulieu, J‐M, Gainetdinov, RR. The physiology, signaling, and pharmacology of dopamine receptors. Pharmacol Rev 2011, 63:182–217.
Tritsch, NX, Sabatini, BL. Dopaminergic modulation of synaptic transmission in cortex and striatum. Neuron 2012, 76:33–50.
Bentivoglio, M, Morelli, M. Chapter I: The organization and circuits of mesencephalic dopaminergic neurons and the distribution of dopamine receptors in the brain. In: Dunnett, SB, Bentivoglio, M, Björklund, A, Hökfelt, T, eds. Handbook of Chemical Neuroanatomy, vol. 21. Elsevier; 2005, 1–107.
Floresco, SB, West, AR, Ash, B, Moore, H, Grace, AA. Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission. Nat Neurosci 2003, 6:968–973.
Sutton, RS, Barto, AG. Reinforcement Learning: An Introduction, vol. 1. Cambridge, MA: Cambridge University Press; 1998.
Bayer, HM, Glimcher, PW. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 2005, 47:129–141.
Matsumoto, M, Takada, M. Distinct representations of cognitive and motivational signals in midbrain dopamine neurons. Neuron 2013, 79:1011–1024.
Nomoto, K, Schultz, W, Watanabe, T, Sakagami, M. Temporally extended dopamine responses to perceptually demanding reward‐predictive stimuli. J Neurosci 2010, 30:10692–10702.
Waelti, P, Dickinson, A, Schultz, W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 2001, 412:43–48.
Bellebaum, C, Jokisch, D, Gizewski, ER, Forsting, M, Daum, I. The neural coding of expected and unexpected monetary performance outcomes: dissociations between active and observational learning. Behav Brain Res 2012, 227:241–251.
Schlagenhauf, F, Rapp, MA, Huys, QJM, Beck, A, Wüstenberg, T, Deserno, L, Buchholz, H‐G, Kalbitzer, J, Buchert, R, Bauer, M, et al. Ventral striatal prediction error signaling is associated with dopamine synthesis capacity and fluid intelligence. Hum Brain Mapp 2013, 34:1490–1499.
Valentin, VV, O`Doherty, JP. Overlapping prediction errors in dorsal striatum during instrumental learning with juice and money reward in the human brain. J Neurophysiol 2009, 102:3384–3391.
Jocham, G, Klein, TA, Ullsperger, M. Dopamine‐mediated reinforcement learning signals in the striatum and ventromedial prefrontal cortex underlie value‐based choices. J Neurosci 2011, 31:1606–1613.
Stephan, KE, Friston, KJ, Frith, CD. Dysconnection in schizophrenia: from abnormal synaptic plasticity to failures of self‐monitoring. Schizophr Bull 2009, 35:509–527.
Schultz, W, Dickinson, A. Neuronal coding of prediction errors. Annu Rev Neurosci 2000, 23:473–500.
Redish, AD. Addiction as a computational process gone awry. Science 2004, 306:1944–1947.
Kumar, P, Waiter, G, Ahearn, T, Milders, M, Reid, I, Steele, JD. Abnormal temporal difference reward‐learning signals in major depression. Brain 2008, 131:2084–2093.
Corlett, P, Taylor, J, Wang, XJ, Fletcher, P, Krystal, J. Toward a neurobiology of delusions. Prog Neurobiol 2010, 92:345–369.
Kapur, S. Psychosis as a state of aberrant salience: a framework linking biology, phenomenology, and pharmacology in schizophrenia. Am J Psychiatry 2003, 160:13–23.
Heinz, A, Schlagenhauf, F. Dopaminergic dysfunction in schizophrenia: salience attribution revisited. Schizophr Bull 2010, 36:472–485.
Murray, GK, Corlett, PR, Clark, L, Pessiglione, M, Blackwell, AD, Honey, G, Jones, PB, Bullmore, ET, Robbins, TW, Fletcher, PC. Substantia nigra/ventral tegmental reward prediction error disruption in psychosis. Mol Psychiatry 2008, 13:239, 267–276.
Romaniuk, L, Honey, GD, King, JR, Whalley, HC, McIntosh, AM, Levita, L, Hughes, M, Johnstone, EC, Day, M, Lawrie, SM, et al. Midbrain activation during Pavlovian conditioning and delusional symptoms in schizophrenia. Arch Gen Psychiatry 2010, 67:1246–1254.
Roiser, JP, Howes, OD, Chaddock, CA, Joyce, EM, McGuire, P. Neural and behavioral correlates of aberrant salience in individuals at risk for psychosis. Schizophr Bull 2013, 39:1328–1336.
Deserno, L, Boehme, R, Heinz, A, Schlagenhauf, F. Reinforcement learning and dopamine in schizophrenia: dimensions of symptoms or specific features of a disease group? Front Psychiatry 2013, 4:172.
Schlagenhauf, F, Huys, QJM, Deserno, L, Rapp, MA, Beck, A, Heinze, H‐J, Dolan, R, Heinz, A. Striatal dysfunction during reversal learning in unmedicated schizophrenia patients. Neuroimage 2014, 89:171–180.
Boehme, R, Deserno, L, Gleich, T, Katthagen, T, Pankow, A, Behr, J, Buchert, R, Roiser, JP, Heinz, A, Schlagenhauf, F. Aberrant salience is related to reduced reinforcement learning signals and elevated dopamine synthesis capacity in healthy adults. J Neurosci 2015, 35:10103–10111.
Roiser, JP, Stephan, KE, den Ouden, HE, Barnes, TR, Friston, KJ, Joyce, EM. Do patients with schizophrenia exhibit aberrant salience? Psychol Med 2009, 39:199–209.
Hiroyuki, N. Multiplexing signals in reinforcement learning with internal models and dopamine. Curr Opin Neurobiol 2014, 25:123–129.
Schultz, W. Updating dopamine reward signals. Curr Opin Neurobiol 2012, 23:229–238.
Roeper, J. Dissecting the diversity of midbrain dopamine neurons. Trends Neurosci 2013, 36:336–342.
Bunzeck, N, Düzel, E. Absolute coding of stimulus novelty in the human substantia nigra/VTA. Neuron 2006, 51:369–379.
Lisman, JE, Grace, AA. The hippocampal‐VTA loop: controlling the entry of information into long‐term memory. Neuron 2005, 46:703–713.
Guitart‐Masip, M, Bunzeck, N, Stephan, KE, Dolan, RJ, Düzel, E. Contextual novelty changes reward representations in the striatum. J Neurosci 2010, 30:1721–1726.
Kakade, S, Dayan, P. Dopamine: generalization and bonuses. Neural Netw 2002, 15:549–559.
Costa, VD, Tran, VL, Turchi, J, Averbeck, BB. Dopamine modulates novelty seeking behavior during decision making. Behav Neurosci 2014, 128:556–566.
Kayser, AS, Mitchell, JM, Weinstein, D, Frank, MJ. Dopamine, locus of control, and the exploration‐exploitation tradeoff. Neuropsychopharmacology 2015, 40:454–462.
Friston, K, Rigoli, F, Ognibene, D, Mathys, C, Fitzgerald, T, Pezzulo, G. Active inference and epistemic value. Cogn Neurosci 2015, 6:187–214.
Fiorillo, CD, Tobler, PN, Schultz, W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 2003, 299:1898–1902.
Schultz, W, Preuschoff, K, Camerer, C, Hsu, M, Fiorillo, CD, Tobler, PN, Bossaerts, P. Explicit neural signals reflecting reward uncertainty. Philos Trans R Soc Lond B Biol Sci 2008, 363:3801–3811.
Niv, Y, Duff, MO, Dayan, P. Dopamine, uncertainty and TD learning. Behav Brain Funct 2005, 1:6.
de Lafuente, V, Romo, R. Dopamine neurons code subjective sensory experience and uncertainty of perceptual decisions. Proc Natl Acad Sci USA 2011, 108:19767–19771.
Hart, AS, Clark, JJ, Phillips, PE. Dynamic shaping of dopamine signals during probabilistic Pavlovian conditioning. Neurobiol Learn Mem 2015, 117:84–92.
Schwartenbeck, P, FitzGerald, TH, Mathys, C, Dolan, R, Friston, K. The dopaminergic midbrain encodes the expected certainty about desired outcomes. Cereb Cortex 2015, 25:3434–3445.
Tomassini, A, Ruge, D, Galea, JM, Penny, W, Bestmann, S. The role of dopamine in temporal uncertainty. J Cogn Neurosci 2016, 28:96–110.
Guggenmos, M, Wilbertz, G, Hebart, MN, Sterzer, P. Mesolimbic confidence signals guide perceptual learning in the absence of external feedback. eLife 2016, 5:e13388.
Friston, KJ, Shiner, T, FitzGerald, T, Galea, JM, Adams, R, Brown, H, Dolan, RJ, Moran, R, Stephan, KE, Bestmann, S. Dopamine, affordance and active inference. PLoS Comput Biol 2012, 8:e1002327.
Iglesias, S, Mathys, C, Brodersen, KH, Kasper, L, Piccirelli, M, den Ouden, HE, Stephan, KE. Hierarchical prediction errors in midbrain and basal forebrain during sensory learning. Neuron 2013, 80:519–530.
Payzan‐LeNestour, E, Dunne, S, Bossaerts, P, O`Doherty, JP. The neural representation of unexpected uncertainty during value‐based decision making. Neuron 2013, 79:191–201.
Preuschoff, K, Bossaerts, P. Adding prediction risk to the theory of reward learning. Ann N Y Acad Sci 2007, 1104:135–146.
Schwartenbeck, P, FitzGerald, TH, Dolan, R. Neural signals encoding shifts in beliefs. Neuroimage 2016, 125:578–586.
FitzGerald, TH, Dolan, RJ, Friston, K. Dopamine, reward learning, and active inference. Front Comput Neurosci 2015, 9:136.
Berridge, KC. From prediction error to incentive salience: mesolimbic computation of reward motivation. Eur J Neurosci 2012, 35:1124–1143.
Robbins, TW, Everitt, BJ. A role for mesencephalic dopamine in activation: commentary on Berridge (2006). Psychopharmacology (Berl) 2007, 191:433–437.
Robinson, S, Sandstrom, SM, Denenberg, VH, Palmiter, RD. Distinguishing whether dopamine regulates liking, wanting, and/or learning about rewards. Behav Neurosci 2005, 119:5–15.
Pellicano, E, Burr, D. When the world becomes `too real`: a Bayesian explanation of autistic perception. Trends Cogn Sci 2012, 16:504–510.
Lawson, RP, Rees, G, Friston, KJ. An aberrant precision account of autism. Front Hum Neurosci 2014, 8:302.
Adams, RA, Stephan, KE, Brown, HR, Frith, CD, Friston, KJ. The computational anatomy of psychosis. Front Psychiatry 2013, 4:47.
Kapur, S, Zipursky, R, Jones, C, Remington, G, Houle, S. Relationship between dopamine D(2) occupancy, clinical response, and side effects: a double‐blind PET study of first‐episode schizophrenia. Am J Psychiatry 2000, 157:514–520.
Haker, H, Schneebeli, M, Stephan, KE. Can Bayesian theories of autism spectrum disorder help improve clinical practice? Front Psychiatry 2016, 7:107.
Franklin, NT, Frank, MJ. A cholinergic feedback circuit to regulate striatal population uncertainty and optimize reinforcement learning. eLife 2015, 4:e12029.
Mesulam, MM, Mufson, EJ, Wainer, BH, Levey, AI. Central cholinergic pathways in the rat: an overview based on an alternative nomenclature (Ch1‐Ch6). Neuroscience 1983, 10:1185–1201.
Newman, EL, Gupta, K, Climer, JR, Monaghan, CK, Hasselmo, ME. Cholinergic modulation of cognitive processing: insights drawn from computational models. Front Behav Neurosci 2012, 6:24.
Sarter, M, Hasselmo, ME, Bruno, JP, Givens, B. Unraveling the attentional functions of cortical cholinergic inputs: interactions between signal‐driven and cognitive modulation of signal detection. Brain Res Brain Res Rev 2005, 48:98–111.
Feldman, H, Friston, KJ. Attention, uncertainty, and free‐energy. Front Hum Neurosci 2010, 4:215.
Yu, AJ, Dayan, P. Acetylcholine in cortical inference. Neural Netw 2002, 15:719–730.
Yu, AJ, Dayan, P. Uncertainty, neuromodulation, and attention. Neuron 2005, 46:681–692.
Vossel, S, Bauer, M, Mathys, C, Adams, RA, Dolan, RJ, Stephan, KE, Friston, KJ. Cholinergic stimulation enhances Bayesian belief updating in the deployment of spatial attention. J Neurosci 2014, 34:15735–15742.
de Kloet, SF, Mansvelder, HD, De Vries, TJ. Cholinergic modulation of dopamine pathways through nicotinic acetylcholine receptors. Biochem Pharmacol 2015, 97:425–438.
Mansvelder, HD, McGehee, DS. Cellular and synaptic mechanisms of nicotine addiction. J Neurobiol 2002, 53:606–617.
Tzschentke, TM. Pharmacology and behavioral pharmacology of the mesocortical dopamine system. Prog Neurobiol 2001, 63:241–320.
Yeomans, J, Forster, G, Blaha, C. M5 muscarinic receptors are needed for slow activation of dopamine neurons and for rewarding brain stimulation. Life Sci 2001, 68:2449–2456.
Okada, K, Nakamura, K, Kobayashi, Y. A neural correlate of predicted and actual reward‐value information in monkey pedunculopontine tegmental and dorsal raphe nucleus during saccade tasks. Neural Plast 2011, 2011:579840.
Kobayashi, Y, Okada, KI. Reward prediction error computation in the pedunculopontine tegmental nucleus neurons. Ann N Y Acad Sci 2007, 1104:310–323.
Gu, X, Lohrenz, T, Salas, R, Baldwin, PR, Soltani, A, Kirk, U, Cinciripini, PM, Montague, PR. Belief about nicotine selectively modulates value and reward prediction error signals in smokers. Proc Natl Acad Sci 2015, 112:2539–2544.
Bunzeck, N, Guitart‐Masip, M, Dolan, RJ, Duzel, E. Pharmacological dissociation of novelty responses in the human brain. Cereb Cortex 2013, 24:1351–1360.
Näätänen, R. The mismatch negativity: a powerful tool for cognitive neuroscience. Ear Hear 1995, 16:6–18.
Umbricht, D, Krljes, S. Mismatch negativity in schizophrenia: a meta‐analysis. Schizophr Res 2005, 76:1–23.
Erickson, MA, Ruffle, A, Gold, JM. A meta‐analysis of mismatch negativity in schizophrenia: from clinical risk to disease specificity and progression. Biol Psychiatry 2015, 79:980–987.
Garrido, MI, Kilner, JM, Stephan, KE, Friston, KJ. The mismatch negativity: a review of underlying mechanisms. Clin Neurophysiol 2009, 120:453–463.
Moran, RJ, Campo, P, Symmonds, M, Stephan, KE, Dolan, RJ, Friston, KJ. Free energy, precision and learning: the role of cholinergic neuromodulation. J Neurosci 2013, 33:8227–8236.
Friston, KJ. Hallucinations and perceptual inference. Behav Brain Sci 2005, 28:764–766.
Smythies, JR. The role of acetylcholine in hallucinatory perception. Behav Brain Sci 2005, 28:773.
Scarr, E, Cowie, TF, Kanellakis, S, Sundram, S, Pantelis, C, Dean, B. Decreased cortical muscarinic receptors define a subgroup of subjects with schizophrenia. Mol Psychiatry 2009, 14:1017–1023.
Raedler, TJ, Knable, MB, Jones, DW, Urbina, RA, Gorey, JG, Lee, KS, Egan, MF, Coppola, R, Weinberger, DR. In vivo determination of muscarinic acetylcholine receptor availability in schizophrenia. Am J Psychiatry 2003, 160:118–127.
Reichert, DP, Series, P, Storkey, AJ. Charles Bonnet syndrome: evidence for a generative model in the cortex? PLoS Comput Biol 2013, 9:e1003134.
Mohammad‐Zadeh, LF, Moses, L, Gwaltney‐Brant, SM. Serotonin: a review. J Vet Pharmacol Ther 2008, 31:187–199.
Andrews, PW, Bharwani, A, Lee, KR, Fox, M, Thomson, JA Jr. Is serotonin an upper or a downer? The evolution of the serotonergic system and its role in depression and the antidepressant response. Neurosci Biobehav Rev 2015, 51:164–188.
Nakamura, K. The role of the dorsal raphe nucleus in reward‐seeking behavior. Front Integr Neurosci 2013, 7:60.
Smythies, J, Section, V. Serotonin system. Int Rev Neurobiol 2005, 64:217–268.
Berger, M, Gray, JA, Roth, BL. The expanded biology of serotonin. Annu Rev Med 2009, 60:355–366.
Liu, Z, Zhou, J, Li, Y, Hu, F, Lu, Y, Ma, M, Feng, Q, J‐e, Z, Wang, D, Zeng, J, et al. Dorsal raphe neurons signal reward through 5‐HT and glutamate. Neuron 2014, 81:1360–1374.
Cohen, JY, Amoroso, MW, Uchida, N. Serotonergic neurons signal reward and punishment on multiple timescales. eLife 2015, 4:e06346.
Celada, P, Puig, MV, Artigas, F. Serotonin modulation of cortical neurons and networks. Front Integr Neurosci 2013, 7:25.
Puig, MV, Gulledge, AT. Serotonin and prefrontal cortex function: neurons, networks, and circuits. Mol Neurobiol 2011, 44:449–464.
Dayan, P, Huys, QJM. Serotonin, inhibition, and negative mood. PLoS Comput Biol 2008, 4:e4.
Huys, QJM, Eshel, N, O`Nions, E, Sheridan, L, Dayan, P, Roiser, JP. Bonsai trees in your head: how the pavlovian system sculpts goal‐directed choices by pruning decision trees. PLoS Comput Biol 2012, 8:e1002410.
Dalley, JW, Roiser, JP. Dopamine, serotonin and impulsivity. Neuroscience 2012, 215:42–58.
Robbins, TW, Gillan, CM, Smith, DG, de Wit, S, Ersche, KD. Neurocognitive endophenotypes of impulsivity and compulsivity: towards dimensional psychiatry. Trends Cogn Sci 2012, 16:81–91.
Doya, K. Metalearning and neuromodulation. Neural Netw 2002, 15:495–506.
Schweighofer, N, Bertin, M, Shishida, K, Okamoto, Y, Tanaka, SC, Yamawaki, S, Doya, K. Low‐serotonin levels increase delayed reward discounting in humans. J Neurosci 2008, 28:4528–4532.
Crockett, MJ, Clark, L, Lieberman, MD, Tabibnia, G, Robbins, TW. Impulsive choice and altruistic punishment are correlated and increase in tandem with serotonin depletion. Emotion 2010, 10:855–862.
Tanaka, SC, Schweighofer, N, Asahi, S, Shishida, K, Okamoto, Y, Yamawaki, S, Doya, K. Serotonin differentially regulates short‐ and long‐term prediction of rewards in the ventral and dorsal striatum. PLoS One 2007, 2:e1333.
Worbe, Y, Savulich, G, Voon, V, Fernandez‐Egea, E, Robbins, TW. Serotonin depletion induces ‘waiting impulsivity’ on the human four‐choice serial reaction time task: cross‐species translational significance. Neuropsychopharmacology 2014, 39:1519–1526.
Cools, R, Nakamura, K, Daw, ND. Serotonin and dopamine: unifying affective, activational, and decision functions. Neuropsychopharmacology 2011, 36:98–113.
Daw, ND, Kakade, S, Dayan, P. Opponent interactions between serotonin and dopamine. Neural Netw 2002, 15:603–616.
Seymour, B, Daw, ND, Roiser, JP, Dayan, P, Dolan, R. Serotonin selectively modulates reward value in human decision‐making. J Neurosci 2012, 32:5833–5842.
Hindi Attar, C, Finckh, B, Büchel, C. The influence of serotonin on fear learning. PLoS One 2012, 7:e42397.
Rygula, R, Clarke, HF, Cardinal, RN, Cockcroft, GJ, Xia, J, Dalley, JW, Robbins, TW, Roberts, AC. Role of central serotonin in anticipation of rewarding and punishing outcomes: effects of selective amygdala or orbitofrontal 5‐HT depletion. Cereb Cortex 2015, 25:3064–3076.
Guitart‐Masip, M, Economides, M, Huys, QJM, Frank, MJ, Chowdhury, R, Duzel, E, Dayan, P, Dolan, RJ. Differential, but not opponent, effects of L‐DOPA and citalopram on action learning with reward and punishment. Psychopharmacology (Berl) 2014, 231:955–966.
Schweimer, JV, Ungless, MA. Phasic responses in dorsal raphe serotonin neurons to noxious stimuli. Neuroscience 2010, 171:1209–1215.
Ranade, SP, Mainen, ZF. Transient firing of dorsal raphe neurons encodes diverse and specific sensory, motor, and reward events. J Neurophysiol 2009, 102:3026–3037.
Nakamura, K, Matsumoto, M, Hikosaka, O. Reward‐dependent modulation of neuronal activity in the primate dorsal raphe nucleus. J Neurosci 2008, 28:5331–5343.
Niv, Y, Daw, ND, Joel, D, Dayan, P. Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl) 2007, 191:507–520.
Dayan, P, Huys, QJM. Serotonin`s many meanings elude simple theories. eLife 2015, 4:e07390.
Stanford, SC. Adrenaline and noradrenaline: introduction. In: eLS. Chichester: John Wiley %26 Sons; 2001.
Berridge, CW, Waterhouse, BD. The locus coeruleus‐noradrenergic system: modulation of behavioral state and state‐dependent cognitive processes. Brain Res Brain Res Rev 2003, 42:33–84.
Aston‐Jones, G, Cohen, JD. An integrative theory of locus coeruleus‐norepinephrine function: adaptive gain and optimal performance. Annu Rev Neurosci 2005, 28:403–450.
Eckhoff, P, Wong‐Lin, K, Holmes, P. Optimality and robustness of a biophysical decision‐making model under norepinephrine modulation. J Neurosci 2009, 29:4301–4311.
Bouret, S, Sara, SJ. Network reset: a simplified overarching theory of locus coeruleus noradrenaline function. Trends Neurosci 2005, 28:574–582.
Dayan, P, Yu, AJ. Phasic norepinephrine: a neural interrupt signal for unexpected events. Network 2006, 17:335–350.
de Berker, AO, Rutledge, RB, Mathys, C, Marshall, L, Cross, GF, Dolan, RJ, Bestmann, S. Computations of uncertainty mediate acute stress responses in humans. Nat Commun 2016, 7:10996.
Joshi, S, Li, Y, Kalwani, RM, Gold, JI. Relationships between pupil diameter and neuronal activity in the locus coeruleus, colliculi, and cingulate cortex. Neuron 2016, 89:221–234.
Browning, M, Behrens, TE, Jocham, G, O`Reilly, JX, Bishop, SJ. Anxious individuals have difficulty learning the causal statistics of aversive environments. Nat Neurosci 2015, 18:590–596.
Behrens, TE, Woolrich, MW, Walton, ME, Rushworth, MF. Learning the value of information in an uncertain world. Nat Neurosci 2007, 10:1214–1221.
David, O, Guillemain, I, Saillet, S, Reyt, S, Deransart, C, Segebarth, C, Depaulis, A. Identifying neural drivers with functional MRI: an electrophysiological validation. PLoS Biol 2008, 6:e315.
Gilbert, JR, Symmonds, M, Hanna, MG, Dolan, RJ, Friston, KJ, Moran, RJ. Profiling neuronal ion channelopathies with non‐invasive brain imaging and dynamic causal models: case studies of single gene mutations. Neuroimage 2016, 124:43–53.
Brodersen, KH, Deserno, L, Schlagenhauf, F, Lin, Z, Penny, WD, Buhmann, JM, Stephan, KE. Dissecting psychiatric spectrum disorders by generative embedding. Neuroimage Clin 2014, 4:98–111.
Corlett, PR, Fletcher, PC. Computational psychiatry: a Rosetta Stone linking the brain to mental illness. Lancet Psychiatry 2014, 1:399–402.
den Ouden, HE, Daw, ND, Fernandez, G, Elshout, JA, Rijpkema, M, Hoogman, M, Franke, B, Cools, R. Dissociable effects of dopamine and serotonin on reversal learning. Neuron 2013, 80:1090–1100.
Düzel, E, Bunzeck, N, Guitart‐Masip, M, Wittmann, B, Schott, BH, Tobler, PN. Functional imaging of the human dopaminergic midbrain. Trends Neurosci 2009, 32:321–328.
Cragg, SJ. Meaningful silences: how dopamine listens to the ACh pause. Trends Neurosci 2006, 29:125–131.
Threlfell, S, Cragg, SJ. Dopamine signaling in dorsal versus ventral striatum: the dynamic role of cholinergic interneurons. Front Syst Neurosci 2011, 5:11.
Mansvelder, HD, McGehee, DS. Long‐term potentiation of excitatory inputs to brain reward areas by nicotine. Neuron 2000, 27:349–357.
Geisler, S, Derst, C, Veh, RW, Zahm, DS. Glutamatergic afferents of the ventral tegmental area in the rat. J Neurosci 2007, 27:5730–5743.
El Mestikawy, S, Wallén‐Mackenzie, Å, Fortin, GM, Descarries, L, Trudeau, LE. From glutamate co‐release to vesicular synergy: vesicular glutamate transporters. Nat Rev Neurosci 2011, 12:204–216.
Devoto, P, Flore, G. On the origin of cortical dopamine: is it a co‐transmitter in noradrenergic neurons? Curr Neuropharmacol 2006, 4:115–125.
Saunders, A, Granger, AJ, Sabatini, BL. Corelease of acetylcholine and GABA from cholinergic forebrain neurons. Elife 2015, 4:e06412.
Tritsch, NX, Oh, WJ, Gu, C, Sabatini, BL. Midbrain dopamine neurons sustain inhibitory transmission using plasma membrane uptake of GABA, not synthesis. Elife 2014, 3:e01936.
Trudeau, LE, Hnasko, TS, Wallén‐Mackenzie, Å, Morales, M, Rayport, S, Sulzer, D. The multilingual nature of dopamine neurons. Prog Brain Res 2014, 211:141–164.
Andrade, R, Haj‐Dahmane, S. Serotonin neuron diversity in the dorsal raphe. ACS Chem Neurosci 2013, 4:22–25.
Pine, A, Shiner, T, Seymour, B, Dolan, RJ. Dopamine, time, and impulsivity in humans. J Neurosci 2010, 30:8888–8896.

Access to this WIREs title is by subscription only.

Recommend to Your
Librarian Now!

Models of neuromodulation for computational psychiatry

Related Articles

Browse by Topic

Access to this WIREs title is by subscription only.

The latest WIREs articles in your inbox