Reward prediction error and fMRI: do they go together?

New paper: Bissonette GB, Gentry RN, Padmala S, Pessoa L, Roesch MR. Impact of appetitive and aversive outcomes on brain responses: linking the animal and human literatures. Front Syst Neurosci. 2014 Mar 4;8:24. eCollection 2014.

With Matt Roesch and folks in his lab, we have a new paper reviewing the animal and animal literatures on appetitive and aversive processing. But what I want to discuss here is one issue that became apparent when we started collecting some pilot fMRI data. Whereas work in the rat has shown prediction errors in a few places in the brain, some fMRI work has shown quite widespread signals. This might be true but one possible confound that we find in many papers in the literature is that regions that simply respond to the US stimulus could be incorrectly described as showing prediction errors because of the way the fMRI analysis is done. This is explained in Fig. 6 of our paper and in the figure below (thanks to Brenton McMenamin).

The problem is that when a regressor modeling the US (say, reward) is also introduced in the model in addition to the prediction error regressor, it can absorb variance in a way that what is left is actually how a prediction error looks like. In that way, the region will appear to show a prediction error simply because it responds to reward itself.

I have no idea how prevalent this problem is, and it is at times even unclear how the data were modeled. (In fact, as an aside, it is at times very surprising how people don’t explain what they do in their papers.) Some people have talked about this problem but again I’m not sure how much people are taking this into account.

fig6_v2 (1)

Dopamine: reward or a lot more? We knew it was a lot more…

I recently took the time to read this paper, something that I should have done a while back… Bromberg-Martin, Matsumoto, and Hikosaka (2010) provide this great perspective on the multi-dimensional nature of dopamine neurons and signaling. I’m not going to summarize it, the authors have done a better job . It’s worth reading the whole quote (and paper) (pp. 827-828; emphasis added):

Scheme by Bromberg, Matsumoto, and Hikosaka.

Scheme by Bromberg-Martin, Matsumoto, and Hikosaka (2010).

“An influential concept of midbrain DA [dopamine] neurons has been that they transmit a uniform motivational signal to all downstream structures. Here we have reviewed evidence that DA signals are more diverse than commonly thought. Rather than encoding a uniform signal, DA neurons come in multiple types that send distinct motivational messages about rewarding and nonrewarding events. Even single DA neurons do not appear to transmit single motivational signals. Instead, DA neurons transmit mixtures of multiple signals generated by distinct neural processes. Some reflect detailed predictions about rewarding and aversive experiences, while others reflect fast responses to events of high potential importance…

Many previous theories have attempted to identify DA neurons with a single motivational process such as seeking valued goals, engaging motivationally salient situations, or reacting to alerting changes in the environment. In our view, DA neurons receive signals related to all three of these processes. Yet rather than distilling these signals into a uniform message, we have proposed that DA neurons transmit these signals to distinct brain structures in order to support distinct neural systems for motivated cognition and behavior. Some DA neurons support brain systems that assign motivational value, promoting actions to seek rewarding events, avoid aversive events, and ensure that alerting events can be predicted and prepared for in advance. Other DA neurons support brain systems that are engaged by motivational salience, including orienting to detect potentially important events, cognitive processing to choose a response and to remember its consequences, and motivation to persist in pursuit of an optimal outcome. We hope that this proposal helps lead us to a more refined understanding of DA functions in the brain, in which DA neurons tailor their signals to support multiple neural networks with distinct roles in motivational control.”

Fantastic! Above is a picture of their scheme (Fig. 7).

Reference: Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010). Dopamine in motivational control: rewarding, aversive, and alerting. Neuron, 68(5), 815-834.