Reinforcement history, unconscious-decision making, and learning from experience
Ciara Sanchez QC ’15 and Jim Stellar
Ciara was a student in a class I taught last year. We had many follow-up conversations about the neuroscience of unconscious decision-making, operant or behavioral psychology, and one’s individual history with repeated reward experiences. I told her about the work of Wolfram Schultz in a phenomenon from neuroscience of called reward prediction error. Briefly, the idea is that dopamine reward-detecting neurons in the ventral tegmental area will reduce their firing to a repeated reward stimulus, but increase their firing to a stimulus that predicts the occurrence of a reward (e.g. the bell in the famous Pavlov’s dog experiment). If the expected reward does not occur, these neurons show decreased firing rates below baseline. It looks like our brains are unconsciously changing our reaction to repeated rewards, but then how does this alter conscious thinking?
In all our conscious thinking, there is always something just behind it. How do you know a person is nice when you first meet them? How does something stand out more than others? Like Gambling, you know it’s more of a loss then a win. But then there is that urge that know you will win this time. You can trust this new friend, he or she is a great person. All this thinking comes because of our ability to predict rewards and it is the history of reinforcement that is creating this urge we can’t explain.
Well put. These processes communicate with our conscious, rational, verbal processes by generating feelings.
Let’s introduce a famous paper by Solomon and Corbit (1974), the opponent process model. You actually found this paper in the scientific literature. Personally, I found that to be pleasing as Solomon was my father’s graduate school roommate and professorial colleague at the university of Pennsylvania and his younger co-author, Corbit, was my father’s student before he wrote this famous paper with Solomon. In any case, this one is another critical paper.
That’s right Jim, this paper is important. In the Solomon and Corbit’s paper, they discuss how an emotional reaction to a stimulus, whether positive or negative, peaks at the highest reaction point to the stimulus, drops a bit and then goes the other way when the stimulus ends, only to then fade and slowly disappear with time. This reaction is classical and the reason they say it happens is that we actually have two underlying processes. For example, imagine a positive emotion-producing stimulus, like chocolate for me. The first process, called the alpha process, creates the positivity. The other, called the beta process, counteracts this first process. The beta process starts right after the alpha process begins, is opposite, grows slowly, and dies out with a long latency. The result is an aftereffect of eating chocolate that provides the craving in me to eat another bite of chocolate. I interpret the craving as a negative state and act to remove it by eating more chocolate. The beta process also learns with repetition, growing stronger. That leads to a better counteraction of the first initial process of positivity and also allows the craving to get bigger after the chocolate is consumed. In seriously hedonically positive stimuli like heroin, eventually the beta process becomes the well-known withdrawal agony and the pleasure becomes less and less. In short, this is how a pleasure reaction to initial heroin turns into an addiction. But it can relate to other pleasures in life as well. The key is the beta process that is either strengthened by use, or weakened by disuse as explained by Solomon and Corbett 1974.
It is amazing to me that this mechanism exists within our brains, governs a lot of how we react emotionally to stimuli, especially when presented over and over again. The most striking part is that we are unaware of it. This work along with the reward prediction error work you presented above, the opponent process model shows how the unconscious decision-making can be strong within us, shaping what value we decide something brings to our conscious decision-making process. Is this why we are fascinated with a new major, only to have it lose its appeal with time maybe even after graduation when we have joined the profession?
Could be. So what was it about your own study of the field of psychology that caused you to mention that a field might gradually lose its appeal?
During my High school days, I felt very mature for my age. I always thought ahead and wanted to achieve my future sooner. I was told by my friend’s, peers, and teachers that I was an amazing listener. They introduced me to general or undifferentiated psychology since I wanted to look at the mind like a science. That led to my goal to being a clinical psychologist and helping people cope. I see this part as the alpha process. Then as I took more courses, I was concerned that undifferentiated clinical psychology was too broad. This was the growing beta process. It peaked when I discovered applied behavioral analysis because that offered a scientific pathway into the problem of clinical psychology and now will apply to master’s programs in applied behavioral analysis.
So what experiences did you have that helped you reach this conclusion?
Professor J’s class was experiential in the classroom; it was a class that let me work one on one with children with autism. Also when I took more courses in psychology, the one class that kept me interested the most was learning and behavioral analysis courses I have taken here in Queens College. I could even add theories of the unconscious mind that I always loved to learn and experiment with but couldn’t find a topic to really relate it to. And I do love working with children which is what lead me to Professor J’s class. Applied behavior analysis merges all my strongest interests in one specific topic!
We both want to note here how the process of experience happened for Ciara within the classroom. Is this experiential education? Does it matter? It does seem to us be the case that many of these unconscious decision-making processes that we cite in this blog post and through the whole blog as being powerfully impacted by outside-class experiences are at work here, right in the classical academic classroom.
