Negative Reinforcement - Is it ethical?

In the previous post I did a brief review of the literature relevant to negative reinforcement. So are there ever cases where negative reinforcement should be viewed as more humane than its current placement on humane hierarchies?

In my opinion it's not a black and white issue. On the one hand, there is ample evidence that negative reinforcement is associated with elevated cortisol and a reduction in approach and explorative behaviour and therefore can be assumed to be more stressful and unpleasant than training with positive reinforcement. This certainly makes sense in the context of literature on emotions as well. So, it seems its place fairly late in the humane hierarchy is justified in most scenarios.

Image source

But wait... What about safety? Safety signals are a powerful inhibitor of fear, which is surely a good thing, particularly in behaviour modification where problem behaviour is motivated by fear. On the other hand, safety signals are only relevant where an animal anticipates danger. Shouldn't our goal be to stringently avoid our animals anticipating danger? On the face of it, I would say yes, we don't want our animals to anticipate danger. But what if they are already anticipating danger in spite of our efforts to protect them from this? So many problem behaviours are distance increasing behaviours - they are designed to buy an animal distance from something they find threatening. The animal is already anticipating danger. They are already in the exact situation we were hoping they wouldn't experience. How do we get them out? Generally the answer is to use counter-conditioning and desensitisation to change their emotional response to that threatening thing so that they no longer find it threatening. But what if you could tell them straight away "It's okay, you are safe" and have them believe you? Is that a worthwhile trick to have up your sleeve?

In my experience, absolutely. These safety signals generalise easily and can be attached to behaviours that are sensible for animals to do when they feel threatened. For example, one of my dogs falls into a formal heel when he feels threatened and the other walks between my feet. They do this because they firmly believe they will be safe if they do. Not only does it calm them and according to them, magically fix scary situations so they don't have to be scared, but it puts them right by me where I can best protect them and my proximity can make them feel more secure. It puts their attention on me so that they are less likely to react to changes in the threatening thing, for example, a dog starting to run instead of walking. It also means they will move with me, so I can calmly walk them right out of danger, and I have done this on occasion around loose dogs that are acting a bit volatile. Is it a replacement for counter-conditioning and desensitisation? Nope. It is an ace in the hole. It can help you out of a sticky situation, it can buffer your animal from an otherwise upsetting moment by inhibiting stress and anxiety during and following it, and there is almost certainly counter-conditioning occurring at the same time, as feeling relieved and confident around something scary instead of scared and anxious is going to be incorporated into associations made with that scary thing and should make it less scary. At least, that has been my experience and the literature supports this.

But don't you need to deliberately apply an aversive in order to train a safety signal? I thought you said that was stupid. Yes, I did. Because it is, and it's also unnecessary. I don't train safety behaviours like they do in studies. I just train them opportunistically. Sooner or later something will upset my animal and it's already too late to avoid it. I can, however, pair a sound with the moment when they realise they are now a comfortable distance from the scary thing, or to make things easier, the moment they retreat (at a run if you like). With my dogs, I can also ask for a behaviour at the moment when my dog has calmed down enough that they are able to perform it and reward that with a treat. If I always ask for the same behaviour, my dog comes to associate that behaviour with the end of the aversive experience, a sense of relief, and a period of immediate safety. The result is my dog starts performing this behaviour earlier and earlier until realising something scary might be going to happen cues the behaviour. They calm down, they focus on me, they successfully get themselves out of the situation without losing their marbles. Why wouldn't I just counter-condition? Because sometimes the environment isn't as controllable as we would like. I need to walk my dogs for their wellbeing. Sometimes our walks unexpectedly dump us way too close to something scary. There is no way around it. This is just a way to use these unfortunate scenarios for everyone's ultimate benefit.

What about escape behaviours and active avoidance? It might reduce fear, but surely teaching an animal to perform a behaviour in order to escape from an unpleasant experience is ethically questionable? Again, this is not black and white. In general, no, we do not want our animals to feel the need to escape. And again, it happens anyway, just as it happens to us. Sharing the road with a vehicle that looks unsafe makes me want to escape. I feel relief when I successfully distance myself from this naturally occurring situation that makes me intensely uncomfortable. I think it is inevitable that our animals will also find themselves in similar situations, particularly dogs who are out and about in the community with us, and dogs with anxiety problems, and dogs that are highly emotionally reactive. There is every possibility they will bump into a dog that frightens them, for example. Teaching them a controlled escape behaviour that will make them feel calm and in control and also avoid troublesome behavioural outbursts seems humane to me. I sure like it when someone tells me how I can escape from situations I dislike. They are much less stressful when you can quickly and confidently handle them. Have you ever set up an agreement with someone to have them call you away from a situation if you signal to them you want to leave? It gives you an almost guaranteed fast and effective out. Does it make you feel more confident going into that situation? Developing reliable escape and active avoidance behaviours gives animals the means to signal they want out. Sometimes it is argued that this means the animal will forever be asking for outs and handlers will need to be vigilant for the rest of the animal's life. I have not found this to be the case. I mentioned that safety behaviours do have a place in treatment of human fears and phobias, allowing people to feel safer and more in control so they can get closer. Exposure is important for overcoming fears. If the animal clings to their avoidance behaviours to the point where they won't venture any closer willingly, they may need a wee bit of encouragement to find they don't need the avoidance behaviours so much anymore. For animals, I believe that on the odd occasion this happens, it is likely to be an antecedent arrangement at the core of it. Change the setup slightly, or change the sequence of events or the behaviours cued slightly and they should pop right out of their rut and make huge bounds forward. 

This is Kivi's expression when heeling for treats, and when heeling away from dogs that scare the bejesus out of him. 

                   



What about stress inoculation? Improving resilience and giving animals the skills and confidence to work at solving problems seems like a positive thing in general. But is it worth exposing animals to stressors? That is a difficult question to answer, because what kind of stressor would be necessary? Puppies are typically exposed to stressors while they are still with their mother, such as being left alone for short periods, handling physical obstacles like uneven ground, and dealing with frustrations such as siblings that are in competition. On another level, there is frustration later in training where dogs may need to learn to persist in trying to solve a problem in order to access something they are motivated to have. And on another level still, there are more significant stressors like older dogs that do not appreciate puppy behaviour, being confined or restrained, experiencing car rides... As you can see, stress is part of everyday life. The positive effects of stress inoculation are likely to benefit any animal that is going to find themselves in novel environments or in novel contexts (which may include training a new behaviour, incidentally). This does not mean we should all rush out and expose our animals to some form of controllable stressor. Just make sure that when your animal does encounter mild stress, they are equipped to control it. If they are not, sometimes letting them find their own way to controlling it if it is not far beyond their comfort zone can have lasting benefits. See the series on risk aversion for more information, particularly this one.

In summary, there may be situations where negative reinforcement doesn't contravene our training goals if those goals are to have optimistic, confident animals that expect good things to happen to them. Those situations can be generally categorised as where the animal has already encountered something that has threatened their sense of safety and where a sense of control and safety would aid rehabilitation as long as arousal (which may be considered a surrogate for how scared the animal is in this context) is low to moderate and no higher.

CAUTION!!

As we have seen, negative reinforcement is not without risk. It behooves us to be careful with this. 

1. Do not use it to train approach behaviours - you don't want an animal approaching something in order to make it leave them alone. This is how we end up with things like dogs rushing and lunging in the first place. Some horse trainers do this kind of thing and it works, but personally, I am wary of it. I rely on whether my animals will approach to tell me if they like or are comfortable with something. Teaching an approach behaviour with negative reinforcement robs me of that information. 

2. Know when to bail - This will vary from animal to animal, but you should know before you do anything how far you will go. Generally speaking, I would abandon training if my animal is trying to escape, if they are frozen, and if their arousal has climbed to the point where they are darting glances around. They should be calm enough to respond to their name readily and be able to perform cued behaviours reliably. 

3. Keep track of indicators of emotional state. The point of using negative reinforcement in the contexts suggested here is to improve welfare and give your animal some flexibility in how they cope with stressors. You need to stop and rethink if it is not obviously doing that. 

4. Beware sticky avoidance behaviours hampering progress - Many psychiatric disorders in people are maintained to some degree by safety and avoidance behaviours. This seems like a minor concern in animals, who do not have such complex psyches, but it is well known that animals can continue with avoidance behaviours long after they are necessary. There is a problem here with prediction errors. In order for an animal to learn that they do not need their avoidance behaviour, they need to see that they didn't use their avoidance behaviour and nothing bad happened. If they are practiced avoiders, they may not have the opportunity to see this. As mentioned earlier, it's not that big a deal. Change the context just a little and they should change their behaviour. Sometimes allowing an animal to continue to avoid if they are comfortable with that is smart. We don't necessarily need them to approach everything and may not want them to.

References

Macoveanu, Julian. "Serotonergic modulation of reward and punishment: Evidence from pharmacological fMRI studies." Brain research (2014).

Smith, J. W., et al. "Dopamine D2L receptor knockout mice display deficits in positive and negative reinforcing properties of morphine and in avoidance learning." Neuroscience 113.4 (2002): 755-765.

Christianson, John P., et al. "Safety signals mitigate the consequences of uncontrollable stress via a circuit involving the sensory insular cortex and bed nucleus of the stria terminalis." Biological psychiatry 70.5 (2011): 458-464.

Christianson, John P., et al. "Inhibition of fear by learned safety signals: a mini-symposium review." The Journal of Neuroscience 32.41 (2012): 14118-14124.

Helmreich, Dana L., et al. "Active behavioral coping alters the behavioral but not the endocrine response to stress." Psychoneuroendocrinology 37.12 (2012): 1941-1948.

Moscarello, Justin M., and Joseph E. LeDoux. "Active avoidance learning requires prefrontal suppression of amygdala-mediated defensive reactions." The Journal of Neuroscience 33.9 (2013): 3815-3823.

Kim, Hackjin, Shinsuke Shimojo, and John P. O'Doherty. "Is avoiding an aversive outcome rewarding? Neural substrates of avoidance learning in the human brain." PLoS biology 4.8 (2006): e233.

Coover, Gary D., and Holger Ursin. "Plasma-corticosterone levels during active-avoidance learning in rats." Journal of comparative and physiological psychology 82.1 (1973): 170.

Innes, Lesley, and Sebastian McBride. "Negative versus positive reinforcement: an evaluation of training strategies for rehabilitated horses."Applied animal behaviour science 112.3 (2008): 357-368.

Lyons, David M., and Karen J. Parker. "Stress inoculation‐induced indications of resilience in monkeys." Journal of traumatic stress 20.4 (2007): 423-433.

Sankey, Carol, et al. "Reinforcement as a mediator of the perception of humans by horses (Equus caballus)." Animal cognition 13.5 (2010): 753-764.

Beerda, B., et al. "Behavioural and hormonal indicators of enduring environmental stress in dogs." ANIMAL WELFARE-POTTERS BAR- 9.1 (2000): 49-62.

Cain, C. K., J. S. Choi, and J. E. LeDoux. "Active avoidance and escape learning." Encyclopedia of Behavioral Neuroscience. New York: Elsevier(2010).

Levy, Hannah C., and Adam S. Radomsky. "Safety Behaviour Enhances the Acceptability of Exposure." Cognitive behaviour therapy 43.1 (2014): 83-92.

Hood, Heather K., et al. "Effects of safety behaviors on fear reduction during exposure." Behaviour research and therapy 48.12 (2010): 1161-1169.

Rachman, Stanley, et al. "Does escape behavior strengthen agoraphobic avoidance? A replication." Behavior Therapy 17.4 (1986): 366-384.

Dalley, Jeffrey W., et al. "Distinct changes in cortical acetylcholine and noradrenaline efflux during contingent and noncontingent performance of a visual attentional task." The Journal of Neuroscience 21.13 (2001): 4908-4914.

Burgdorf, Jeffrey, and Jaak Panksepp. "The neurobiology of positive emotions." Neuroscience & Biobehavioral Reviews 30.2 (2006): 173-187.

Dawkins, Marian Stamp. "The science of animal suffering." Ethology 114.10 (2008): 937-945.

Deldalle, Stephanie, and Florence Gaunet. "Effects of 2 training methods on stress-related behaviors of the dog (Canis familiaris) and on the dog–owner relationship."Journal of Veterinary Behavior: Clinical Applications and Research 9.2 (2014): 58-65.

Koolhaas, J. M., et al. "Coping styles in animals: current status in behavior and stress-physiology." Neuroscience & Biobehavioral Reviews 23.7 (1999): 925-935.

Burman, Oliver HP, et al. "Sensitivity to reward loss as an indicator of animal emotion and welfare." Biology letters 4.4 (2008): 330-333.