Negative Reinforcement - The good, the bad, and the ugly

Disclaimer: I am mostly a behavioural scientist. I know about animal behaviour from the outside. I know about measuring emotional states, assessing welfare, and looking for behavioural indicators of stress. Some of the topics in these blog posts on negative reinforcement are not my strongest areas. I can only offer my interpretation of the literature. I am certainly open to discussing alternative interpretations.

Negative reinforcement (R-, NR) is an operant conditioning quadrant. Quadrants basically predict how stimuli will affect the frequency, duration, and/or intensity of future behaviour depending on whether they are rewarding or punishing and whether they are added or taken away. In the case of negative reinforcement, future behaviour increases in frequency, duration and/or intensity when something is taken away. In other words, the animal will learn to perform a behaviour in order to gain relief from something they find aversive. Some well known trainers have created what's called a humane hierarchy of training methods as a guide to rank training methods on how humane an intervention they represent. Negative reinforcement is conspicuously far up on these humane hierarchies, prompting many trainers to stringently avoid it. This seems like a tenuous reason to condemn an entire learning quadrant. It makes a few very broad assumptions, such as even the mildest aversive experiences strong enough for an animal to want to avoid have no place in training behaviour. Is having someone brush past you, making you feel too close to them so that you step away really the same beast as having someone scream in your face until you move? I use extreme examples to show the breadth of the quadrant we're talking about. Some forms of negative reinforcement are extremely mild and some are jumping on the toes of downright punishment. And is avoiding an aversive experience always less humane than, say, reinforcing a different behaviour. Has anyone ever asked you to do something still like lie down when you'd rather be running away? But you can have a chocolate if you lie down. Thanks, but I think I'll run.  So what's the story? Can negative reinforcement be a humane way to train an animal? How do we assess that? Let's have a good look at the related scientific literature and see if we can reason out an answer. 

Susan Friedman's "humane hierarchy"

It would seem like the place to start is examining what it feels like for an animal to be negatively reinforced. Perhaps this is the most important question and yet the hardest to answer. I can present what we know of emotions and welfare in animals. I am hesitant to delve into neurotransmitters, neuroanatomy, and stress physiology because it is not my area of expertise. My understanding of the literature may be simplistic. Yet, there are claims being made that negative reinforcement should be avoided on that basis, so let's have a quick look at it. 

Neuroscience

These are chemicals that carry signals throughout the brain. The type of neurotransmitter is important, as is where it is going, but do not for a second think that this is remotely straight forward. The brain is crazy complicated, and very adaptable. Our understanding of it is not complete. Some neurotransmitters are associated with 'good' emotional experiences. One example is dopamine, which is heavily implicated in reward and the anticipation of good things happening. That is a good feeling! But, the lack of a certain type of dopamine receptor inhibits learning of both positive and negative associations with a physical place, and an active avoidance task. That suggests dopamine is implicated in unpleasant feelings as well... All right, so there's also serotonin involved in negative reinforcement. According to recent research, serotonin may have a role to play in how both rewarding (appetitive) and punishing (aversive) stimuli are processed. What does that mean? It means we don't really know exactly what serotonin does and we are probably going to need to study specific serotonin receptors to better understand it. So then there is noradrenaline, (or norepinephrine to use the American name), which is implicated in negative reinforcement learning and also features strongly in stress responses. Its role in stress responses is largely an arousing one, which, as it happens, appears to enhance memory and learning in discrimination tasks. It would be important in understanding what this all means to an animal to know where the neurotransmitters were going and what neural systems were involved. I would go into this except it would probably take me years to understand it enough to be able to condense it into a blog post. Suffice to say, nothing is straight forward in the brain. Even the amygdala, which everyone 'knows' is all about fear, flight and fight and freezing, also plays a critical role in positive emotional states. 

Stress

The concern about neurotransmitters and neuroanatomy involved in negative reinforcement may be in its association with stress and negative emotional states. So let's try there for some clearer answers. Is negative reinforcement stressful to animals? First, let's define what we mean by 'stressful', here. The body's stress response is very adaptive and can handle everything from minor stressors such as being hungry to major "I'm going to die" moments. And it covers positive experiences as well. A dog that is chasing a ball is very aroused and will be experiencing elevated stress hormones. The strength of a stress response is typically proportional to the intensity of the emotion associated with it. A strong stress response associated with a negative event basically means a lot of fear or anger, but a weak stress response means being a little perturbed. The strength of stress responses can be measured in a variety of ways, but most commonly through concentrations of stress-related hormones such as cortisol, either in blood, urine, or saliva. It is not quite an exact science. Check out this excellent blog post for a really nice summary of the issues. At any rate, lots of normal, everyday things raise cortisol concentrations, and cortisol (and glucocorticoids, and other stress-related hormones) are not "bad" per se; we need them! This system is fabulous at what it does, which is to keep us engaged and motivated when we need to be, and to keep us safe and help us recognise opportunities and threats and take appropriate action. We can't learn without stress, and we don't remember things that weren't very stressful all that well. But stress can be very unpleasant, and prolonged or frequent stress responses are dangerous and can cause serious disease and illness. Most people in modern society have experienced chronic stress in some form. It is not fun. To delve into this fascinating topic more, I cannot recommend Professor Robert Sapolsky enough. He has many videos available free on YouTube and his book "Why Zebras Don't Get Ulcers" is entertaining and very informative and still one of my favourites. 

Stress is also not always as simple as isolated events. There are unique stress responses for all kinds of stressors, which speaks to just how finely tuned stress responses are. There is a large body of literature on uncontrollable versus controllable stressors and how exposure to various kinds of stressors moulds future stress responses. All of these changes are 'good' in that they are adaptive and help animals best handle the cards they have been dealt. But some hands are terrible and the best you can do is try to minimise how much you lose out. Some hands offer the beginnings of a better hand down the track.  An example of where stress adaptation may be the beginnings of something better is in what is usually called stress inoculation. Animals that have learned to control stressors are more resilient in the face of future stressors. They try for longer to make things better for themselves when they are stressed, which means they are more likely to succeed and less likely to sink into despondency. They also show more curiosity, better emotional processing, and better cognitive control. All of these things are good for an individual animal. It will help an animal respond appropriately to stress and also handle mild challenges in their life better, such as social situations and impulse control.

So if stress is an important part of everyday life and plays critical roles not just in keeping us safe, but also motivating us, helping us remember and learn, and in some cases has a positive effect on future experiences and responses to stress, then how do we assess the role of stress in negative reinforcement and whether it is good stress, unpleasant stress, or beneficial stress?

At a basic level, looking at what animals find negatively reinforcing can tell us what they wish to avoid, and therefore, what they find unpleasant. Indeed, this has been proposed as an indicator of welfare by a leading animal welfare scientist. And we do find indications that training animals with negative reinforcement results in less approach behaviour towards people, and stronger emotional responses towards people, and these animals are also less engaged with their handlers than animals trained with positive reinforcement. And this brings us to an enormous body of research in avoidance learning, which is where things get interesting.

Active avoidance and stress inoculation

Avoidance is something we all do, keeping ourselves safe, comfortable, and healthy. Lots of research has been done on avoidance learning in animals, but I'm going to focus on active avoidance because it is most like what trainers do when they use negative reinforcement to train animals. Active avoidance is where an animal performs a behaviour on cue to avoid an aversive event. An example we might see often with animals is dashing under furniture to hide when a child comes into the room. The presence of the child cues the avoidance behaviour. Another variation is escape behaviour, where the animal learns to perform a behaviour to 'switch off' an aversive experience. So the animal dashes under the furniture when the child is too rough with it. Both are sensible strategies. The animal gains immediate refuge. But we wouldn't think this was necessarily an ideal sequence of events. Active avoidance is associated with a large elevation in cortisol concentration during the learning phase. It is distressing for animals when they are exposed to something unpleasant. Their distress is generally proportional to how strong the unpleasant experience is (brushing past vs yelling in face). And if their response (run and hide) is one that requires a fair bit of energy, their arousal will be elevated as soon as they see what they will need to run from. Heightened arousal means more intense feelings. Furthermore, if their response isn't always reliable in gaining them refuge, or they sometimes miss the cue, or the cue comes often and randomly, that is a layer of uncertainty that will make them very vigilant and very focused on potential threats to the point where they are likely to see them where they don't exist. All in all, not good.

But what if they are exposed to something unpleasant and have a reliable way to handle it? Is it just as distressing? A significant drop in cortisol concentrations occurs once an avoidance behaviour has stabilised. It does not drop to baseline, because it is arousing to perform an avoidance behaviour no matter how nonchalantly. But the drop in cortisol correlates with an increase in behaviours associated with a relaxed state, suggesting that fear has diminished quite a lot. Commonly, there is no outward appearance of fear or distress, and this nonchalance has been noted by several researchers. Animals are not frantically performing an avoidance behaviour to stave off the bad things. In fact, they tend to wait until the last possible moment to perform it and otherwise go about their usual business. Humans also report reduced fear and an increased sense of control when they use active avoidance during phobia treatment. Furthermore, studies suggest that animals that are taught active behaviours to successfully avoid aversive experiences learn to suppress their natural freezing responses, which in turn enables them to control the stressor through operant means. Freezing up helplessly is considerably worse from a stress perspective than actively coping because it is probably experienced more intensely. Active coping also leads to stress inoculation, which we have already discussed. 

So it seems like successful avoidance is not necessarily associated with significant fear or distress. How could that be? The animals still don't like the aversive they are trying to avoid, right? If they are trying to avoid something it must be unpleasant. That's what Dawkins' suggestion for negative reinforcers as indicators of poor welfare was all about. It turns out there may be some pretty cool and strange things going on...

Safety signals

Bear with me while I give a brief but necessary background, here. A safety signal tells an animal that they are safe in the immediate future. It is trained by giving the animal an aversive experience and then pairing the ABSENCE of that aversive experience with a particular signal. So the signal comes to mean "You are safe for now". Safety signals appear to be able to inhibit fear surrounding an uncontrollable aversive experience AND inhibit the anxiety expressed after that event. This is pretty amazing stuff when you think about it. Safety signals themselves are not entirely negative reinforcement because no response is necessary so no particular behaviour is being reinforced, although an aversive experience is required in order to train one. But, an operant behaviour can take on a similar role, and these are learned through negative reinforcement.

Safety behaviours

 New research shows that it is inherently rewarding to avoid an expected aversive event - in other words, avoidance itself can be a form of positive reinforcement. Wha...? How does THAT work?? It has been suggested that there comes a point where an animal is not so much avoiding the aversive stimulus but approaching safety. Let's call these safety behaviours. It's not necessarily an official name. This is a pretty poorly understood area of science and I am not aware of anyone that has tested whether these behaviours have the same properties as a safety signal. They can be escape behaviours or avoidance behaviours in that they may switch off something unpleasant or avoid it completely, and that's how they become safety behaviours. It seems like a slippery distinction that could be used to justify some quite terrible things that are done to animals using negative reinforcement in the name of training, so let's be very clear about what this means. Safety has to be real to be sought. This means several things for the development of safety behaviours: 

1) It needs to be very clear to the animal that safety has now been attained - ideally, a specific signal (safety signal, possibly a cue or marker can take on this role). 

2) That safety has to be real and meaningful to the animal, not simply declared by a trainer or handler. 

3) Learning a behaviour to attain safety is actually quite hard in many circumstances, because animals already have natural behaviours they will tend to use when they feel threatened. If they are being taught a behaviour that runs counter to their goals (i.e. get distance from the scary thing), they may never be reliable or never learn it at all. 

4) The effectiveness of safety signals are inversely proportional to the strength of the threatening stimulus - in other words, the ability of safety signals to inhibit fear is influenced by arousal. The more threatening something is, the more aroused an animal becomes, and the more aroused they are, the less effective a safety signal will be in inhibiting fear. In short, if someone routinely threatens to clobber you with a baseball bat, it won't make you feel very safe to know that they never will as long as you run to the other side of the room whenever they make the threat. But if someone routinely threatens to swat you with a newspaper, knowing they won't as long as you run to the other side of the room probably will make you feel pretty nonchalant about the threat. 

5) Animals that are regularly becoming afraid are most likely going to become pessimistic, even if they are practiced at avoiding aversive experiences and seeking safety. 

There is no line between "animals like safety" and "I should therefore create scenarios in training where I can reward them with safety." They like safety, but they like tangible rewards more, and if we want happy, optimistic animals, we should be very much focused on giving them as many opportunities to access tangible rewards as we can. We should also be very aware that their sense of safety comes first to the point where if they feel unsafe they will be primarily motivated to seek safety. Food, play, social contact... all of these things come secondary to seeking safety. We should therefore make their safety our first priority. Compromising their sense of safety ourselves is not clever and runs counter to our goals if we want happy animals first and foremost. Training safety signals and safety behaviours can and should be done opportunistically. If you are able to keep your dog safe and protected from aversive experiences at all times, you do not need safety signals. Although it may mean your animal is both more sensitive to aversive experiences and may experience them more intensely. I do not believe it is in any animal's best interests to attempt to protect them from all aversive experiences. They have evolved a truly wondrous system to handle them, just as we have. We just have to be careful that what they experience is well within their coping abilities and does not have a lasting impact on their mood or health. 

Animals look after their safety first and foremost.

Safety behaviours can also play a role in the treatment fears and phobias by increasing the acceptability of exposure. This has, to my knowledge, only been done in humans where safety behaviours can be quite problematic. However, in some circumstances they can offer a stepping stone to further treatment, making sufferers feel less fear during exposure and they tend to approach closer to the object of their fear. I have done this with a wild hare and found indications of similar results. The hare was taught to 'ask' for space by pulling away from me. If he pulled away, I respectfully did not follow or I backed up. He soon began allowing me to touch his flanks, head, and legs, which put him in a very vulnerable position, as if I had wanted to grab him (something that is probably always on a hare's mind), I was in an excellent position to do so. 

Emotional state

All this talk about the ambiguity in neuroscience, stress, and even approach and avoidance behaviours has left us with a bit of a quandary. If neuroscience is crazy complicated, and sometimes stress is good, and sometimes avoidance is positive reinforcement, and safety signals inhibit fear, and negative reinforcement can be very scary or barely register and everything in between, how can we tell if negative reinforcement is an ethical training approach? I believe the answer is in emotional states. Possibly because I have done a lot of work in detecting emotional states. But really, emotional states are at the center of all this. Whatever the animal is experiencing, it should be detectable in behavioural changes, although they may be subtle and take some careful observation. Whether stress or avoidance is good or bad will directly influence emotional state, either positively or negatively, and that will affect behaviour.

An animal will tend to develop a positive mood if they experience a lot of good things, and if they experience unpleasant things, they will tend to develop a negative mood. There are passing few ways to reliably measure emotional state in animals, which is why I was able to do a PhD on it. Lacking the ability to measure neural activity, cortisol concentrations, reward sensitivity, and cognitive bias, we are left with the terrible inadequacy that is behavioural indicators. The biggest problem with behavioural indicators is that they are hard to identify and open to interpretation. There are a few behaviours we know in dogs are associated with elevated cortisol concentrations in an environment where this is almost certainly due to emotional distress, such as increased urinating, physical activity, and increased displacement behaviours (lip licking and paw lifts in particular). In turn, there are very few indicators of positive emotional state. Play is one, and anticipatory behaviour surrounding rewards is another. It is pretty hard to identify positive anticipatory behaviour in dogs because the work hasn't been done, but it's probably fair to say if they are looking something like the picture below, we're on the right track. 

Anticipatory! Image Eric Danley

This is not as useful as we might hope. These are behaviours generally associated with major, chronic stress. It is quite unlikely that we would see this as a result of simply using negative reinforcement in training. Even if we ONLY used negative reinforcement in training. We need something more sensitive. While we can't formally measure cognitive bias and reward loss sensitivity, we can look for it in everyday behaviour. 

1. Exploration and approach behaviour - We would expect a reduction in exploration and approach behaviour if emotional state is tipping towards negative, and an increase where the emotional state is tipping towards positive. It fits in very nicely with what we know about optimism. The horse study cited earlier showed horses trained with negative reinforcement were less explorative and approached people less, which suggests the training has a negative effect on the horses' emotional state. This will hurt our training goals if we like to shape behaviours!

2. Interest in training - If an animal becomes less willing to participate in training, which may manifest in distractibility, nervousness, skittishness, lots of displacement behaviour (sniffing the ground, staring into the distance, scratching, anything to delay having to train), disinterest, and general unwillingness to approach either the trainer or the training environment, this is BAD. It suggests the animal does not enjoy training. We want to see them engaged and readily coming to you, prick eared and leaning forward. Unless they don't have visible ears. Then just leaning forward.

3. Willingness to offer behaviours - Animals in a negative emotional state are expected to be behaviourally suppressed to some degree. They will not really want to try new behaviours and may be reticent to offer those they know even when cued because it is risky to them. If we have an animal that is either offering behaviours on its own or can easily be coaxed into doing something new, they are most likely in a positive emotional state. 

4. Sensitivity to reward loss - Animals that are in a negative emotional state feel keenly when they think they have missed out on a reward they were expecting. In training this may manifest in relative slowness and reluctance particularly where reward rate has decreased or when attempting to move to a variable reinforcement schedule. 

So... what's the verdict?

Click here for an analysis of the literature and my take on the ethics of using negative reinforcement in training and behaviour modification (plus a reference list). 

Negative Reinforcement - Is it ethical?

In the previous post I did a brief review of the literature relevant to negative reinforcement. So are there ever cases where negative reinforcement should be viewed as more humane than its current placement on humane hierarchies?

In my opinion it's not a black and white issue. On the one hand, there is ample evidence that negative reinforcement is associated with elevated cortisol and a reduction in approach and explorative behaviour and therefore can be assumed to be more stressful and unpleasant than training with positive reinforcement. This certainly makes sense in the context of literature on emotions as well. So, it seems its place fairly late in the humane hierarchy is justified in most scenarios.

Image source

But wait... What about safety? Safety signals are a powerful inhibitor of fear, which is surely a good thing, particularly in behaviour modification where problem behaviour is motivated by fear. On the other hand, safety signals are only relevant where an animal anticipates danger. Shouldn't our goal be to stringently avoid our animals anticipating danger? On the face of it, I would say yes, we don't want our animals to anticipate danger. But what if they are already anticipating danger in spite of our efforts to protect them from this? So many problem behaviours are distance increasing behaviours - they are designed to buy an animal distance from something they find threatening. The animal is already anticipating danger. They are already in the exact situation we were hoping they wouldn't experience. How do we get them out? Generally the answer is to use counter-conditioning and desensitisation to change their emotional response to that threatening thing so that they no longer find it threatening. But what if you could tell them straight away "It's okay, you are safe" and have them believe you? Is that a worthwhile trick to have up your sleeve?

In my experience, absolutely. These safety signals generalise easily and can be attached to behaviours that are sensible for animals to do when they feel threatened. For example, one of my dogs falls into a formal heel when he feels threatened and the other walks between my feet. They do this because they firmly believe they will be safe if they do. Not only does it calm them and according to them, magically fix scary situations so they don't have to be scared, but it puts them right by me where I can best protect them and my proximity can make them feel more secure. It puts their attention on me so that they are less likely to react to changes in the threatening thing, for example, a dog starting to run instead of walking. It also means they will move with me, so I can calmly walk them right out of danger, and I have done this on occasion around loose dogs that are acting a bit volatile. Is it a replacement for counter-conditioning and desensitisation? Nope. It is an ace in the hole. It can help you out of a sticky situation, it can buffer your animal from an otherwise upsetting moment by inhibiting stress and anxiety during and following it, and there is almost certainly counter-conditioning occurring at the same time, as feeling relieved and confident around something scary instead of scared and anxious is going to be incorporated into associations made with that scary thing and should make it less scary. At least, that has been my experience and the literature supports this.

But don't you need to deliberately apply an aversive in order to train a safety signal? I thought you said that was stupid. Yes, I did. Because it is, and it's also unnecessary. I don't train safety behaviours like they do in studies. I just train them opportunistically. Sooner or later something will upset my animal and it's already too late to avoid it. I can, however, pair a sound with the moment when they realise they are now a comfortable distance from the scary thing, or to make things easier, the moment they retreat (at a run if you like). With my dogs, I can also ask for a behaviour at the moment when my dog has calmed down enough that they are able to perform it and reward that with a treat. If I always ask for the same behaviour, my dog comes to associate that behaviour with the end of the aversive experience, a sense of relief, and a period of immediate safety. The result is my dog starts performing this behaviour earlier and earlier until realising something scary might be going to happen cues the behaviour. They calm down, they focus on me, they successfully get themselves out of the situation without losing their marbles. Why wouldn't I just counter-condition? Because sometimes the environment isn't as controllable as we would like. I need to walk my dogs for their wellbeing. Sometimes our walks unexpectedly dump us way too close to something scary. There is no way around it. This is just a way to use these unfortunate scenarios for everyone's ultimate benefit.

What about escape behaviours and active avoidance? It might reduce fear, but surely teaching an animal to perform a behaviour in order to escape from an unpleasant experience is ethically questionable? Again, this is not black and white. In general, no, we do not want our animals to feel the need to escape. And again, it happens anyway, just as it happens to us. Sharing the road with a vehicle that looks unsafe makes me want to escape. I feel relief when I successfully distance myself from this naturally occurring situation that makes me intensely uncomfortable. I think it is inevitable that our animals will also find themselves in similar situations, particularly dogs who are out and about in the community with us, and dogs with anxiety problems, and dogs that are highly emotionally reactive. There is every possibility they will bump into a dog that frightens them, for example. Teaching them a controlled escape behaviour that will make them feel calm and in control and also avoid troublesome behavioural outbursts seems humane to me. I sure like it when someone tells me how I can escape from situations I dislike. They are much less stressful when you can quickly and confidently handle them. Have you ever set up an agreement with someone to have them call you away from a situation if you signal to them you want to leave? It gives you an almost guaranteed fast and effective out. Does it make you feel more confident going into that situation? Developing reliable escape and active avoidance behaviours gives animals the means to signal they want out. Sometimes it is argued that this means the animal will forever be asking for outs and handlers will need to be vigilant for the rest of the animal's life. I have not found this to be the case. I mentioned that safety behaviours do have a place in treatment of human fears and phobias, allowing people to feel safer and more in control so they can get closer. Exposure is important for overcoming fears. If the animal clings to their avoidance behaviours to the point where they won't venture any closer willingly, they may need a wee bit of encouragement to find they don't need the avoidance behaviours so much anymore. For animals, I believe that on the odd occasion this happens, it is likely to be an antecedent arrangement at the core of it. Change the setup slightly, or change the sequence of events or the behaviours cued slightly and they should pop right out of their rut and make huge bounds forward. 

This is Kivi's expression when heeling for treats, and when heeling away from dogs that scare the bejesus out of him. 

                   



What about stress inoculation? Improving resilience and giving animals the skills and confidence to work at solving problems seems like a positive thing in general. But is it worth exposing animals to stressors? That is a difficult question to answer, because what kind of stressor would be necessary? Puppies are typically exposed to stressors while they are still with their mother, such as being left alone for short periods, handling physical obstacles like uneven ground, and dealing with frustrations such as siblings that are in competition. On another level, there is frustration later in training where dogs may need to learn to persist in trying to solve a problem in order to access something they are motivated to have. And on another level still, there are more significant stressors like older dogs that do not appreciate puppy behaviour, being confined or restrained, experiencing car rides... As you can see, stress is part of everyday life. The positive effects of stress inoculation are likely to benefit any animal that is going to find themselves in novel environments or in novel contexts (which may include training a new behaviour, incidentally). This does not mean we should all rush out and expose our animals to some form of controllable stressor. Just make sure that when your animal does encounter mild stress, they are equipped to control it. If they are not, sometimes letting them find their own way to controlling it if it is not far beyond their comfort zone can have lasting benefits. See the series on risk aversion for more information, particularly this one.

In summary, there may be situations where negative reinforcement doesn't contravene our training goals if those goals are to have optimistic, confident animals that expect good things to happen to them. Those situations can be generally categorised as where the animal has already encountered something that has threatened their sense of safety and where a sense of control and safety would aid rehabilitation as long as arousal (which may be considered a surrogate for how scared the animal is in this context) is low to moderate and no higher.

CAUTION!!

As we have seen, negative reinforcement is not without risk. It behooves us to be careful with this. 

1. Do not use it to train approach behaviours - you don't want an animal approaching something in order to make it leave them alone. This is how we end up with things like dogs rushing and lunging in the first place. Some horse trainers do this kind of thing and it works, but personally, I am wary of it. I rely on whether my animals will approach to tell me if they like or are comfortable with something. Teaching an approach behaviour with negative reinforcement robs me of that information. 

2. Know when to bail - This will vary from animal to animal, but you should know before you do anything how far you will go. Generally speaking, I would abandon training if my animal is trying to escape, if they are frozen, and if their arousal has climbed to the point where they are darting glances around. They should be calm enough to respond to their name readily and be able to perform cued behaviours reliably. 

3. Keep track of indicators of emotional state. The point of using negative reinforcement in the contexts suggested here is to improve welfare and give your animal some flexibility in how they cope with stressors. You need to stop and rethink if it is not obviously doing that. 

4. Beware sticky avoidance behaviours hampering progress - Many psychiatric disorders in people are maintained to some degree by safety and avoidance behaviours. This seems like a minor concern in animals, who do not have such complex psyches, but it is well known that animals can continue with avoidance behaviours long after they are necessary. There is a problem here with prediction errors. In order for an animal to learn that they do not need their avoidance behaviour, they need to see that they didn't use their avoidance behaviour and nothing bad happened. If they are practiced avoiders, they may not have the opportunity to see this. As mentioned earlier, it's not that big a deal. Change the context just a little and they should change their behaviour. Sometimes allowing an animal to continue to avoid if they are comfortable with that is smart. We don't necessarily need them to approach everything and may not want them to.

References

Macoveanu, Julian. "Serotonergic modulation of reward and punishment: Evidence from pharmacological fMRI studies." Brain research (2014).

Smith, J. W., et al. "Dopamine D2L receptor knockout mice display deficits in positive and negative reinforcing properties of morphine and in avoidance learning." Neuroscience 113.4 (2002): 755-765.

Christianson, John P., et al. "Safety signals mitigate the consequences of uncontrollable stress via a circuit involving the sensory insular cortex and bed nucleus of the stria terminalis." Biological psychiatry 70.5 (2011): 458-464.

Christianson, John P., et al. "Inhibition of fear by learned safety signals: a mini-symposium review." The Journal of Neuroscience 32.41 (2012): 14118-14124.

Helmreich, Dana L., et al. "Active behavioral coping alters the behavioral but not the endocrine response to stress." Psychoneuroendocrinology 37.12 (2012): 1941-1948.

Moscarello, Justin M., and Joseph E. LeDoux. "Active avoidance learning requires prefrontal suppression of amygdala-mediated defensive reactions." The Journal of Neuroscience 33.9 (2013): 3815-3823.

Kim, Hackjin, Shinsuke Shimojo, and John P. O'Doherty. "Is avoiding an aversive outcome rewarding? Neural substrates of avoidance learning in the human brain." PLoS biology 4.8 (2006): e233.

Coover, Gary D., and Holger Ursin. "Plasma-corticosterone levels during active-avoidance learning in rats." Journal of comparative and physiological psychology 82.1 (1973): 170.

Innes, Lesley, and Sebastian McBride. "Negative versus positive reinforcement: an evaluation of training strategies for rehabilitated horses."Applied animal behaviour science 112.3 (2008): 357-368.

Lyons, David M., and Karen J. Parker. "Stress inoculation‐induced indications of resilience in monkeys." Journal of traumatic stress 20.4 (2007): 423-433.

Sankey, Carol, et al. "Reinforcement as a mediator of the perception of humans by horses (Equus caballus)." Animal cognition 13.5 (2010): 753-764.

Beerda, B., et al. "Behavioural and hormonal indicators of enduring environmental stress in dogs." ANIMAL WELFARE-POTTERS BAR- 9.1 (2000): 49-62.

Cain, C. K., J. S. Choi, and J. E. LeDoux. "Active avoidance and escape learning." Encyclopedia of Behavioral Neuroscience. New York: Elsevier(2010).

Levy, Hannah C., and Adam S. Radomsky. "Safety Behaviour Enhances the Acceptability of Exposure." Cognitive behaviour therapy 43.1 (2014): 83-92.

Hood, Heather K., et al. "Effects of safety behaviors on fear reduction during exposure." Behaviour research and therapy 48.12 (2010): 1161-1169.

Rachman, Stanley, et al. "Does escape behavior strengthen agoraphobic avoidance? A replication." Behavior Therapy 17.4 (1986): 366-384.

Dalley, Jeffrey W., et al. "Distinct changes in cortical acetylcholine and noradrenaline efflux during contingent and noncontingent performance of a visual attentional task." The Journal of Neuroscience 21.13 (2001): 4908-4914.

Burgdorf, Jeffrey, and Jaak Panksepp. "The neurobiology of positive emotions." Neuroscience & Biobehavioral Reviews 30.2 (2006): 173-187.

Dawkins, Marian Stamp. "The science of animal suffering." Ethology 114.10 (2008): 937-945.

Deldalle, Stephanie, and Florence Gaunet. "Effects of 2 training methods on stress-related behaviors of the dog (Canis familiaris) and on the dog–owner relationship."Journal of Veterinary Behavior: Clinical Applications and Research 9.2 (2014): 58-65.

Koolhaas, J. M., et al. "Coping styles in animals: current status in behavior and stress-physiology." Neuroscience & Biobehavioral Reviews 23.7 (1999): 925-935.

Burman, Oliver HP, et al. "Sensitivity to reward loss as an indicator of animal emotion and welfare." Biology letters 4.4 (2008): 330-333.

Forbes Creek, Woronora

We are very fortunate in the Sutherland Shire as there are plenty of bushy places we can take our dogs. I mentioned the Woronora pipeline in the post about the Needles. The pipeline is generally a reliable starting place for lots of bushy adventures with dogs. At the northern end of the pipeline, there is a section between Woronora and Woronora Heights/Yarrawarah that goes through a nice little patch of bush surrounding Forbes Creek. There is a surprising network of narrow mountain bike tracks east of the creek, and the pipeline is on the western side. I have prepared a map with a loop walk using the large access track alongside the pipeline and then into the bike trails. Don't worry too much about following the correct bike trail. I think most of them join up eventually.

A loop walk around Forbes Creek, Woronora.

Access the pipeline in this section either from the end of Thorp Rd in Woronora or Bundanoon Rd in Woronora Heights. The pipeline access road is easily visible on Google Maps. We come the Woronora end and walk up alongside the pipeline. We turn off the pipeline road close to Bundanoon Rd. The track is small and difficult to see. Check the photo below for what it looks like where it joins the pipeline track.

The trail leaving the pipeline road is hard to see.

Follow the narrow bike track down to Forbes Creek and look for the track on the other side. It goes both ways along the creek, but turn to the right and follow it south. There will be a few more branches. Go right again, then left to curve around up higher above the creek and it will double back and head north. Eventually it will wind down towards the creek again. There are more branches around this area, but they all end up in much the same place. You will come out at the creek, then go up the hill a little ways again and eventually end up on a large fire trail. Follow it down the hill where there is a bridge across Forbes Creek, and then you will be back on the pipeline track.

Forbes Creek crossing on bike trail. 

Size

This patch of bush is not huge, but certainly big enough for a good wander. The loop walk on the map is about 5.4km long and takes about 1 1/2-2 hours if you take it at a leisurely pace. 

Safety

As with most bushy areas, there are snakes here. There are also bikes. If you visit on a nice, sunny weekend you will probably see one or two bikes. The bike trails are narrow and winding and you may not get a lot of warning before they arrive. There are also a few wooden structures that give bikes purchase on steep, slippery slopes and keep them off muddy areas. These are not dog friendly, as there are gaps between the wood slats that paws can fall into. There are ways around, though. There are some short sections that are very steep and a bit loose. The dogs will cope fine, but people will need to be steady on their feet and wearing shoes with decent grip. Be aware that when it has rained, the soil here has a lot of clay in it and can become extremely slick and treacherous in places. The pipeline road is all right, but around the bridge across Forbes Creek it can get pretty slippery. Forbes Creek itself can also get quite swollen after rain. Crossing it might require gumboots and carrying small dogs. 

Steep, loose sections and mountain bike wooden structures can be tricky for dogs and people.

Other dogs and owners

Dogs are supposed to be on leash. We rarely see other dogs and their owners in this area. If you do see other dogs, it is likely they will not be on leash and they may not be friendly. However, we have only ever seen dogs on the pipeline road and there is plenty of room there to keep clear of other dogs. 

Further notes

We are very fortunate to be able to walk our dogs in these bushy areas. Please show your gratitude by cleaning up after your dogs and minimising their impact on the environment. As well as the odd snake, there are also sometimes wallabies and water dragons in this area, and we have seen echidnas nearby. The birdlife is also quite healthy. Keep your dogs close and under control so they don't harass the wildlife. 

 

Happy as a pig in mud - a beginner's guide to using research in discussions

A blog post has been doing the rounds lately entitled "Don't Get Mud On Your Face! Citing Research In Discussions", which lists some very sensible and thorough rules to follow when considering citing research in a discussion. Presumably in support of a point you are making, which is generally the accepted purpose of citing research in discussions. In the humble opinion of this young scientist, the list of rules is a very good list and I follow most of the rules when I am reading papers. But reading papers and citing research is part of my job that I have been formally trained to do over many years. Maybe the average person with a curious mind and Google Scholar at their fingertips can get a lot out of simply dipping their toes into research literature without being expected to dive in and swim a lap or two. So here is a friendly guide I have put together for beginners, hobbyists, and those that are just up for a paddle and not much more. Science is for everyone! And I heartily encourage anyone from any field to go poking around primary literature.

Science is for everyone! (Image source)

1) Chasing down the original article is indeed an excellent first step. Reporters and journalists are notorious for slightly mangling published research in the retelling. If you cannot get hold of the original article, or you don't understand much of it, that's okay, all is not lost. Knowing something about the credibility of the reporter will help you. If they have good credentials, such as being a scientist themselves, that is a very good thing. Ideally, they would have university level qualifications in the field they are reporting on. Also, where is it being reported? An online magazine or blog dedicated to reporting science is better than a general news site, but be aware that even dedicated science journalism sources tend to twist things around a bit to make it sound more significant, exciting, and potentially game-changing than it really is. Their job is to get people to read it. If it's a personal blog, be aware that the writer may have an agenda they are pushing.

2) If you can track down the original article and you don't understand it, DON'T PANIC. And don't give up there. A well-written paper that is easy to read is a rare gem. Most scientists are appalling writers, and some fields are thick with jargon. Back when I was a wee little undergrad, I read papers in the order they were written. I recommend this for beginners. The Abstract is a good place to start. The Introduction will put the study in context. Some Introductions do this well and some do them terribly, so don't be dismayed if you still don't really get it. Methods can be heavy going. It's possible I will be lynched for this, because to people like me this section of the paper is crucial, but if you can't easily follow it, just skim it. It's unlikely that a beginner will get a whole lot out of it anyway. Same for the Results, although give the graphs some attention. The Discussion is where things get interesting and you will probably get the most use out of it. It will outline the important results and tell you whether they were expected and offer explanations for why they differed from expected if they did, as well as hopefully identifying limitations with the study. If you just understand a few key points from the paper, that is okay. It doesn't mean you are too ignorant to present those points to other people. Just acknowledge that you are a beginner and offer it up politely rather than beating someone over the head with it.

3) Keep your critical thinking cap on and step away from your biases. It could be the hardest bit about reading papers. Try to remember you are not reading it to ultimately find proof for an argument you hold dear. That is not how science works. Science provides evidence in support of ideas. The literature often isn't clear, particularly in behaviour. If you look, you will probably find papers that seem to support things you don't believe. Don't be afraid of them. If you are trying to argue a point, knowing about those papers that seem to refute it is a good thing! And sometimes it will change your mind. If you are brave enough. Below is the first of a series of helpful little shorts on critical thinking by TechNyou that can be found on YouTube. They are really good!

4) One reference does not an argument make. For all that science is about gathering evidence objectively and testing hypotheses with as little bias as possible, it's unlikely that a single paper will solve the issue once and for all. Science is a cumulative process after all, and it's ridiculously hard to test all variables at once. 

5) Lastly, a few tips about critiquing peer reviewed research:

Don't. 

This is where you generally need a good, solid background in the field with several years of practical experience in research behind you. Be aware that the study was done by scientists and then judged by other scientists to be suitable for publication. Do you know better than 3+ scientists and a journal editor or two? Think very carefully before you decide you are. It is true that there is plenty of poor science out there, but don't be too hasty with your cynicism. 

* Remember that a single study can only really examine one slice of the pie, but that's okay. As long as the authors know that there is a pie and have some idea what it looks like and therefore some idea of which slice they have and what is not in it. 

* Statistics can do some clever things and can make up for some tricksy variables that are hard to pin down. Statistics can also find patterns in data that are not obvious otherwise. 

* Scientific methods have to be as standardised as possible. We can't, for example, apply rules for one dog but change them for another simply because they don't work as well for that dog for whatever reason. To make comparisons or find patterns, all dogs need to have been exposed to the same treatment or procedure. Individual differences is not something that is especially compatible with statistics, but that doesn't mean scientists have never noticed them. It just means it's very hard to do anything about it. We're working on it, though!