Behavioral Science

Riding an Extinction Burst?

Once a behavior has been learned to be a source of reinforcement and the reinforcement ends extinction will begin. So if a behavior was reinforced but now is not, the behavior will begin to fade. A behavior that is not maintained with reinforcement won’t maintain itself, it will begin to be less reliable, accurate, and consistent. The behavior won’t just fade away, it will actually burst like a bubble. Starting at the narrow point of the accurate, correct, and consistent display of the behavior. As the behavior goes un-backed up the learner will begin to vary the behavior (maybe I really need to do it a little different?) often making the behavior bigger and more dramatic to try to get the reinforcement response they were looking for (if you don’t want it like this, how about I do it twice as big?!)

…

This phenomenon of behavior bursting before extinguishing happens with all species and carries with it the same feelings of frustration, confusion, anger, and desperation. So when we’re training our horses and reinforce a behavior we like, then decide later we no longer want that behavior, be aware of extinction bursts. Even if you didn’t originally intend to reinforce that specific behavior. For example, often when people start hand feeding their horses the horse learns to invade the human’s space and take treats out of the human’s pockets directly (why not it’s easier than waiting?) We decide we don’t like this new “mugging” behavior, and find it rather rude, so we stop hand-feeding, but the horse gets worse and worse, eventually even biting! Now we are resorting to punishment to defend ourselves, our horse is frustrated and we think “hand feeding was such a terrible mistake!” But what really happened was a misunderstood extinction burst. The lack of clear behavioral criteria with the food in the beginning, plus the sudden removal of reinforcement for a behavior that had previously worked is the combination that lead to the inappropriate expression. Just like the child at the store counter.

Understanding this pattern can help us recognize it and avoid it. However there are some trainers who try to use this to their advantage in training, we call this “riding the extinction burst”. Where they train a behavior, reinforcing it lightly and unpredictably. This helps the horse understand what behavior is wanted, but then when it’s not reinforced they exaggerate and vary the behavior making it bigger and better. Then the behavior is reinforced again to stop it from growing beyond this exaggerated point. As the behavior settles back into it’s normal lower expression, reinforcement with be withheld again, causing it to begin bursting again. This is extremely upsetting, confusing, frustrating, and enraging for the learner. It becomes just like gambling, even addictive. It’s an inappropriate and dangerous approach to training. It can easily lead to an outburst of behavior that’s misplaced, it creates negative emotions that could easily be turned into aggressive behaviors. This is the most inappropriate use of R+ training I’ve seen out there. Extremely dangerous for human and extremely upsetting for the horse.

So let’s make sure we work well to prevent this dangerous cycle. As we’ve seen removing the reinforcement isn’t the ideal option, avoiding the problem doesn’t solve the problem. So we need to use reinforcement appropriately:

1. Rate of reinforcement – make sure your RoR is high enough to match the effort of the behavior, in early learning this will begin very rapid and can settle into a more relaxed rate as the horse becomes comfortable with which behavior earns which reinforcement.

2. Food value – using a lower food value can help reduce the learner’s desperation for the reinforcement and reduce the amount of over the top effort they will put towards that behavior. So make sure to match the value of the food with the difficulty of the behavior (for your individual, a high spirited horse may need more satiating and low value food compared to a mild-mannered, quiet horse, who may need slightly higher value, more interesting food to be a strong motivator)

3. Food quantity – using larger quantities of lower value food is more appropriate, like getting a bag of rice rather than a candy bar, it’s satiating and satisfying, but not exhilarating.

4. Alternate source – making sure there are other options available to the learner, so their only source of reinforcement isn’t just through working with you. This will reduce their desperation and frustration levels. Using hay or a treat toy while you’re training can help this.

5. Focus on calm – reinforce soft, mild expressions of behavior if the horse gets worked up while training…

6. Dial it down – provide a moment of satiation, using an easy behavior you can reinforce heavily and frequently, then break down the criteria of the behavior you’re having trouble with, into smaller and easier reinforceable steps.

Conditioned Aversives

“The whip is just an extension of my arm”, “The whip is just encouragement”, “The whip just provides direction”, “my horse isn’t afraid of the whip”.

These are super common phrases used by people who just don’t understand how conditioning happens. Conditioning is when A=B. It only needs to happen once or twice for someone to learn A=B. For example most of us know not to touch an open flame, it will likely burn, we’ve seen flames heat things enough to know how flames=hot.

So whether the whip gently touches the horse, swings gently behind the horse, or just follows the horse without ever touching them – if the horse is working to avoid the whip, it’s aversive. You don’t need to touch the horse with the whip for it to work effectively as an aversive motivator for a horse. It’s still an aversive. Still negative reinforcement.

I see so often traditional and natural horsemanship trainers work so hard to desensitize their horse to the whip, rubbing the whip on them, moving the whip around – but if the horse flinches or seeks escape the horse is punished. All until the horse stands and tolerates the whip flapping all about them. However in the next move they use the whip to ask the horse to move away – if the horse does not the whip is flailed, flapped and smacked until the horse can bear the fear no more and moves off. If the horse is actually no longer afraid of the whip and does not move away they either strike the horse with the whip, reconditioning it as something to be feared and escaped from – or they tie a plastic bag onto the end to make it extra scary!!

So why desensitize them to something you do want them to be afraid of? Because most of us don’t want to believe we are using fear and threats to control our horses. Most of us truly love our horses and train them in the ways we’ve been taught as “correct” and even ethical. We want to believe our horses are doing what we ask out of love for us, trust or even “respect”. But the fact of the matter is that’s just not the way it works.

If they aren’t afraid of the whip more than they dislike the work being asked of them, the whip won’t work.

It’s as simple as that, a math equation, which is better, stand here and allow the whip to flap around, maybe even hit me – or work? It varies horse by horse how aversive the tool needs to be to get how much “work” out of the horse (and how aversive that horse finds the job being asked), but the equation remains the same. “But my horse isn’t actually afraid of the whip”, many people say this too, but if it were true the horse wouldn’t work to avoid it. It’s really as simple as that.

The same is true but opposite for a target. A horse will work to seek out a target so long as it’s value is stronger than the work – except in this case the value is appetitive, not aversive. The target can give direction more clearly and completely than whips – in that it shows them where TO go, not where NOT to go.

Often people, in an attempt to justify the use of their tools or training techniques, in their own mind or others – people will soften the language around these tools. “It’s just an extension of my arm”. Maybe if your hand is as aversive as a whip? It could be! Many of my rescues move rapidly away from a fast moving hand… But is that really the relationship you want your horse to have with your hands? With you? Don’t forget when conditioning is happening it’s not the tool alone that’s being conditioned, you and everything else that’s around are being wrapped up in that picture.

How do you want your horse to see your tools? your hands? You? Do you want to be a conditioned aversive? Or a conditioned appetitive?

Let’s talk conditioning… what’s the difference between classical and operant conditioning? And when do you utilize each in training? Why does it matter?

Classical conditioning is what most people are familiar with, the bell rings, dog anticipates food. Some people know this as Pavlovian conditioning or respondent conditioning. Simple pairing of two stimuli X=Y. Respondent behaviors are behaviors that happen reflexively, involuntarily, such as blinking, increasing heart rate or respiration, wincing, etc… Respondent behaviors happen in “response” to the environment. For example food will trigger salivation. Respondent conditioning is when we add a predictor signal, so while the food causes salivation, the bell predicts the food. Once the bell and food have been sufficiently paired, the bell alone will trigger the response, salivation in anticipation of food.

We can utilize this way of learning through pairing in our training in a wide variety of ways. Most commonly we will condition a sound (like a clicker or whistle) with food, so we can use it in clicker training. We can also condition aversive stimuli, the presence of a whip being paired with the physical sensation of being hit, the whip becomes conditioned as a predictor of pain. This is what we are seeing when a horse picks up speed when the rider is just holding the whip, not using it, because the pairing is understood. We can take any stimuli (sound, sight, touch, etc…) and condition it to predict something else. Once paired consistently the first stimuli takes on the meaning of the second stimuli.

This is important to remember in training. The first stimuli predicts the second one. If you feed, then click, then food is being conditioned to predict the sound. When what we really want is for the sound to predict the food. So we click and then present the food.

We can also take this respondent conditioning one step further. If the horse has an association with a stimuli we can change their response to it through pairing. So a horse may dislike being sprayed with fly spray, but by pairing the fly spray with something they do like (food) we can change how they feel about the spray. So by spraying (starting somewhere the horse is comfortable with the spray) then adding food, the horse learns that the spray predicts food. Soon you have counter conditioned the spray. But keep the order in mind, if food predicts the spray, you may counter condition the food to be something they dislike. We see this often where horses who have been bribed with food learn “the food is a trap”. This is conditioning happening in an unintended direction. The key is the order in which the stimuli are presented.

Operant conditioning is not just giving a stimuli a meaning, it adds a behavioral component. Rather than the environment triggering a reflexive response, this time it triggers a voluntary behavior which will be influenced by the results. The environment triggers a behavior which has a consequence. If the consequence is good the learner will do that behavior more in that scenario again. If the consequence is bad the learner will do that behavior less in that scenario again.

This is where our learning quadrants come into play. These are the 4 possible consequences of a behavior.

Reinforcement is when the consequence a behavior will happen more in the scenario.

This can be done by adding (positive) something the learner likes

Or removing (negative) something the learner dislikes.

Punishment is when the consequence causes the behavior to happen less in the scenario.

This can be done by adding (positive) something the learner dislikes

Or by removing (negative) something the learner likes.

Did you know that the old theories related to dominance have been debunked?

Dominance is not related to leadership, but rather about ownership. The dominant animal has priority access to resources, meaning the other animals in the group know to let them have what they want first. They know this because they’ve previously fought and determined who will have first choice. Because they live in groups it’s wise to make these determinations and not question it often, otherwise constant fighting would weaken their group. Often these dominant individuals will vary based on the resource, some many have first choice of food resources, some may have first choice when it comes to mating, others may defend limited shelter, and so on. Dominance will also vary if the group changes, sometimes 2 horses will work together to get a resource from a horse who is dominant over each separately. Depending how limited the resources are and how desperate the individual is for the resource, these priorities may change. If food is plentiful (like horses on plains full of grass) no one really cares who eats first. The individuals with priority access to resources have no other benefits, they aren’t leaders or bosses, they aren’t in charge.

In a domestic setting dominance can be more linear than in nature, because resources are often far more limited and groups are made of unrelated peers chosen by humans, not families and chosen friends. When resources are limited the importance of priority access is increased. If there is only 1 hay pile, the horses need to determine who gets it, if the hay pile runs out quickly, they’re less likely to be willing to share, for fear of “starving”. Remember horses are designed to eat 24/7, even if our food is too rich for them and they are fat, they still feel hungry when they go without forage. So there may be increased aggression in domestic settings if we don’t spread resources and ensure there is plenty to go around.

While we should take our horse’s established dominance roles into consideration when we manage the environment, it doesn’t affect our training. We can reduce inter-herd fighting and displaced aggression and food related anxiety by managing the environment with plentiful access to resources. With the use of enrichment and slow feeding, track systems and feed variety we can make sure the horses have everything they need without being unhealthy. This being said, dominance has nothing to do with our training.

Because we own and provide everything for our horses, we have no need to fight over the resource. We already decide who, what, when, where and how our horses live their entire life. We already have all the control over all the resources, so acting aggressively towards our animals has nothing to do with dominance, it’s just acting aggressive. Instead of controlling our horses with force and attempted dominance displays, we should use our brain. Using science based training techniques we can positively reinforce the behaviors we want to see more of.