Beat It by Compulsion

A lot of supernatural beings are described as having certain compulsive ways of behaving, which can be used against them. There are several varieties:

Magical
This is when there is either something magical that forces them to act compulsively, or some magical consequences for not doing so. Examples include a Magically-Binding Contract or geas that is in some way related to the nature of that creature.

Obsessive-Compulsive
This type, similar to the Real Life condition known as Obsessive-Compulsive Disorder, is when there are unpleasant emotional consequences to breaking the compulsion, such as extreme anxiety. Heroic Willpower may make it possible for them to overcome this.

Happy Compulsion
This is when they act compulsively because they want to act that way. Maybe it's really fun to laugh maniacally or answer riddles or whatever, to the point where it's hard to restrain yourself from doing so.

Stupid Compulsion
This is when the character is apparently unaware of the consequences of following the compulsion. In these cases, it can be hard to distinguish from ordinary stupidity, but a highly patterned way of outsmarting a certain kind of creature suggests some sort of compulsiveness. If this is present in AI, then it goes under Artificial Stupidity.

Compare Idiosyncrazy. Often a subtrope of Sanity Has Advantages.

Anime & Manga

 * In Soul Eater, Death the Kid has a crippling obsession with symmetry, resulting in him stopping mid-battle to chew out one of his Weapons for being off center, going home in the middle of a mission to check whether the picture in his room was centered properly (it was), refusing to attack a symmetrical opponent and going berserk and obliterating an asymmetrical one.
 * He manages to ignore such obsessions (specifically not freaking out about the line in his hair) when he needs to deal with Mosquito. But that arguably is a case of one compulsion being overridden by another - his need to create order between life and death.
 * The Contractors of Darker than Black have to fulfill a "remuneration", which is a compulsion to do something after using their powers—and the compulsion is so powerful that they must do it (though it's never really explained what would happen to a Contractor who's unable to fulfill his or her remuneration). This can vary from folding the corners of every page in a book to drinking beer to breaking one's own fingers. The only exception is for someone whose mind is no longer in their original body; for example, Mao, who has the ability to possess animals and whose human body was killed, doesn't have a remuneration.
 * Teru Mikami of Death Note always uses exactly one page of the Death Note each day - no more, no less. This means that Near can replace the page that he uses on the day of the confrontation at the Yellow Box Warehouse weeks in advance. All of the other pages work, so the criminals that Mikami writes down keep dying and he doesn't get suspicious, but when he tries to kill Near and the rest of the SPK, he only confirms his guilt and Light's.

Comics

 * With a Rogues Gallery rife with mental instability, many are the times Batman has won the day simply by playing on his foe's idiosyncrasies. Two-Face is probably one of the most consistent examples, with his need to consult his trademark coin: in theory, the results of the toss are equally as likely to turn up in the hero's favor. (In practice...)
 * In Arkham Asylum: A Serious House on Serious Earth: The doctors, attempting to "cure" Two-Face, try to show him that life has more than two outcomes by replacing his coin with dice and eventually a deck of cards. This reduced Dent to a complete wreck, unable to even go to the bathroom. In the end of the comic, Batman returns the coin and Two-Face immediately flips it to decide whether to kill Batman. He looks at the coin and lets him go...
 * A currently-running story has Two-Face completely unable to control himself after a rival villain stole his coin. Dent's dialogue and internal monologue makes much of his frustration with his lack of control.
 * In an old storyline Batman secretly replaced Two-Face's coin with a gimmicked duplicate that always came up heads. While this prevented Two-Face from making some nasty decisions, his fury when he discovered the trick kind of canceled out the benefit.
 * Which is kind of ironic since Two-Face's coin is a two-headed trick coin: he simply scratched one of the sides.
 * Canonically Thanos has been defeated because he has an uncontrollable compulsion to ultimately let himself lose. Seriously.
 * The Riddler had a nervous breakdown when he realized that his OCD made it so that he couldn't stop leaving clues for the police and Batman to find.
 * A character called Fatman had the power to morph himself into any size and shape, but his Weaksauce Weakness was that he would irresistibly turn back if he ever saw something tasty to eat.

Film

 * In the second Austin Powers movie, Mustafa feels compelled to answer any question that he's asked three times in a row.
 * In Who Framed Roger Rabbit? the titular hero is goaded into revealing his hiding spot when the Big Bad begins tapping out 'shave and a haircut' a gag that no toon can resist. This apparently isn't hyperbole as Roger fails to physically restrain himself and leaps through the wall and answers with 'Two Bits!'

Literature

 * Played with a fair bit in Terry Pratchett's Discworld, where methods to defeat vampires included mixing up their sock drawer (they would then have to pair and sort the socks), and forcing other obsessive-compulsive type behaviour. Another example would be the werewolves in The Fifth Elephant. Saying the word 'bath' actually gets them to wince, and throwing something has them automatically run over and grab it in a doggy-esque manner.
 * The bit about socks is genuine folklore: if you steal a vampire's left sock, fill it with stones and throw it in a river, the vampire will be unable to do anything except search for the sock. And since he can't cross running water, he'll never find it.
 * In the first Artemis Fowl book, the titular Villain Protagonist uses the fairies' inability to enter a human dwelling without permission, compulsion to obey a human while in their dwelling, and inability to come back later to try get revenge to extract a lot of gold from them.

Live Action TV

 * In The X-Files episode "Bad Blood", Mulder diverts an attacking vampire by flinging a bag of sunflower seeds. The vampire is compelled to stop and pick them up. Mention is also made of the alleged vampiric compulsion to untie knots, such as tied shoelaces.
 * In actual vampire folklore, the creatures ARE compelled to untie knots, or to count things. Putting knots in the grave or strewing newspapers about a victim's house, are ways to distract one. So yes, The Count is quite accurate in that respect.
 * In Supernatural they distract a leprechaun (played by Robert Picardo) by spilling salt on the ground and forcing him to count them, giving them time to figure out how to banish him.

Myth and Legend

 * Japanese kappa traditionally have a depression on the top of the head, which must contain water for them to have any power. Fortunately, they are compulsively polite and so if you bow to them, they will bow back - spilling the water and rendering them helpless.
 * The sneaky tactic of pouring really strong alcohol into the head-bowl in an attempt to reduce the kappa to helpless drunkeness has, as of this writing, produced unreliable results.
 * The Jiang Shi (Chinese vampire) can be defeated by strewing anything small, such as grains of rice, in its path. It will be compelled to stop and count them.
 * According to legend, this worked quite well against other vampires too. A vampire would get so caught up in counting the grains that by the time he got them all, the sun would be up and it would be "bye-bye vamp."
 * "One! One grain of rice! Two! Two grains of rice! Three! Three grains of rice! AH AH AH AH AH!"
 * According to Hebrew myth the same applied to certain demons, most notably Asmodeus.
 * In Britain and Germany, you could often trick a changeling into revealing their true age by doing something really strange, usually boiling water in eggshells. Then they'd say something like 'I'm as old as the Eastern Woods, but I've never seen anyone boil water in eggshells before!' Some stories claimed that they'd give up the trick and be defeated just from saying something like that, others used it merely as confirmation before they tortured the changeling to get the true child back.

Table Top Role Playing Game

 * In the Old World of Darkness splatbook for were-cats, each tribe of cats had three or four unbreakable compulsions. As most of these compulsions were significant tactical disadvantages to the were-cats, they strongly discouraged their members from revealing these secrets to ANYONE. For example, one tribe of cats had to follow a line of salt. The effectiveness of this tactic when the cat in question was on large natural salt-deposits was not explored.
 * In the new WoD, changelings who grow powerful enough suffer from "frailities", similar to the old stories about The Fair Folk. One may be unable to drink anything but alcohol, while another may have to dance when she hears a clock chime.
 * The furry RPG Ironclaw has things known as flaws, physical or personality-based quirks that hamper one's abilities elsewhere. While external (physical) flaws don't fit into this category, internal (personality-based) flaws do, and are expected to be followed. Considering that said flaws are usually Lawful Stupid Chaotic Stupid stuff, depending on how strongly you have that flaw, it can be very easy for opponents to exploit them to their advantage and overwhelm you.
 * Khornates and orks in Warhammer 40,000 are murder machines in close combat, but easy to defeat if you outnumber and outgun them: just shoot as they charge into your guns.

Video Games

 * Star Control 2 has The Words: "Hold! What you are doing to us is wrong! Why do you do this thing?" However, they'll still attack you afterwards. The reason it works is

Webcomics

 * The Kingfisher features vampire "progenitors" with mental frailties along these lines. Theodore is a younger vampire who learns the secret weaknesses of the progenitors and uses them for revenge. His first victim, Inka, was compelled to count beans while he attacked her.

Web Original

 * Part of containment procedures for SCP Foundation-578-3 is covering the containment area with sand and painting a labyrinth on the floor, since the undead wight in question compulsively counts salt grains and traces labyrinth lines, slowing it down when it inevitably manifests. There was apparently some resistance to this kind of "folkloric strategy" at first among the scientifically-minded SCP researchers.
 * Had it ever been continued past chapter three, Hanna Is Not a Boy's Name would've featured this tactic with the Super OCD Vampire Hunter, Abner. According to the author's sketches, he would somehow be defeated by the masterfully disgusting Doc Worth; using Worth's repulsiveness to drive the mysophobic (that's 'fear of germs') hunter away.

Western Animation

 * In an episode of Danny Phantom, Desiree the Wishing Ghost is compelled to grant any wish she hears. Sam takes advantage of this to undo her mischief so that Danny can defeat her. In her first appearance, Danny ends up wishing her back into her bottle, lampshading that if he were smarter, he would have just done that in the beginning.
 * Disney's Aladdin series had a Greek inventor character who compulsively revealed weaknesses to his inventions... tied to a compulsion to dictate notes to himself not to do that again.
 * Not to mention his obsession with cleanliness. Splash a bit of mud on his tunic and he'll be incapable of anything but whining until he scrubs it all out.

Real Life

 * In ancient Egypt, cats were considered sacred animals, and to harm one meant bringing down a severe punishment on the harmer, up to and including death. When the Romans invaded, rumor has it that one of their tactics was to have their front line carry cats out in front of them, gambling that this compulsion would prevent the opposing Egyptian army from striking through the cats to get at the Roman troops. As we all know from history, it apparently worked. This compulsion against harming the sacred animals shows up, of all places, in the first Mummy movie, where Branden Frasier's character scares off Imhotep with a cat.
 * Autistic people of all kinds may exhibit this behavior, helpful or not.