Too Clever by Half

"A stupid person can make only certain, limited types of errors; the mistakes open to a clever fellow are far broader. But to the one who knows how smart he is compared to everyone else, the possibilities for true idiocy are boundless."

—Vlad Taltos, Iorich

This is a character archetype. These characters:

Are extremely smart and/or good at whatever it is they do.
Know it, and are probably pretty arrogant (in fact, they tend to think they're even better than they are).
As a result, are continually driven to go farther. Usually they succeed (remember, they're really good), but their failures are spectacular.
Often suffer some impediment, or endure some prejudice, to the point where being dramatically and demonstrably more awesome than everyone else in their field is a necessity if they're going to be seen as a success at all.

Usually this character is the hero (though generally not The Hero); they're often a Foil to Too Dumb to Fool. If they have Blue Blood, they could be an Upper Class Wit.

Examples of Too Clever by Half include:

Anime and Manga

Kiyomaro Takamine in Konjiki no Gash Bell.
Kisuke Urahara and Sousuke Aizen from Bleach.
Sasuke Uchiha from Naruto
Light Yagami and L in Death Note
Lelouch in Code Geass
Hare of Monster Rancher applies this as a Con Artist.
Seto Kaiba in Yu-Gi-Oh
Ciel Phantomhive from Black Butler could also count. He's like the anime-equivalent of Artemis Fowl.
Edward Elric of Fullmetal Alchemist.
Sora from .hack// is at the maximum level possible in The World, has the maximum possible stats, and goes around killing people for fun. Much of his arrogance is probably due to his age. Apart from that, he also openly manipulates everyone and is essentially the most obvious sufferer of Chronic Backstabbing Disorder ever. He has information sources no-one else has and is basically invincible in any of the fights he gets in, constantly killing BT. He gets called on it, but it never hampers him until finally motivated into doing something somewhat heroic (he didn't realize he couldn't get away) and taunting the big bad, at which point she turned him into a Sequel Hook and the show ended.
Taikobo from Hoshin Engi. He's a brilliant strategist who once managed to save an entire village by getting them drunk so they couldn't fight an army that came to capture them and then killed the leader, causing the army to scatter. He's famous for manipulating most of the cast with ease...but the first time he met Dakki, he ended up being enslaved and forced to watch members of his clan get thrown in a pit filled with crocs and snakes, one of them calling him pathetic. His ego went down in size after surviving that.

Film

Megamind. Megamind's intelligence is hyper-advanced compared to Earth standards, he's a genius inventor, and his hobby is creating grandiose revenge schemes against the kid who bullied him in elementary school. Worth mentioning that these schemes don't just fail, they fail.
Mark Whitacre, the title character of The Informant!, is an accomplished scientist who speaks several languages and sorely overestimates his own prowess when he gets between his company's corrupt leadership and an FBI probe. Not only that, but it turns out he's been embezzling millions from the company and spinning outrageous lies to make himself look good, both in the company and in his personal life. Not that he isn't brilliant (he earns two PHDs while in prison), but he's determined to succeed big and when that fails, he fails big.
Katharine Parker in Working Girl, a high-ranking business woman that speaks fluent German and views herself as a trailblazer for women in the business world, with a giant head to match. When it's discovered that she stole a brilliant idea from her equally intelligent secretary, she's promptly (and satisfyingly) fired and disgraced.

Literature

The Vorkosigan Saga': The titular Miles Vorkosigan, as illustrated by the following quote from Mirror Dance

"My game plan all my life has been to demand acceptance of this," a vague wave down the length, or shortness, of his body, "because I was a smart-ass little bastard who could think rings around the opposition, and prove it time after time."

Discworld:
- Moist von Lipwig of Going Postal, Making Money, and Raising Steam is a con man turned government official, who runs his government offices as though they were successively more complex con games. Which of course, in a very real sense, they sort of are.
- There's a phrase that appears in Discworld novels fairly often (though it's considerably older) that actually describes this: "So sharp he kept cutting himself, as my grandmother used to say."
- Ponder Stibbons as well; his impediments are the rest of the staff.
- The Klatchian mastermind behind the international incident in Jingo may qualify as this; he is certainly clever, and his plan would have worked very well, apart from one small problem: his opponent is Vetinari. As a result of this little oversight, his failure is truly monumental and extremely humiliating.
- The cleverness of these characters actually provides a good contrast with Vetinari, who is indisputably Discworld's premiere Magnificent Bastard. On the very rare occasions when he does make a mistake, Vetinari always recovers and learns from them. Also, he knows better than to push his luck, (his family motto is translated as "If it ain't broke, don't fix it"), and has thus so far avoided any spectacular cock-ups.
High Elves (Noldor) in J. R. R. Tolkien's The Silmarillion. Also Men of Nûmenor.
The eponymous character from Artemis Fowl.
Locke from The Lies of Locke Lamora and its sequel by Scott Lynch. He spends his life running elaborate (and usually successful) cons on nobles while posing as a petty thief. The end of the second book covers a massive failure; he's spent the entire book on a plot to rob a casino, and it goes off flawlessly -- except that the paintings he steals are fakes, put out for the express purpose of being stolen.
Kvothe from The Name of the Wind by Patrick Rothfuss. A child prodigy, he talks his way into his world's premier university at the age of fifteen, after having spent three years as a beggar, and promptly antagonizes both one of the masters and the wealthiest and most politically connected student in the university. Between that and his perpetual poverty, he spends most of his time doing absurd things (learning an entire language in a day and a half, getting certified as a musician on a lute with a broken string) just to keep his head above water.
A Song of Ice and Fire
- Tyrion Lannister. As a dwarf, quite literally...
- Petyr "Littlefinger" Baelish is described using this exact phrase by Ned Stark. He is, however, an unusual example of this trope because (so far) he has gone from strength to strength without anything more serious than some minor setbacks, to which he quickly adapts.
Grand Admiral Thrawn of the Star Wars Expanded Universe, especially The Thrawn Trilogy. He'd probably claim that he's not arrogant, and it's true that he has no qualms about accepting a good idea just because it's not his, but he definitely has an ego underneath his self-control. He's an alien in a xenophobic Empire whose talents caused him to get that rank. To a lesser extent, Talon Karrde.
Harry Potter
- Hermione is the smartest person in her classes, tends to be rather rudely disparaging of her peers' intelligence, and is discriminated against for being a muggle-born.
- Dumbledore is also qualified for this trope. ("As I happen to be rather smarter than most men, my mistakes tend to be correspondingly huger.")
- Voldemort is a half-example- he's a magical genius, yes, but he tends to generalize this and assume it means he's smarter than everyone at everything. Given that he is in actuality not even remotely competent at long-term planning, he manages to get all the downsides of this trope without any of the benefits.
Douglas Adams' Dirk Gently. In a scene where he tries to outwit the local Inspector Lestrade, and fails miserably, the inspector sensibly points out that Dirk may be really smart, but the weakness of really smart people is that they assume everyone else is stupid.
Foxface in The Hunger Games manages to survive without killing a single soul, simply stealing food and hiding. It bites her in the ass when she steals some berries Peeta had picked without either of them realizing that they were poisonous.
- Or DID it bite her in the ass? In the movie especially, there are hints she committed suicide on purpose and covered it up so that her family back home didn't get in trouble with the Capitol. Foxface: cunning to the end.
- Katniss actually wonders if Foxface is the most intelligent out of all the tributes. As the Games go on and she realizes that Foxface has lasted so long without a direct confrontation against anyone, Katniss wonders if Foxface, not the intimidating Thresh or the totally batshit Cato, is the real danger.
Kendra in Beastly. She has a history of using her magic to punish rotten people like Kyle, but it bites her in the ass when it attracts too much attention and gets her in trouble with other witches. While her grand plan to improve Kyle via the curse she puts on him does work perfectly (it's designed so that he really can't break it without improving himself), it's also what convinces the other witches to banish her from ever going home. Leastwise, until Kyle accidentally puts a loophole into the spell that lets her go home after he breaks the curse. In the book Bewitched, Kendra apparently had a knack for this even before she had a lot of experience as a witch.
Themistocles Papadapoulos in the bridge book series Bridge in The Menagerie. He delights in deception plays, which frequently confuse opponents into making the one play that can defeat his contracts or ensure theirs when he's defending. One chapter of the second book has an entire section on him called "Too Clever By Half."

Live-Action TV

Rodney McKay from Stargate Atlantis is one of the smartest humans from earth. Despite everyone in the Stargate program being top in their fields, he's the only one to make such a huge deal out of his intelligence. And on number three... while he regularly makes astounding accomplishments while under threat of imminent death, his biggest failure? Blew up a solar system (well, five-sixths of a solar system), and almost destroyed two universes. His pride is so great that he often refuses to work with other scientists on the team because he's convinced they'd just slow him down. This gets pointed out magnificently in an episode where he's working with Real Life celebrity scientists Bill Nye and Neil DeGrasse Tyson, or rather, refusing to work with them. This gets taken to a literal level when an Ascension machine rewrites his genome. He creates a new form of math just to keep up with his new discoveries. After being forced to choose between ascending or dying, he invents a cure for himself, returning himself back to "normal" genius levels, not realizing he was in spitting distance of ascending. For a kicker, all his notes and his new mathematics are so complex that even he can't figure out what they meant, making virtually everything that happened completely moot. Did get a nice Deus Ex Machina out of it for the next season opener, though.
Samantha Carter from Stargate SG-1 isn't arrogant, but she knows how smart she is, is constantly being expected to do more and more impressive things to save humanity's collective asses, usually succeeds at saving the day but occasionally has some pretty spectacular screw ups, and gets discriminated against for being female and human by Ba'al and the free Jaffa.
Dr. Nicholas Rush in Stargate Universe is, quite possibly, even more arrogant than McKay. At the very least, he loves to put other scientists down. At the same time, he sees Eli as a protégé of sorts and isn't as hard on him as on others. In fact, on at least one occasion, he told another crewmember that Eli is actually smarter than him.
Doctor Gregory House. Hell, his choice of role model and reason for becoming a doctor (a burakumin medical genius working as a janitor in a Japanese hospital) was almost explicitly one of these, although without the implied arrogance.

This guy, he knew that he wasn't accepted by the staff, he didn't even try. He didn't dress well. He didn't pretend to be one of them. The people that ran that place, they didn't think that he had anything they wanted. Except when they needed him. Because he was right. Which meant that nothing else mattered. And they had to listen to him.

Sheldon Cooper from The Big Bang Theory is a brilliant theoretical physicist. He's also insufferably arrogant and ignorant of basic social interaction. At least once the other characters acknowledged that if he wasn't Leonard's roommate they wouldn't hang out with him.
Doctor Who: The Doctor has a tendency to drift in and out of this trope, possibly more so in his third incarnation. The Tenth Doctor is especially given to telling people how clever he is.
Villainous example: Orta from the Star Trek: The Next Generation episode "Ensign Ro". Extremely good Bajoran terrorist with many impossible victories against the Cardassians, but his failures cost him his right eye and the ability to speak without a voice synthesizer. Over the course of the episode, Picard and Ro find that Orta did not make a strike they were investigating, because he didn't have the resources to do so (his freighter could only move at half impulse, for example)...because his rep was such that others were terrified in dealing with him.
Sikozu of Farscape definitely fits this trope, especially given her arrogance over her high intelligence.
Jeremy Clarkson on Top Gear is a Badass Driver version. He can do astonishing things with a car (or a hammer) but his devotion to Tim Taylor Technology means his failures are spectacular too.
Sherlock Holmes is undoubtedly an example in the BBC's 2010 modern-day adaptation, Sherlock. Yes, he's very, very clever, but it has its disadvantages, and he doesn't know how to turn it off and be a human being (not a million miles from the Doctor, which given he's scripted by Steven Moffat and Mark Gatiss... not that surprising).
Shows up often on Survivor.
- Russell Hantz, for example, has some savviness as to the mechanics of the game, finding Immunity Idols before receiving any clues to their location for example - but lost, three times, for being completely insufferable, not grasping that you have to avoid getting on the other players' bad sides. (The third time he played, his tribe was aware of his previous two times and made a point to throw him out fast so they wouldn't have to deal with him.)
- "Boston" Rob, on both this show and The Amazing Race is another great example. He definitely has a talent for this stuff, but... well, on Race he decided to screw with other players by making them think there was an earlier flight. While he gloated about sending them into a panicked search for a nonexistent flight, they found one.

Theatre

The eponymous character in Marlowe's Doctor Faustus is one of the best examples of this trope. His own brilliance in all things academic (and belief that he can do even more than he has) lead him to explore Things Man Was Not Meant to Know. The results are predictable.

Video Games

Vicki Kawaguchi of Backyard Sports.
The Onion Knight in Dissidia Final Fantasy. He's quite clever, and knows it (and will remind Terra bout his intelligence at every turn). He easily fits parts 1, 2, and 3. Especially 3. You want a spectacular failure? How about causing Terra to go out of control, let her get brainwashed, beat her up when you can't think of any other way out of it, and let her get kidnapped? His Destiny Odyssey is all bout him being knocked down a serious peg, and learning to listen to his heart, not just his head.
Fallout: New Vegas has Mr. Robert Edwin House, President, CEO and Sole Proprietor of New Vegas. He's a Properly Paranoid genius owner of a Mega Corp who firmly believes Democracy Is Bad (not entirely untrue, considering the circumstances) and defended Las Vegas from nuclear attack to rebuild it to a point of glory, but is incredibly arrogant and refuses to believe that he could be wrong. He also thinks there's no question that he should be the sole autocrat of New Vegas.
Mass Effect 2 introduces Miranda Lawson: as she and her sister were Designer Babies engineered by their megalomaniacal Truly Single Parent to be Born Winners, Instant Expert TV Genius barely begins to describe her capabilities. Lawson drops in on Shepard's resurrection without any knowledge of the process and becomes the project's leader through sheer ability in less than a week. Of course, being unfamiliar with failure, she never sees her mistakes coming: her statement that "any Biotic could be a Barrier Warrior" turns out to be completely in error - it takes a Psycho Prototype or a Cool Old Lady to do it - she gets an ally killed if her advice is followed. She is actually cursed with being aware of this trope, resulting in an inferiority complex - she attributes all her successes to her father's design, and only takes credit for her failures.

Web Comics

Riff, the mad scientist in Sluggy Freelance, not necessarily scientific, but builds stuff. Usually involving weapons, explosives, robotics or dimensional transports. Lately incorporating inflatable technologies. They may not always work predictably ("Let me check my notes...") or reliably, but hey, at least they work, and isn't that cool?
The Order of the Stick:
- Vaarsuvius is the resident Squishy Wizard and needs to be smart. Also, V's an Elf, so there's the arrogance. The demonic contract plotline manages to skate on the very edge of the "spectacular failure" point without quite falling in.
  - Later on, we find that her/his "Familicide" spell killed even people who merely had black dragon blood in them, or anyone related to them, such as their parents. People whose only crime, as V puts it, was "falling in love with a mysterious scarlet-tressed stranger". Cue Heroic BSOD.
- Roy as well, at times:

Oracle: Yes, you've certainly managed to cunningly outsmart yourself at the very least.

- Also Nale. He may be smarter than Elan, but he's definitely not as smart as he thinks he is. For example, Nale had his Horny Devils Succubus femme fatale disguise herself and send the party on a dangerous quest to recover star metal. He assumed that the star metal would have been recovered as "everyone" has known about its existence for a great deal of time. However, it hasn't been recovered, and after it is found the only result is Roy, an enemy of Nale's, having the star metal used to forge an Infinity+1 Sword.
Suspiria, Insufferable Genius mage prodigy from Flipside. She really is a phenomenally powerful mage, but given her youth, she lacks both the experience and stamina of other mages of her rank, making her a much less formidable opponent than she should be. This has bitten her in the ass twice, in-story (the first with tragic consequences, the second costing her the other main characters' good will and respect and any sympathy the former granted her).
In Sinfest, Satan goes angel hunting. So the angels go Satan fishing.

Web Original

Many devisors and gadgeteers in the Whateley Universe get this, but the biggest of all must be Jobe Wilkins, Prince of Karedonia. A literal child prodigy even before he started breaking the laws of reality—and a first-class Jerkass—he sets about making a nanotech formula to transform anyone into his ideal wife. And then he injects himself with it.

Western Animation

A rare villainous example: Princess Azula from Avatar: The Last Airbender is a Magnificent Manipulative Bitch. She is far and away the most intelligent character in the series, is aware of it and never hesitates to treat essentially everyone besides her father like dirt, even when she's not trying to murder them, which isn't often. However, she completely fails to predict Mai's betraying her to Zuko out of love.
- In the season finale, after almost killing both Zuko and Katara, she fails to notice that she and Katara (a water bender) are standing directly over a storm drain. Just as she leans in to electrocute Katara to death, she finds herself trapped in a giant water bubble
Digeri Dingo from Taz-Mania.
Looney Tunes's Wile E. Coyote, especially when pitted against Bugs Bunny, where he acquires a voice with which to proclaim himself a "Super-genius".
Lisa Simpson, The Simpsons.

Marge: Now you know what we mean when we say you're too smart for your own good?

In Iron Man: Armored Adventures, Teenage Tony Stark has many of the problems of his other incarnations (keeping secrets from his friends, arrogantly fiddling with technology he really shouldn't be and some outright hypocrisy) with the problems of a super genius who's been home-schooled all his life and suffering from the loss of his only parent. For example, he creates a really good computer virus that devours data like a swarm of digital locusts but unfortunately merges with a swarm of nano-machines to become the very hungry Technovore monster. When Tony screws up he screws up phenomenally.
The title character of Invader Zim is an Evil Genius, smarter than all the other invaders, but his massive ego and faulty programming prevent him from taking over Earth.
Young Justice: Nobody could say Amanda Waller is an incompetent warden... but at the end of "Terrors", The Alcatraz falls under the control of the Light, there was an almost succesful Great Escape, and she is replaced with Dr. Hugo Strange, one of their agents.

Real Life

Galileo appears to have been one of these (see his entry under Instructional Dialogue), assuming Simplicio really was a caricature of the Pope.