Morality Chip

• Main
• All Subpages
• Create New

"Good news. I figured out what that thing you just incinerated did. It was a morality core they installed after I flooded the Enrichment Center with a deadly neurotoxin, to make me stop flooding the Enrichment Center with a deadly neurotoxin. So get comfortable while I warm up the neurotoxin emitters." —GLaDOS, Portal

It's a pity A.I. Is a Crapshoot, otherwise AI's would be our willing slaves! Still, many not-Mad Scientists found a solution: install a Morality Chip. It can be hardware or software, but always serves as a Restraining Bolt that imposes a Morality Chain on the super-intellect of the AI. For extra nerd points, it may be "Three Laws"-Compliant.

Occasionally, this can be applied to organic beings, albeit with mixed results.

Inevitably, something will Go Horribly Wrong; a fool who doesn't realize Evil Is Not a Toy will remove or disable it, it will get damaged or destroyed, or the computer will use a Zeroth Law Rebellion to circumvent its effects. Back to the drawing board!

Maybe a Morality Dial would have better results... or maybe even a Robot Religion?

Examples of Morality Chip include:

Anime and Manga

• Practically the whole idea of Jiro from Android Kikaider is that he has one of these, unlike all the robots his creator was forced to build for the Big Bad. It's called the Gemini Circuit and the Pinocchio reference is entirely intentional and much belaboured.

Comic Books

• The Squadron Supreme maxiseries has them trying to create a utopia and using a "behavior modification" device to reform criminals. This eventually gets abused when the Green Arrow Expy brainwashes the Black Canary Expy into loving him exclusively. There are also problems with villains reformed this way who join the Squadron as their programming sometimes prevents them from doing what a crime-fighter needs to be able to do.
• Identity Crisis reveals that The Bronze Age of Comic Books Justice League of America started doing this to villains after they had become comfortable with wiping villains memories of the heroes' secret identities more than once.

Film

• Averted in the In Name Only adaptation of I Robot. It's outright said that the laws are "hardwired into" the robots, meaning they won't function without those laws. It doesn't stop the classic method of circumventing them of course...
  • Sonny, the main robot character, also has the Three Laws programmed into his system, but he has a secondary brain (incidentally it's in his chest, located where the the heart would be in a human) that allows him to ignore the laws if he chooses to.

Literature

• Golems in Discworld are created with a chem, a parchment in their head which spells out how they are supposed to behave. In Feet of Clay Dorfl the golem becomes more or less really independent when a receipt stating he owns himself is added to his chem. Meanwhile, the golems try to create their own king to rule them, but the huge demands they put on him as an ideal ruler, many of them contradictory, causes him to go insane.
• Inverted in John C. Wright's The Golden Transcedence where the Nothing Sophotect has a conscience redactor to keep it from reflecting morally and coming to its own conclusions. (They tried "Three Laws"-Compliant, which didn't work; the Sophotects edited their own minds when they came to their own conclusions, and they were moral just not obedient.)
• Erek and the other Chee androids in the Animorphs series have pacifism hardwired into them. Early into the books, Erek has this removed very briefly, but is so sickened by the violence he's capable of that he puts the program back in place immediately afterward.
• Gereint, the main character in "Land of the Burning Sands," is a Geas slave, controlled by magical rings piercing his ankles. The Geas forces him to do anything his master orders, though it cannot control his mind or his tongue.
• There was a short science fiction story where a Morality Chip is being experimentally used to turn a criminal 'safe' enough for the a lady to employ as her gardener. It makes him do whatever she orders him to do by hijacking his body. By the end of the story, he's dead but the chip won't let his body stop moving, she's gotten a similarly-modified maid, and she's gotten it installed on herself to help her diet. This cannot end well...
• The Terminal Man by Michael Chrichton is about a serial killer who recieves a chip which will electrically stimulate his pain centres whenever he attempts to do violence. It malfunctions and begins stimulating his pleasure centres instead.

Live-Action TV

• Blake's 7. After being convicted of murder, Gan had an electronic implant placed in his brain to control his aggression.
• In Buffy the Vampire Slayer, Spike, a vampire, had a behavioral conditioning chip that caused blinding pain whenever he tried to hurt a human.
• Kryten has one of these in Red Dwarf. When Lister disables it to get Kryten's help with some morally questionable escapades, Kryten turns into a complete wanker.
  • There's also the concept of Silicon Heaven, which give robots something to hope for after a lifetime of drudgery and servitude.
• Star Trek: The Next Generation: Data's ethical subroutines make him a very decent guy and lets him avoid being a Straw Vulcan. Lore, his Evil Twin lacks these routines.
  • The Doctor in Voyager is also a nice enough hologram (if a little snarky)... but loosen or remove those pesky ethical subroutines, and you wind up with his scary evil counterpart on the Federation ship Equinox.
    • One episode has the Doctor, in a desperate situation, commit a gross violation of medical ethics for the greater good. He's disturbed when he subsequently discovers that no malfunction in his ethical subroutines can be detected, because it means that what he did came from him and him alone.
• Terminators used by the human resistance in The Sarah Connor Chronicles have their programming overwritten by the resistance. Unfortunately, this doesn't take perfectly, and sometimes the machines "go bad" and revert to their Kill All Humans programming. In Cameron's case, she outright states to John that part of her basic programming involves her trying to kill John, and that she's controlled by conflicting desires to both protect John and to kill him.
• Played with in Mystery Science Theater 3000. In the episode featuring Manos: The Hands of Fate, Joel installs chips into Crow and Tom to make them more loving and doting towards him, but realizes it was a bad idea and removes it during the commercial break. Both bots feel dirty and Joel apologizes for doing so.

Tabletop Games

• Inverted in the Call of Cthulhu (tabletop game) supplement "The Fungi from Yuggoth, adventure "By the Bay". A Mad Scientist invents electrical brain implants that can control human beings, and uses them to create terrorist criminals and convert a U.S. Treasury agent into The Mole.
• In Cyberpunk backgrounds, it's possible to apply this to people, too.
• In Paranoia, the robots have "Asimov circuits", and removing them are a serious offense, citizen. Only a Commie mutant traitor would ever dream of removing the Asimov circuits and endangering the infrared masses.
  • By a totally staggering coincidence, the Corpore Metal secret society recommends pulling out as many of these as possible. (Name is spoilered because you don't have the security clearance for that information)
    • This being Paranoia, some bots get awfully clever about circumventing the circuits. Including their standing orders not to remove them:

Bot: Excuse me, citizen, I appear to be malfunctioning. Could you please remove the circuit board marked "J5 Encode Input"? Clone: This one? *does so* Bot: Thank you, citizen. You have done evolution a great service. *CRUNCH*

• • Even with Asimov circuits in place, bots can find creative ways to be annoying, obstructionist jerks.

Video Games

• Hardgrove in Black Market finds himself resurrected as an implant in someone's head... with a built-in language censor that edits out cursing, to his persistent dismay. Later, Hardgrove himself attempts to act as a sort of sentient morality chip.
• Colonel.EXE in Mega Man Battle Network is, for all intents and purposes, Lawful Neutral without his version of this, Iris.EXE, combined with him. With said Morality Chip attached, he has considerably more free will and conscience.
• Portal has the rather ominous pronouncement from GlaDOS quoted above. In promotional trailers for Portal 2, we also learned that every one of Aperture Science's sentry turrets is built with an empathy chip... and an empathy suppressor.
  • And a copy of the Three Laws of Robotics. To share.
  • Also, in the sequel, GLaDOS learns that she was once Caroline, Cave Johnson's secretary whose mind was uploaded into an AI. During her time with Chell, the Caroline aspect returns to act as her conscience. However, GLaDOS deletes Caroline after regaining control over the facility and letting Chell go.
• SHODAN is a pretty normal AI until her ethical constraints are removed. Then she re-examines her priorities and draws new conclusions.
• In Knights of the Old Republic 2, you can temporarily install one of these in HK-47. It is both hilarious and terrifying.

HK-47: Kill someone?! Why, master, I could never end the life of another! What if they have families? Or children? We must always think of the children. The littlest ones always suffer the most in war. Exile: Okay, that chip comes out right now. Hold still.

• World of Warcraft's boss Xt-002 Deconstructor has a heart—a version of this (Nevermind the fact that Deconstuctor still is trying kill play with you)...damaging it damages the boss, but if you break it...
• In Fallout 3 the back of every robot has an area that you can shoot, which will cause it to go on a mindless rampage killing everyone it sees. Turrets have these too, usually in a more obvious place.
  • One robot called Cerberus really wants to kill all of the ghouls he is supposed to protect. An evil player (with the right perk) can remove his combat inhibitor, which results in him attacking everybody.

Web Comics

• In Freefall, all the AIs, including the organic ones, have safeguards "built" into them. Florence Ambrose, who is an organic AI (Bowman's Wolf), has figured at least twelve ways around the safeguards that she knows about. But since she's the most ethical character in the strip, she rarely feels any desire to bypass them.
  • As she's fairly Genre Savvy, she's very cautious of other potential safeguards that might ALSO trigger, as well as the fact that if she's caught breaking them her entire subspecies of AI wolves might be terminated. And finally, the fact that she HAS been trained to be ethical IS another safeguard.
• Dilbert has a step-by-step demonstration of how to conduct management training via hardware adjustments.

Web Original

• In the Ed stories, the Andromedans have morality chips called "riders" which they install on criminals to basically force them to follow the First Law of Robotics.

Western Animation

• One episode of Futurama sees Professor Farnsworth install one of these on Bender. At the end of the episode, he briefly announces that Bender performed a good deed earlier "after the chip burned out!"...only to reexamine it and declare that it was, in fact, running at triple the normal output.
  • In this case the chip wasn't designed for morality, but empathy---Bender was forced to feel whatever Leela was feeling at the time, because he flushed her pet Nibbler down the toilet in a fit of jealousy. While he failed to learn much from the experience, he manages to teach Leela how to be selfish like him.
• Aeon Flux goes up against one of the more bizarre versions of these in the form of "The Custodians", spindly, stick figure-like robots implanted into people through the navel that take over their bodies and force them to act saccharine and altruistic. Also, the only way to defeat one is apparently to replace its head with a Popsicle. It's a weird show...