Everything in Firefly is set up in an anti-authoritarian manner. In the Civil War analogy we side with the Independents and the Confederacy counterpart against Unity. Inside the ship, Mal is the captain but he doesn't rule the roost by a strict structure or enforcement of the maritime (or.. spacetime?) code of conduct. Instead the crew is held together by a strong sense of True Companions and Mal beating people with The Chains of Commanding. Sometimes its the heavy burden associated with the metaphorical chains of commanding, other times its real metal chains.
Just look at Mal himself, an Anti-Hero with the tagline "I aim to misbehave", forget helping the hopeless.
Sin is Essential to Humanity
In Serenity in particular, a major theme is that the Seven Deadly Sins are integral to humanity's nature. The Alliance's villainy is a result of them attempting to control people, force them to act against human nature in the name of eliminating sin. On Miranda, in attempting to create a "world without sin", they removed everything that made colonists human. They not only no longer felt any "sinful" emotions, they no longer felt anything at all, and could do nothing but just lay down and die. Or became Reavers.
You can't make people "better". You have to accept humanity as it is, sins and all.
Innocence is the cost of strength
Tying in with the "Sin is essential" idea above is this concept of the loss of innocence within the series. A big part of what makes Mal who and what he is happens to be the destruction of his relatively naive beliefs, especially with the crushing defeat at the end of the Battle of Serenity Valley. The loss of that innocence is what sets him free and makes him the man he is now. On the opposite end of the spectrum is a similar loss in River; her innocence is stripped from her in a brutal fashion similar to the loss Mal went through, and like Mal it left her damaged yet more powerful than she was when she went in. At the same time, she's a fragile, psychotic, emotional wreck in spite of her overwhelming physical and psychic power.
Both of these characters are strong physically, but they paid for it in terms of a loss of innocence; in Mal's case, the brutal destruction of his faith and ideals, and in River's case, the death of her childhood and mental stability.
- To add onto the above: A character's physical prowess can be correlated to how far gone their innocence is. On one far end of the spectrum, we have Wash, Simon, and Kaylee, who are just this side of being Wide Eyed Idealists at times (well, Kaylee is far on the other side of that most times...), and they are all generally weaker in confrontations. On the other end, we have Inara, Zoe, and Jayne, who are considerably less innocent, and far stronger as a result. Similarly, one can plot these characters on the Sliding Scale of Idealism Versus Cynicism, and see that the more cynical a character is, the better suited they are in a confrontation.
- Simon Took a Level in Badass, and the first thing he does is steal medicine from a hospital, in a sort of parallel to Mal's theft of medicine on a much smaller scale towards the beginning of the series, showing that he has slid towards the Cynical end of the scale.
- Book appears to be on the Idealistic end of the spectrum, but we gradually realize (and later have confirmed) that he is in fact far on the Cynical end, having taken his vows possibly as a means of atonement, enforcing a more Idealistic outlook on himself, thus leaving him far more formidable in a confrontation than one would expect. River, having suffered badly from Break the Cutie, constantly dances back and forth on this scale, due to her mental instability, and is thus both very formidable and very vulnerable. Serenity herself has been through many rough times, being a very old ship (but not as old as some that Book has traveled on in his day), and in fact had hit rock bottom when she joined Mal's crew, and proves to be one of the most formidable characters despite being completely unarmed.
The crew as parts of Mal
Inara has been described as Mal's heart, and we can see he gets in a darker, more callous mood whenever she's not on the ship. I submit that each of the characters fill a hole in his being left by Serenity Valley. As far as this editor can discern: Kaylee is his joy and happiness, Zoe his will to fight or to lead, Book his conscience. Simon, as hard as it may be to believe at first glance, is his inspiration and aspiration to great, heroic deeds. River is his curiosity and innocence. Wash I'm not so sure about, he may be Mal's basic motivation - the ability to take action and make decisions. Jayne is what we might politely call his lower urges. Wash may be his sense of humor, which isn't as lame as it sounds. And Serenity herself, she is his wings.
Joss Whedon mentioned at one point that he considers each member of the crew to be a part of Mal's personality that he lost at the Battle of Serenity Valley that he's trying to regain.
Which would definitely reflect his attitude in the movie well, in that Mal shows heartlessness when he throws the begging settler off the Mule during the Reaver attack and a distinct lack of faith in his own principles when Zoe mentions they never would have done this during the War, and he glibly comments "Maybe thats why we lost". Kaylee even mentions he's been acting this way ever since Inara (his heart) and Book (his faith) left Serenity.
Each of the couples on the ship are the opposite ends of some spectrum. Mal represents independence and freedom and, in both his resistance to government meddling and his 'go where the wind takes him' spirit, represents chaos and the frontiersman. Inara on the other hand comes from and misses the cultured life of the core planets and supported the unification of the Alliance; she is the least criminal of the crew, and furthermore she lives and works according to a traditional code of conduct, so she represents order and civilization. On the other hand, she essentially works as a prostitute, putting herself up for sale, so she doesn't live a life that is as honest or free as the one Mal leads, although it is more comfortable and glamorous. Kaylee, always covered in engine grease and with the bluntest way of speaking and maybe least formal education (if Jayne didn't take the crown for both) represents simplicity and unsophistication -and is always the happiest because of it. Simon, on the other hand, represents gentility and sophistication and science, and has a lot of trouble cutting loose. Wash is the comic relief. Zoey is serious and intense almost all the time. You can add to this a little bit more, even though they aren't a couple, but as each of the other members of the crew are paired off, Jayne and Book form and unlikely close friendship - Jayne is the least morally fettered of the crew, and Book the most moral. Each of these pairs belong together in a way because they complement each other, and many times throughout the series it is made clear that they are attracted to one another precisely for the traits that polarize them.
As an addendum to the above, this may be why the River/Jayne pairing is so popular in the fandom. Jayne is rational, crude, hefty, powerfully built, not particularly well-learned, and around Mal's age. River is small, lightweight, relatively refined, educated, intelligent, completely irrational, and in her late teens. Considering the degree by which the show already has characters who are opposites that complement each other becoming friends, it makes sense that this pairing would exist and be popular.
Why the name Serenity? Obviously, it was named for the battle, but why did the battle take place in Serenity Valley?
When Mal lost at Serenity Valley, he lost his serenity, in one sense, but gained it in another. He was no longer calm and content, but had gained a sort of resignation. As time goes on, his serene life on the ship is challenged by others who want to take Serenity. His least serene moment in the entire series—the moment in the film where he threatens to kill the crew—happens when they disguise the ship, so he no longer has Serenity. When the ship is restored, his serenity is restores. And now serenity doesn't look like a real word anymore.