The armies clash. Rengar the Red leads the charge out of the front gates and into the fray. The air fills with the clash of steel on steel and the stink of sweat, skin, leather, piss and death. Amid the melee Rengar cuts through warrior after warrior on his way to the enemy standard and the enemy general that awaits behind it. As Rengar's steed hits the last defensive line a lone arrow streaks from his own army and takes Rengar the Red down from behind....
The scene above, and others like it, are fairly common across the scope of fiction and not uncommon in historical accounts of war and important events. Betrayal, the word or blade of a traitor, has always been among an adversaries most powerful tools. Because of this it is not unheard of for GMs to want to use betrayal as a story point in their game. However, unlike many other story elements that can be used with little preparation or just on a whim, Betrayal (yes, capital b) is something that has to be handled with care.
Why Betrayal Works
Betrayal works so well because it is basically turning the enemy against themselves. How can you hope to win a fight when your own hand is stabbing you in the chest with a knife? How hard is it to defend a castle when a gate is opened for the enemy's choice of their own troops? How challenging is it to get a matter through the courts before the king when your opposition is forewarned of every move you are planning to make? Betrayal works like this, but there really is a lot more to it as well.
Defending Against Betrayal
The two defenses against betrayal are Loyalty and Vigilance.
Loyalty (instilled in your troops, supporters, etc) defends against betrayal by making those who are approached to betray you less likely to accept. Ultimately this means that the strength of their loyalty is greater than the force trying to turn them into a traitor. Loyalty can work in small groups, but the larger one's allied force gets the more likely betrayal is to become possible. After all, how many people are going to choose you over their family and loved ones, or enough money to live comfortably for the rest of their lives?
Vigilance on the other hand doesn't require trust, but it does have its own cost. The more you check, double check, and guard against betrayal the less you can get done. Think of it in terms of an army: if you have to triple check the catapaults for sabotage every night what are you losing the chance to do in return? More to the point is that excessive vigilance quickly becomes confused with paranoia and once you are seen as paranoid people begin to throw around words like "crazy" and "unhinged." In other words, the more you stress vigilance, the more you strain trust and why should someone be loyal to you when it is clear you don't trust them?
Once Bitten, Twice Shy
Betrayal works by either defeating the opponent with some aspect of him or her self in one move, or by wearing away at the enemy who has to guard against pending betrayal. Now we get into the problematic parts of betrayal for a game, and it can be boiled down into one statement: once bitten, twice shy.
The thing with betrayal is you only get to do it once before your players will start to look for it everywhere. The problem here is that it is the players not the characters who are paranoid. Some of it may bleed over, sure, but you won't get the same impact on subsequent betrayals if you don't do your work and set up ahead of time.
Beyond that you need to use betrayal in arcs very carefully and with plenty of space between them. If the players get betrayed too often they will begin to wonder what the point of trying to even get to know the NPCs are - after all, anyone they meet is just going to betray them later.
How To Handle It...
The trick to handling a Betrayal story arc is to know going into it what you are going to do. Figure out the how, why, when, and where of a betrayal before you even start the plot. Is someone giving information, opening a gate for the enemy army, or thrusting a knife in to the PC's back? Why is the traitor doing this in the first place? Presumably they joined up with the PC as friends and allies, so why turn their back on the PCs now? What was it that made them switch sides, and just whose side are they switching to with their betrayal?
For example, in an L5R game it could be that the players are going to get betrayed because they've raised the ire of a certain Scorpion Lord. That Scorpion Lord has sent an agent to kill the PCs, or at least send a message. To do this the agent has given a gift of coin to the servant of an ally of the PCs. On a certain night, while the PCs sleep, the servant is going to let the Scorpion Agent into the castle unmolested by guards, and the Agent will get a shot at the PCs while their guard is down.
This is a fairly minor betrayal. In fact, this is the kind of thing I might start a plot line off with in one of my games, waking the PC up (automatically or with a die roll) before the assassin struck, and then letting their investigation discover the betrayal, the why, and the who to try and end the conflict. I would also have the reactions of the people around planned out. If nothing else the Hosting Lord has been embarrassed and should act in a way to help show the PCs he was not involved.
Even with that, I'd be surprised for the PCs to not have a watch while they slept as a group for a while, even in the homes of people they trust. Why? Because the players will always remember that one time trusting the house guard was used against them.
Betrayal is a powerful tool that can ratchet up the tension in a story and push a good arc into becoming a great arc. However, it has to be handled with care. Betrayal tends to have a lot of meta implications and even if your players don't realize that they've been affected you can see negative changes like disassociation from NPCs if you don't handle it carefully.
In the end I'd recommend against using betrayal if at all possible, and using it sparingly when it is impossible to not use it. Whichever way you go, just make sure you're both careful and prepared when you opt to use this kind of plot.