Boost and Setback dice are a way for the GM to represent environmental conditions or other miscellaneous things that influence the difficulty of a check. Essentially it separates the difficulty from the circumstances of a situation. For example, in D&D a lock might have a base difficulty of 15 to be opened, and that difficulty could be increased to 20 if the thief had to pick the lock with improvised tools, at night, in the rain. Why? Because those three things make the check harder, and that is represented with difficulty. This is not how it is handled in FFG's Star Wars system.
There, the lock is always either an easy, fair, hard, daunting, or impossible difficulty. The GM would then assign setback dice for the fact that the thief is using improvised tools, the low visibility of night, and the rain - probably one for each. The player would then roll the dice pool of his skill dice + his ability dice vs. the difficulty dice + the setback dice and see how many successes/advantages/triumphs he was left with after the roll.
Now, the GM can do the same in reverse too. They can assign boost dice for advantages the thief has. Perhaps they had studied that exact lock recently, had taken the time to prepare a specialized bump-key for the lock, and had been warned about the trick tumbler. Again, one for each of these would be assigned.
Part of the Fun Is LOTS of Dice
Now one of the fun rules is that when X boost dice and Y setback dice are assigned they do not cancel out. Instead, you roll them all and see what comes of it. This is awesome for a couple of reasons. One, it lets the player roll a lot of dice, and rolling a lot of dice is generally a fun thing to do. Second, since each dice is there for a reason you can tell exactly what aspect of the check was responsible for the success. By this I mean that if the player only won the roll because one of the boost dice came up with a good result then you know one aspect of the player's prep is the reason that they managed to get past the lock. The same is true for the skill, ability, and other dice as well. Each die is there for some reason and the system then lets you see which parts help and which do nothing.
The Fun for the GM
On the GM side of things I love these dice more than the others because they let me go nuts with combat. I can add narrative flairs to the fight, represent them mechanically, and ultimately it doesn't mean anything because I'm not actually changing much in the probability.
So let's say I have a PC in melee with an NPC. The NPC lands a hit and gets a couple of advantages on the roll. In other systems...well...there are no advantages, but in this system I say that the NPC hits the player with a crate, which shatters dumping its contents on the floor. This makes the terrain uneven giving the PC a setback die and dazes him. However, the NPC is no longer armed and using crates is hard so he gets a boost as the NPC is off balance.
Outside of combat it lets me add other bits of flair. The lock is sticky, but the corridor is empty so you feel safe. Now the PC has a setback and a boost die which creates more opportunities for other extras to happen.
Couldn't You Do It Anyhow?
Now the thing is a lot of this could be done anyhow. But doing so in other systems makes things feel a bit clunkier. Why? Because you are bringing mechanical things into play somewhat arbitrarily, and in the end if you add one of each it ultimately doesn't matter because numerical bonuses cancel out. So who cares if you get a +2 because you have good tools, but -2 because it is dark, because that fundamentally works out to a +0 to the roll. With boost and setback dice though, that's 2 more dice to the pool and you can see exactly how much impact it has on the roll's outcome.
It's definitely something that has grown on me a lot. To the point that the joy found in the system is worth the price of the specialty dice, and I wish more systems had something similar.