Some RPGs sit in an awkward spot where the game wants you to be telling a story, but they also want to be a tactical combat game. Because it is a tactical combat game, there are abilities that can lock a character down for one or multiple rounds. How the lock down happens is different depending on the ability, but whether it is a stun, being locked in a box, being knocked unconscious despite HP level, or anything else the effect is more or less the same: one character loses the ability to act or do other things.
Now these abilities are fine and fair to use. They're part of the game. They're in the arsenal for PCs to use, and it makes sense that NPCs would have them at their disposal as well. However, there is one big difference between a PC casting a spell like Banishment, and an NPC casting the spell on a PC: one removes a character from the game. The other removes a player from the game.
Hard CC (stuns, banish, etc) on some or all the monsters in a fight just speeds up the game. The PCs get free hits. They get to position themselves how they want. They can setup and plan. You can break combat time for a bit and just tell folks how long they have and do it. Or, if it is just one of several enemies that enemy just doesn't get turns.
That same thing to a player means their character is gone. They can do nothing on their turn. And considering in a big fight one round can be 30-60 minutes of real time, and you can effectively end up with someone in a position where they may as well have not even come to the game because they sure as hell aren't playing.
Again, this doesn't make it an unfair ability to use. However, as a GM you do need to keep it in consideration. At least if someone isn't in the scene and it is RP there is entertaining story going on. In a fight it is a lot of waiting while rolls are calculated, moves plotted, and plans executed. All while knowing you never get a turn.
Beyond that, it also starts the death spiral. One PC is now out of the fight. The enemies are not. The dynamic of the encounter just shifted by 1 body, and while the ability to hard cc is part of that challenge it doesn't change the fact that action economy a lone may cause problems for you.
Don't be afraid to use these abilities. But do be aware of the effect, both in game and out of game. The point of the game is to have fun. You know what your group has fun doing. And if someone's not going to have fun sitting and watching for 5-6 rounds of heavy combat...maybe give the enemy a different ability, one that is a bit more interactive.