Some sessions you know what is going to happen next. The players left off about to start the climactic battle, or you left with them entering a city for plot items and quest progress. Maybe you're just starting something new. The point is, you have a good idea of at least what the majority of the session is going to be. Sometimes though, you don't have a clue what is going to happen. When walking into those sessions it can be near impossible to prepare properly. After all, the game could go anywhere. Instead, you need to prepare to improv.
Preparing to Improv
Preparing to Improv is not all that different than normal game preparation. Every session has some improv in it, and so odds are your prep isn't nailing everything down as is. That said, the big difference between preparing to improv is that we're a lot more focused on breadth and not depth. We want a lot of things lightly sketched, not a few things with some depth to dig into.
First, What Is Around The Players
The first thing to go over is what is around the players. What areas are they near, or could they get to quickly? In fantasy games this can be easy. If they're in a kingdom, odds are they can't easily leave the kingdom because of the realities of travel (at least in the earlier levels.) So you can make notes for yourself for the area the PCs are in, and the areas around where the PCs are.
Remember, the idea here is breadth not depth. You don't need a ton of history and nuance to what is going on, just a general idea of what could be going on, and key descriptive elements and themes you want to hit with the area.
Second, Lists of Names
You should have these anyhow, but make sure you have some lists of names, titles, and ranks/jobs on hand for quick NPC generation. If you're improving a session you can't have an innkeeper and his family ready to go, nor can you have the captain of the spaceship ready. You won't even know you need either until it is coming up, at which point you'll be grabbing a name and going from there. The D&D GM screen has a quick NPC generation chart on the left that is decent. The internet has a bunch of "random characteristic generators" you can use. Or you can just wing it.
Don't be afraid to start with stereotypes and characatures. You can add depth to the NPC later if they come up more often. Right now you literally just need a name and enough that they can interact with the PCs without completely revealing the dog behind the curtain.
Third, Some Events - Random or Otherwise
Third, you want to have some events, encounters, or other things that can happen in the session. This doesn't need to be super detailed, but just a sentence or two for what is going on. If you're in an improv situation but the PCs are dealing with a big problem (i.e. hunting down a pack of gnolls rampaging a country side like in my game) then have some events related to that. More of those than not, in fact. If there is no over-arcing plot concern, than just make a grab bag of good and bad things going on.
One of my favorites is having the PCs stumble onto a wedding. You can sometimes spin them up in the festivities, let them be generous rich PCs to the bride and groom...and, well, you can always attack the wedding to put a bunch of innocent people at risk and let the PCs go for thrilling heroics (or just loot a massacred wedding....)
Fourth, Combat Is A Good Way To Stall
If you absolutely need time to prepare for the direction the PCs are going - i.e. they hone in on a key plot things and start pursuing and you need more time to prep that proper - then be ready to throw combat and other things in the way as needed. Combat is a good way to stall for time as, well, it takes a long time. Other things can also slow PCs down, but traps and puzzles are harder to make up on the spot.
There's no shame in stalling for time if you need it. And one of the classic story telling tricks is "if you're not sure what to do, have the bad guys kick in the door." So employ that here. You can figure out the how and such after the session for that too.
The Trick to Improv
The trick to improv, especially at the table, is to have fun with it. Your players aren't going to know you're flying by the seat of your pants. Your players aren't going to care if you need to take 30 seconds or 5 minutes to figure out what is happening somewhere. Don't beat yourself up on the fact you're grasping at straws to make things work, that is part of how GMing works. Just try to have fun, because if you are having fun your players will pick up on that and have fun as well. And if everyone has fun? Well, that's a good session!