At the tables I've played at lately there has generally been a pool of communal dice. This is normally supplied by the host of the game, but enough of us have enough dice that we can loan them freely to the few without their dice. I specifically keep about 2 pounds of dice in a large tube near the table that are 'public' dice. You need dice? Grab a set or two of your choice from the tube. Just put them back once the game is over.
Basically, I've seen both extremes of treating dice. The groups where you didn't touch anyone else's dice and those where dice were just up for grabs on the table for everyone. Both groups worked fine - though the people who hoarded dice tended to be faster. I'm curious if that's bias or related to something else.
What about you though? Does your group have a strict dice policy? Do you have communal dice? Do people bring their own? If they bring their own, do they leave the sets in the house where game happens or do the dice travel?
Do you have a special set for your character?